Tag Archives: MODBUS

IoT versus SCADA/DCS Data Acquisition Patterns

Introduction

The term Internet of Things (IoT) is used in a variety of contexts where it is often misunderstood, because it can be replaced by other terms much better describing the matter we deal with or the definitions are not compliant with each other. Let me remind you about the very beginning of this term life.

That ‘Internet of Things’ Thing

The phrase “Internet of Things” started its life as the title of a presentation made in 1999 and aimed at explaining a new idea of radio frequency identification (RFID) in the context of the supply chain performance. It is clear that it doesn’t mean that someone has any right to control how others use the phrase, but my point is that a precise term definition is important for working together on: common rules, architecture, solutions, requirements, capabilities, limitations, etc. In practice having a common definition it is possible to check a selected technology, solution or product capabilities against requirements of the application entitled to use this term.

The main goal of this article is to contribute to the community work aimed to distinguish the IoT applications domain features. The main challenge faced up is to narrow the definition to make it unambiguous and meaningful.

In most publications I know the term IoT can be simply replaced with the following well-known terms:

  • SCADA – Supervisory Control and Data Acquisition
  • DCS – Distributed Control Systems

and the text still will be perfectly OK. In this context the “sensor” term plays the role of the “thing” and the nodes network is a synonym of the Internet. No one usually takes care whether we are talking about the temperature sensor in a bedroom or in a boiler drum in a power plant except that in case of a power plant sometimes the prefix Industrial is added. To make our life easier let’s forget about the “I” (Industrial) prefix at all because it doesn’t change the most important application domain features.

To illustrate the further discussion let me provide examples that can be recognized as SCADA/DCS and IoT applications respectively.

SCADA/DCS example

Let’s assume that an OPC UA Server exposes 123456 values representing the crude oil refining process. Using SCADA on top of this server we can monitor and manually control the process. Using DCS it is possible to implement a supervisory control algorithm to provide macro optimization.

For this scenario we can apply the following work-flow:

  • The server instantiates an OPC UA Information Model for the crude oil refining process.
  • All plant floor devices equipped with sensors are fetching data representing the current process state (for example the flow meter #A-4321 supporting the Modbus RTU communication protocol) and are waiting for data requests coming from the server communication engine.
  • The communication engine embedded in the server polls all plant floor devices including the flow meter #A-4321 to recover the current process state.
  • Finally, the OPC UA Server exposes the data (updates value attributes of the relevant variable nodes, e.g. #A-4321 object representing the virtual flow meter #A-4321) in his Address Space Management component (i.e. in the Address Space instantiated according to the Information Model of the crude oil refining process).
  • OPC UA Clients connected to this server are updated in a standardized way.

Note in this scenario that the OPC UA Client and OPC UA Server can establish connection over the Internet using any existing transport protocol, e.g. HTTP, HTTPS, TCP, UDP, AMQP. Selection of the transport protocol between the client and server is negotiated and limited by the OPC UA specification. To get more read OPC Unified Architecture – Main Technological Features.

You can use the server I have exposed many years ago to test this scenario. The instruction how to connect is included in the following document:

http://www.commsvr.com/Products/OPCUA/OPCUAServers/OPCUAADIServer.aspx

IoT example

Do you think a box (or even pack) of cigarettes could be the “thing”. It has a bar-code, so it is the source of data. Is it the sensor – NO because the bar-code reader (industrial scanner) is the sensor in this case. Can we recognize the bar-code reader as the “thing” – again the answer is NO if the goal is to provide a GLOBAL cigarettes tracking system. The same applies to drugs for example. Is it an IoT solution – my answer is YES, no doubts. Is it SCADA/DCS –the answer is NO, because the server (undelaying communication engine) cannot poll all possible places spread over the world where the box could appear. There are two reasons:

  • It is impractical or even impossible to manage such a huge set of addresses.
  • The server doesn’t know when to poll because the relevant data appear as an event instead of a process state value.

Assuming that the server is interested or even allowed to collect product data of one vendor only – not all codes fetched by any bar-code reader are relevant to the server.

Is the “thing” smart – I don’t think we can call the bar-code something smart. Is it controllable – NO.

The most interesting observation is that we can recognize this use case as an IoT application, but we have not mentioned OPC, AMQP, MQTT, SOA, Internet, WI-FI, wireless, Modbus, etc. at all, but only that we have important mobile data and the solution is globally scoped. It is good because we can check the available technology capabilities against this application requirements. As I have said selected communication technology is not the goal, but we must know how it scales to applications like this.

Now, let’s replace the word GLOBAL by LOCAL (for example cash desks farm in a shop) and the same application is no longer an IoT deployment, isn’t it? It is true even if the cash desks are interconnected using the IP protocol (e.g. Intranet)!

Sometimes it is stressed that any IoT application must guarantee a high level of robustness, but importance of the sensor and data robustness requirement is applicable to many kind of applications, e.g. controlling an airplane engine during flight. The same engine could be monitored and tracked after landing in any airport using the local WI-FI by uploading archival data to a central advanced analytic system (like the cigarettes box bar-code). Is it IoT? It isn’t during the flight, but the solution is life sensitive. After landing it is IoT, but the reliability of the data and data transfer is not so important, isn’t it?

IoT Paradigms

My proposal of the Internet of Things definition is as follows:

Internet of Things is all about:

  • mobile data fetching – how to gather the data from mobile devices (things);
  • mobile data subscription – how to transfer the data over the Internet to a place where it could be processed;
  • mobile data processing – how to integrate the data into a selected application to improve process behavioral performance.

Data fetching is related to a variety of last mile communication technologies, for example RFID, WI-FI, VHF, Bluetooth, etc. Subscription could be supported using messaging systems, e.g. AMQP, MQTT, etc. A good candidate for leveraging data consumption is for example OPC Unified Architecture.

Referring to previous examples, the data fetching process looks very similar in both cases – we have a data source and a sensor coupled together at some point in time responsible for sampling the data. Analyzing the examples from the application functionality point of view we cannot compare them because there are no requirements defined at all – only very general descriptions are provided. It looks like application capabilities are not relevant to the term definition. It is the reason why the terms SCADA, DCS and IoT are used interchangeably neglecting the fundamental differences between the following data acquisition patterns:

  • Data polling–continuous checking of the sensors to see what state they are in, usually in multipoint or multidrop communication (a communication engine with multiple devices attached that share the same line) by sending a message to each device, one at a time, asking each to respond and send new data.
  • Data subscription– senders of messages containing the process data fetched by the sensor, called publishers, do not prepare the messages to be sent directly to specific receivers, called subscribers, but instead they categorize published messages into topics without knowledge of which subscribers, if any, may receive the message. Similarly, subscribers express interest in one or more topics and only receive messages that are of interest, without knowledge of which publishers, if any, there are.

It is worth stressing that in both cases reusability of the fetched data is assured. In data polling scenario the server coupled with the communication engine may be connected to by many clients at the same time. In data subscription scenario the publisher is responsible for multicasting the data to all attached subscribers directly or indirectly using a broker.

To deploy the IoT scenario:

  • the mobile data must be sent over the Internet (or Intranet) using messages;
  • the payload of these messages is consumed asynchronously by a server (e.g. OPC UA Server) responsible for exposing it in an address space;
  • the applications (e.g. OPC UA client) process the exposed data to reach selected key performance indicators (KPI).

It is required that the messages payload is formatted in a standardized way to be factored on the fetching site and meaningfully consumed by the analytic application (e.g. OPC UA client).

Data Acquisition patterns

In the above discussion the application functionality has been excluded as a factor, which can be used to recognize IoT applications. Now let’s analyze the impact of the data acquisition pattern on the application behavioral model.

Using data polling we must deal with the synchronous data acquisition pattern. In this case the application must follow the interactive behavioral model, because it actively polls the data source for more information by pulling data from a sequence that represents the process state in time. Such behavior is represented by an iterator, which is used to iterate through a data stream. The application is active in the data retrieval process – it controls the pace of the retrieval by sending the request messages at its own convenience. This enumeration pattern is synchronous, which means that the application might be blocked while polling the data source. Such polling pattern is similar to visiting the books shop and checking out a book. After you are done with the book, you pay another visit to check out another one. If the book is not available you must wait, but you may read what you selected.

On the other hand, in the reactive behavioral model, the application is offered more information by subscribing to a data stream, and any update is handed to it from the source. The application is passive in the data retrieval process: apart from subscribing to the source data stream, it does not actively poll the source, but merely react to the data being pushed to it. In this case, the application will not be blocked by waiting for the source to update. This is the push pattern employed by IoT. It is similar to joining a books club in which you register your interest in a particular genre, and books that match your interest are automatically sent to you as they are published. You do not need to wait but you must read what you get. Employing a push pattern is helpful in many scenarios, especially if data is available asynchronously as events.

The push model implemented by IoT requires additional resources responsible for multicasting the pushed data to all subscribers. This functionality may be accomplished by a middleware fulfilling the broker role or supported by the network infrastructure, e.g. IP multicast.

The fundamental differences between the interactive and reactive behavioral model must obviously impact the final application functionality, for example:

  • process controllability;
  • data destination discoverability;
  • maintainability.

Using data polling the request message may also contain data to control the state of a selected actuator. In this case the response message usually contains positive or negative acknowledge that can be used by the application as a condition selecting further activity. For example in case of communication disruption the request message may be resent. In case of actuator failure an alarm may be raised. In case of the pushing data scenario it is hard to implement remote control functionality in a similar way because the communication path is like a one-way route.

 

In interactive behavioral model the communication engine must have all information including addressing in advance to properly prepare request messages. The messages must be self-contained to be used by the network routing mechanism. In case of reactive behavioral model the application doesn’t know the source of data in advance. Therefore the sensors responsibility is to format the message and push it to an appropriate distribution channel. In this case the messages are not self-contained, because the information carried by them is only indirectly used by the routing mechanism.

For the polling data scenario configuration modification may be required after replacing the sensor if the data sources are not isomorphic for the data acquisition process. On the other hand the pushing data scenario requires that any replacement or modification of the data source must not need any modification of the application configuration.

Conclusion

From the above discussion we can derive the following dependency models:

Interactive applications (e.g. SCADA, DCS):
  • The application engine depends on the synchronous data acquisition engine – data is observed as a stream of entities.
  • The acquisition engine depends on the process devices (sensors or actuators), which it must know in advance.
Reactive applications (e.g. IoT):
  • The application engine depends on the asynchronous data acquisition engine – data is observed as a stream of events.
  • The data source depends on the data distribution channel.
  • The asynchronous acquisition engine depends on the distribution channel.
  • The data source and acquisition engine are associated by the data distribution channel proprietary mechanism.

There is a well suited open source project to start prototyping with this scenario.

https://github.com/mpostol/OPC-UA-OOI

Advertisements

OPC UA Makes Process Observer Archetype Possible

Integration

Usually modern manufacturing automation systems consist of numerous different IT systems located at business management/operation and process control levels. It is a broad class of application domains where business IT and control systems are converged to make a large whole with the aim of improving performance as a result of the macro optimization and synergy effect. This domain is called Industrial IT. Frequently the systems are distributed geographically among multi-division organizations.

To deploy the above-mentioned convergence the systems have to be integrated – they must interoperate. After integration the systems should make up a consistent system, i.e. each subsystem (as a component) must communicate with the others. The final information architecture is strongly dependent on organization, culture, type of technology and target industrial process. Communication is necessary for exchanging data for production state analysis, operation actions scheduling, supervisory control and task synchronization in the process as a large whole.

To make up a consistent system as an ultimate result of the integration process the following architectures can be applied:

  • Peer to peer: manually created point-to-point links to meet short-term ad hoc objectives.
  • All in one: a product dedicated to both functions: process control and business management.
  • Process Observer: a consistent, homogenous real-time representation of the process control layer.
Process Observer

Fig. 1 Process Observer Archetype

Process Observer (Fig. 1) is a kind of a virtual layer, which is a “big picture” of the underlying process layer composed of unit data randomly accessible by means of a unified and standardized interface. It allows the process and business management systems, using international standards of data exchange to share data from plant floor devices. Process Observer is like a bridge connection between the plant-floor control and the process and business management levels.

Thereby, the structure of the links becomes systematic and the existing functionality of the upper layers is preserved. Using the Process Observer archetype the number of links between components can be substantially reduced and, what is very important it is a linear function of the number of nodes.

Now, the links can be used to gather the process data in a unified, standardized way (see fig.2).

Process Observer archetype greatly reduces the whole complexity and decreases interdependence by decoupling application associations and underlying communication routes. Additionally, it allows applying systematic design methodology and building information architecture independently of the underlying communication infrastructure.

Process Observer Deployment

Implementation

The Process Observer concept has been implemented in the CommServer™ software family. That communication server is optimized for applications in distributed process control systems. To provide a consistent sole representation of a distributed real-time process at the upper layer boundary – according to the model – the CommServer™ has to implement unique functionality, provide redundancy and optimize utilization of underlying communication infrastructure.

CommServer-Process Observer Implementation

Fig 2. CommServer-Process Observer Implementation

Functionality

Communication

To meet scalability and open connectivity requirements, the CommServer™ exposes the OPC Unified Architecture (OPC UA) to be consumed by upper layer applications. One of the main objectives of using the OPC UA is to provide a uniform bridge between digital plant-floor devices and systems providing services at the process and business management level. At the very beginning, this bridge was invented as a translator between vendor specific languages (protocols) used by the devices for data access and a widely accepted one – OPC. Therefore, each OPC UA server has to be equipped with a vendor specific component called DataProvider that implements selected protocol and communication infrastructure management functions. Popularity of the OPC UA standard grows, but still many applications do not support it. For that reason, another member of the family, DataPorter™ offers SQL and XML connectivity (Fig 2).

Process Simulation

CommServer™ does not only play the role of a translator and communication engine. Offering the possibility of creating simulators and publishing simulation data in the same way as the process data, the final process representation can be complemented by directly unavailable information obtained by processing current and historical values. To commence factory approval tests of any system, we need to build a testing environment. Using simulators instead of communication drivers, it is possible to seamlessly switch between production and test environments reducing the cost by order of magnitude.

Resource Monitoring

In a production environment, monitoring and management of the recourses that make up the information processing and communication infrastructure is often of the same importance as access to the real time process data. CommServer™ allows for publishing data gathered from the active network devices in the same way as the process data.

Server to Server Interactions

It is a scenario using interactions in which one Server acts as a Client of another Server. In the presented architecture it is implemented using a dedicated OPC Classic or OPC UA DataProvider. Server to Server interactions allow for the development of servers that: exchange data with each other on a peer -to-peer or vertical hierarchy basis to offer redundancy, aggregation, concentration or layered data access management.

3 levels of redundancy

Using the Process Observer archetype with only one common component responsible for interconnecting plant floor devices and process and business managements systems creates a single point of failure. To overcome it and eliminate the risk the proposed solution offers tree levels of redundancy to increase availability. They can be applied independently according to an appropriate analysis and assessment of the risk.

  • Hardware: To provide true fault tolerant systems redundant hardware can be used. This solution provides the same processing capacity after a failure as before. We have two options: boxes and components redundancy. The first one is achieved by using a primary server and a backup server. We can also use fault-tolerant hardware designed from the ground by building multiples of all critical components, such as CPUs, memories, disks and power supplies into the same computer in order to ensure reliability. In the event one component fails, another takes over the communication without skipping a beat. The fact of switching from one server to the other should be transparent for the clients.
  • Communication paths: To increase availability the CommServer™ assures redundancy of “data transmission paths”. It is designed to recover from a communication path failure by detecting the failed route and switching to another one if available. Paths redundancy improves robustness, because the same remote unit can be reached using different physical layers to eliminate single point failure dependency. The server is responsible for the selection of a route to transfer the data and to control availability of inactive paths. Duplication of the communication paths may be costly, because data transfer over distributed networks is usually not for free. The crucial feature of paths redundancy is the provision of the path multiplication without the necessity of transferring the same data over the network many times and controlling backup path availability at the same time.
  • Signals: For reliability, this feature allows to define replicated signals. Thus, if one signal fails, a second one is available as one OPC tag. To determine whether a fault has occurred (fault detection) and which one signal is affected (fault isolation) two methods are available: source and statistical ones. Source detection relays on information about signal quality received from a plat floor device. Statistical methods use the confidence level as an interval estimate of a population parameter.

Optimal communication

Engaging of an intermediate component as a driver for plant-floor devices is a middleware archetype used worldwide in thousands of applications.
But to provide a consistent sole representation of a distributed real-time process at the upper layer boundary – according to the model – the CommServer™ has to implement
unique features optimizing utilization of the underlying communication infrastructure:

  • Multi-Protocol Capability: many protocols can be implemented as DataProvider components and plugged-in and utilized simultaneously;
  • Multi-Medium Capability: any physical layer technology can be used to start building a communication stack;
  • Multi-Channel Connectivity: numerous independent communication routes can be activated simultaneously to gather raw process data;
  • Adaptive Retry Algorithm: each protocol retries to acquire data after a communication error, but adapting the number of retries to current conditions allows to increase greatly the whole bandwidth;
  • Adaptive Sampling Algorithm: is responsible for adjusting the plant floor devices sampling rate according to the current process state;
  • Optimal Transfer Algorithm: is responsible for minimizing the difference between the individual process data update rate as required by clients and the current sampling rate of process control units.

Related articles

OPC UA Makes Complex Data Processing Possible

From the definition, the Industrial IT domain is an integrated set of ICT systems. System integration means the necessity of the information exchange between them (the nodes of a common domain). ICT systems are recognized as a typical measure of processing information. The main challenge of deploying an Industrial IT solution is that information is abstract – it is knowledge describing a situation in the selected environment, e.g. temperature in a boiler, a car speed, an account balance, etc. Unfortunately machines cannot be used to process abstraction. It is also impossible to transfer abstraction from one place to another.

Fortunately, there is a very simple solution to address that impossibility, namely the information must be represented as binary data. In consequence, we can usually use both ones as interchangeable terms while talking about ICT systems. Unfortunately, these terms must be distinguished in the context of further discussion on the complex data, because before stepping forward we must be aware of the fact that the same information could have many different but equivalent representations – different binary patterns. For example, having interconnected system A and system B, system A can use one representation, but system B another one. Moreover, to integrate them, the transferred stream of bits may not resemble any of the previous ones. It should be nothing new for us, as it is obvious that the same information written as a text in regional newspapers in English, German, Polish, etc. does not resemble one another.

To understand a newspaper we must learn the appropriate language. To understand the binary data we must have defined a data type – a description how to create an appropriate bits pattern. Simplifying, the data type determines a set of valid values and rules needed to assign the information (understand the data) to a selected bits pattern. Therefore, to make two systems interoperable, apart from communication, they should be prepared – integrated to be able to consume data from each other, and so communication is only a prerequisite for interoperability.

The type is usually not enough to make the data meaningful. Referring to the above example the newspaper name (i.e. the location where the information came from) and timestamp (a single point in time when the information was valid) are attributes of the text that is representation of the information.

To have a similar ability to add common attributes to the representations of many information entities at the same time the complex data types must be used. Complex in this context means that the data type must additionally define a relationship between the components of the binary data, i.e. how to selectively get a component of the complex data.

The OPC UA offers two well-known and widely used relationships:

  • Arrays – components are indexed and all components must have a common data type.
  • Structures – components are named and components may have different data types.

Anyway, indexes and names must be unambiguous, and a complex data type has the responsibility to provide a precise definition of them, i.e. selectors of the components.

The complex data has a very important feature, namely all components are considered to be consistent with one another. For example, if we need to represent time at least three components must be distinguished: hour, minute, and second. In this case, even if there is no need to add any attribute to the binary data it must be consistent, i.e. it has to represent information in a single point in time. Other criteria for describing the data consistency could also be applied.

On the other hand using complex data simplifies data integrity if there is a need to store or transfer it. If intermediaries are present, the initial data creator and the ultimate consumer need to trust those intermediaries to help provide end-to-end data integrity, because each hop is processed separately. Thus, using complex data it can be processed and transferred as one item what finally mitigates any risk of integrity compromising.

Using the data type definitions to describe information exposed by a server allows:

  • Development against type definition.
  • Unambiguous association of the information with the data.

Having defined types in advance, clients may provide dedicated functionality, e.g. displaying the information in the context of specific graphics.

Typical scenarios can be recognized when we can define appropriate complex data types in advance. The OPC UA offers a variety of standard types ready to be used in common cases. If this out of the box set is not capable of fulfilling more demanding needs users may define custom data types. The OPC UA allows servers to provide data type definitions. The type definitions may be abstract, and may be inherited by new types to reflect polymorphism. They may be of generic use or they may be application domain specific. Custom types must have a globally unique identifier, which can be used to identify the authoring organization responsible for that type definition.

If the data publisher – an OPC UA server is not running in an environment capable of creating the complex data there must be taken special precaution to fabricate it if required. An example of this scenario is a standalone OPC UA server pooling data from plant floor devices using a custom protocol, e.g. MODBUS. If that is the case the protocol used to gather process data is usually not data complex aware. Reading and writing the data is accomplished using REQUEST/RESPONSE frame pairs. Moreover, one request can be used to read a set of values that has the same simple type only. The fabrication is an operation that uses group of requests to gather components and embeds them into a single value of a selected complex data type. It is optional server capability.

Related articles

Continue reading