Dynamic Data Acquisition
Main.DataAcquisition History
Hide minor edits - Show changes to markup
See the Data-Driven Engineering course for additional information on SQL in Python.
- OPC Client for Python (OpenOPC)
This example code Show OPC Example Code writes to an OPC Server with an OPC Client with OpenOPC. The code writes 16 values to 16 individual OPC tags on a Kepware Server.
(:toggle hide opc button show="Show OPC Example Code":)
- OPC DA Client for Python (OpenOPC)
- OPC UA Server and Client for Python
This example code Show OPC DA Example Code writes to an OPC Server with an OPC DA Client with OpenOPC. The code writes 16 values to 16 individual OPC tags on a Kepware Server.
(:toggle hide opc button show="Show OPC DA Example Code":)
The newer OPC UA (Unified Architecture) is platform independent and does not rely on the DCOM communication. It is more secure and includes functionality that is in OPC DA (Data Access) and OPC HDA (Historical Data Access).
from pymodbus.server.asynchronous import StartTcpServer from pymodbus.server.asynchronous import StartUdpServer from pymodbus.server.asynchronous import StartSerialServer
from pymodbus.server import StartTcpServer
from pymodbus.transaction import (ModbusRtuFramer,
ModbusAsciiFramer, ModbusBinaryFramer)
- configure the service logging
import logging FORMAT = ('%(asctime)-15s %(threadName)-15s'
' %(levelname)-8s %(module)-15s:%(lineno)-8s %(message)s')
logging.basicConfig(format=FORMAT) log = logging.getLogger() log.setLevel(logging.ERROR) # logging.DEBUG
identity.MajorMinorRevision = '2.3.0'
identity.MajorMinorRevision = '3.0.2'
StartTcpServer(context, identity=identity, address=("127.0.0.1", 502))
StartTcpServer(context=context, host='localhost', identity=identity, address=("127.0.0.1", 502))
print('Modbus server started on localhost port 502')
from pymodbus.client.sync import ModbusTcpClient as ModbusClient
from pymodbus.client import ModbusTcpClient as ModbusClient
from pymodbus.client.sync import ModbusTcpClient as ModbusClient
import numpy as np
print('Start Modbus Client after Modbus Server is Running')
- initiate client
print('Start Modbus Client')
slave_address = 0
- initialize watchdog
wtd = 0
- builder.add_string('abcdefgh')
- builder.add_bits([0, 1, 0, 1, 1, 0, 1, 0])
- builder.add_8bit_int(-0x12)
- builder.add_8bit_uint(0x12)
- builder.add_16bit_int(-0x5678)
- builder.add_16bit_uint(0x1234)
- builder.add_32bit_int(-0x1234)
- builder.add_32bit_uint(0x12345678)
- builder.add_16bit_float(12.34)
- builder.add_16bit_float(-12.34)
- builder.add_32bit_float(22.34)
- builder.add_32bit_float(-22.34)
- builder.add_64bit_int(-0xDEADBEEF)
- builder.add_64bit_uint(0x12345678DEADBEEF)
- builder.add_64bit_uint(0x12345678DEADBEEF)
- builder.add_64bit_float(123.45)
- builder.add_64bit_float(-123.45)
tf = 0 while True:
time.sleep(0.1) v = np.linspace(0,115,116)+0.1 # increment watchdog wtd += 1 v[0] = wtd reg = 0 ts = time.time() builder = BinaryPayloadBuilder() for i in range(len(v)): builder.add_16bit_int(int(v[i]))
reg=0; address=0
- initialize data
data = [0.1,1.1,2.1,3.1,4.1]
for i in range(10):
print('*5,'Cycle ',i,'*30) time.sleep(1.0) # increment data by one for i,d in enumerate(data): data[i] = d + 1 # write holding registers (40001 to 40005) print('Write',data) builder = BinaryPayloadBuilder(byteorder=Endian.Big, wordorder=Endian.Little) for d in data: builder.add_16bit_int(int(d))
# write multiple registers at once # limited to 256 registers
skip_encode=True, unit=int(slave_address)) tf += time.time()-ts if wtd%10==0: print('Average Write Time: ' + str(tf/wtd) + ' sec')
skip_encode=True, unit=int(address)) # read holding registers rd = client.read_holding_registers(reg,len(data)).registers print('Read',rd)
See MODBUS Transfer on Data-Driven Engineering for additional information and examples.
Instrumentation and Control Course at Notre Dame
This example code Show Modbus Example Code connects to a Modbus Server (Slave) with a Modbus Client (Master) using pymodbus. The code writes a table of 16 values to 16 individual registers (2 each for a floating point number). The registers start at 40001.
(:toggle hide modbus button show="Show Modbus Example Code":) (:div id=modbus:)
This example code is a Modbus server and client that communicate using pymodbus. The code writes 115 values to integer registers. The registers start at address 40001.
(:toggle hide modbus_server button show="Modbus Server":) (:div id=modbus_server:)
- #######################################
- Modbus write
- #######################################
try:
# import the various server implementations from pymodbus.client.sync import ModbusTcpClient as ModbusClient from pymodbus.constants import Endian from pymodbus.payload import BinaryPayloadBuilder from pymodbus.client.sync import ModbusTcpClient as ModbusClient from pymodbus.payload import BinaryPayloadDecoder # initiate client ##client = ModbusClient('192.168.0.1') client = ModbusClient(host='localhost', port=502) slave_address = 0 # AVG Registers Modbus 40001,3,5,7 # MAX Registers Modbus 40009,11,13,15 # MIN Registers Modbus 40017,19,21,23 # INST Registers Modbus 40025,27,29,31 # registers reg = 0 # AVG, MAX, MIN, INST for i in range(0,4): # Channels for j in range(0,4): builder = BinaryPayloadBuilder(endian=Endian.Little) builder.add_32bit_float(opcm[j][i]) payload = builder.build() result = client.write_registers(int(reg), payload, skip_encode=True, unit=int(slave_address)) # two registers for floating point numbers reg = reg + 2 client.close()
except:
print 'Modbus communication failed' pass
- Modbus server (TCP)
from pymodbus.server.asynchronous import StartTcpServer from pymodbus.server.asynchronous import StartUdpServer from pymodbus.server.asynchronous import StartSerialServer
from pymodbus.device import ModbusDeviceIdentification from pymodbus.datastore import ModbusSequentialDataBlock from pymodbus.datastore import ModbusSlaveContext, ModbusServerContext from pymodbus.transaction import (ModbusRtuFramer,
ModbusAsciiFramer, ModbusBinaryFramer)
- configure the service logging
import logging FORMAT = ('%(asctime)-15s %(threadName)-15s'
' %(levelname)-8s %(module)-15s:%(lineno)-8s %(message)s')
logging.basicConfig(format=FORMAT) log = logging.getLogger() log.setLevel(logging.ERROR) # logging.DEBUG
def run_async_server():
nreg = 200 # initialize data store store = ModbusSlaveContext( di=ModbusSequentialDataBlock(0, [15]*nreg), co=ModbusSequentialDataBlock(0, [16]*nreg), hr=ModbusSequentialDataBlock(0, [17]*nreg), ir=ModbusSequentialDataBlock(0, [18]*nreg)) context = ModbusServerContext(slaves=store, single=True) # initialize the server information identity = ModbusDeviceIdentification() identity.VendorName = 'APMonitor' identity.ProductCode = 'APM' identity.VendorUrl = 'https://apmonitor.com' identity.ProductName = 'Modbus Server' identity.ModelName = 'Modbus Server' identity.MajorMinorRevision = '2.3.0' # TCP Server StartTcpServer(context, identity=identity, address=("127.0.0.1", 502))
if __name__ == "__main__":
run_async_server()
(:sourceend:) (:divend:)
(:toggle hide modbus_client button show="Modbus Client":) (:div id=modbus_client:) (:source lang=python:)
- Modbus client
from pymodbus.client.sync import ModbusTcpClient as ModbusClient from pymodbus.constants import Endian from pymodbus.payload import BinaryPayloadBuilder from pymodbus.client.sync import ModbusTcpClient as ModbusClient from pymodbus.payload import BinaryPayloadDecoder import numpy as np import time
print('Start Modbus Client after Modbus Server is Running')
- initiate client
client = ModbusClient(host='127.0.0.1', port=502) slave_address = 0
- initialize watchdog
wtd = 0
- builder.add_string('abcdefgh')
- builder.add_bits([0, 1, 0, 1, 1, 0, 1, 0])
- builder.add_8bit_int(-0x12)
- builder.add_8bit_uint(0x12)
- builder.add_16bit_int(-0x5678)
- builder.add_16bit_uint(0x1234)
- builder.add_32bit_int(-0x1234)
- builder.add_32bit_uint(0x12345678)
- builder.add_16bit_float(12.34)
- builder.add_16bit_float(-12.34)
- builder.add_32bit_float(22.34)
- builder.add_32bit_float(-22.34)
- builder.add_64bit_int(-0xDEADBEEF)
- builder.add_64bit_uint(0x12345678DEADBEEF)
- builder.add_64bit_uint(0x12345678DEADBEEF)
- builder.add_64bit_float(123.45)
- builder.add_64bit_float(-123.45)
tf = 0 while True:
time.sleep(0.1) v = np.linspace(0,115,116)+0.1 # increment watchdog wtd += 1 v[0] = wtd reg = 0 ts = time.time() builder = BinaryPayloadBuilder() for i in range(len(v)): builder.add_16bit_int(int(v[i])) payload = builder.build() # write multiple registers at once # limited to 256 registers result = client.write_registers(int(reg), payload, skip_encode=True, unit=int(slave_address)) tf += time.time()-ts if wtd%10==0: print('Average Write Time: ' + str(tf/wtd) + ' sec')
client.close()
result = client.write_registers(int(reg), payload, skip_encode=True, unit=int(slave_address))
result = client.write_registers(int(reg), payload, skip_encode=True, unit=int(slave_address))
The Arduino is a popular micro-controller that allows data acquisition, limited on-board processing, and output capabilities. With a large developer community and supported sensors, this platform is a popular choice for data acquisition and automation applications.
The Arduino is a popular micro-controller that allows data acquisition, limited on-board processing, and output capabilities. With a large developer community and supported sensors, this platform is a popular choice for data acquisition and automation applications. The Temperature Control Lab (TCLab) connects to a computer with a serial USB connection with MATLAB or Python. The serial USB connection sends commands (heater level, LED level) and receives data (temperature) to enable real-time control.
(:html:) <video width="550" controls autoplay loop>
<source src="/do/uploads/Main/tclab_linear_mpc.mp4" type="video/mp4"> Your browser does not support the video tag.
</video> (:htmlend:)
This example code **Show Modbus Example Code** shows how to connect to a Modbus Server (Slave) with a Modbus Client (Master) using **pymodbus**. The code writes a table of 16 values to 16 individual registers (2 each for a floating point number). The registers start at 40001.
This example code Show Modbus Example Code connects to a Modbus Server (Slave) with a Modbus Client (Master) using pymodbus. The code writes a table of 16 values to 16 individual registers (2 each for a floating point number). The registers start at 40001.
This example code **Show OPC Example Code** shows how to write to an OPC Server with an OPC Client (OpenOPC). The code writes 16 values to 16 individual OPC tags on a Kepware OPC Server.
This example code Show OPC Example Code writes to an OPC Server with an OPC Client with OpenOPC. The code writes 16 values to 16 individual OPC tags on a Kepware Server.
This example shows how to connect to a Modbus Server with a Modbus Client. The code writes a table of 16 values to 16 individual registers (2 each for a floating point number). The registers start at 40001.
This example code **Show Modbus Example Code** shows how to connect to a Modbus Server (Slave) with a Modbus Client (Master) using **pymodbus**. The code writes a table of 16 values to 16 individual registers (2 each for a floating point number). The registers start at 40001.
This example code **Show OPC Example Code** shows how to write to an OPC Server with an OPC Client (OpenOPC). The code writes 16 values to 16 individual OPC tags on a Kepware OPC Server.
(:div id=gekko:)
(:div id=modbus:)
(:toggle hide opc button show="Show OPC Example Code":) (:div id=opc:) (:source lang=python:)
- #######################################
- OPC write
- #######################################
try:
# OPC connection import OpenOPC opc=OpenOPC.client() b=opc.connect('Kepware.KEPServerEX.V5') #opc.connect('Kepware.KEPServerEX.V5','localhost') Load1_avg = opcm[0][0] Load2_avg = opcm[0][1] Load3_avg = opcm[0][2] Load4_avg = opcm[0][3] Load1_max = opcm[1][0] Load2_max = opcm[1][1] Load3_max = opcm[1][2] Load4_max = opcm[1][3] Load1_min = opcm[2][0] Load2_min = opcm[2][1] Load3_min = opcm[2][2] Load4_min = opcm[2][3] Load_T12 = opcm[3][0] Load_T21 = opcm[3][1] Load_T32 = opcm[3][2] Load_T41 = opcm[3][3] opc.write( ('Channel2.Device1.T_12_Load_AVG',Load1_avg) ) opc.write( ('Channel2.Device1.T_21_Load_AVG',Load2_avg) ) opc.write( ('Channel2.Device1.T_32_Load_AVG',Load3_avg) ) opc.write( ('Channel2.Device1.T_41_Load_AVG',Load4_avg) ) opc.write( ('Channel2.Device1.T_12_Load_MAX',Load1_max) ) opc.write( ('Channel2.Device1.T_21_Load_MAX',Load2_max) ) opc.write( ('Channel2.Device1.T_32_Load_MAX',Load3_max) ) opc.write( ('Channel2.Device1.T_41_Load_MAX',Load4_max) ) opc.write( ('Channel2.Device1.T_12_Load_MIN',Load1_min) ) opc.write( ('Channel2.Device1.T_21_Load_MIN',Load2_min) ) opc.write( ('Channel2.Device1.T_32_Load_MIN',Load3_min) ) opc.write( ('Channel2.Device1.T_41_Load_MIN',Load4_min) ) opc.write( ('Channel2.Device1.T_12_Load_INST',Load_T12) ) opc.write( ('Channel2.Device1.T_21_Load_INST',Load_T21) ) opc.write( ('Channel2.Device1.T_32_Load_INST',Load_T32) ) opc.write( ('Channel2.Device1.T_41_Load_INST',Load_T41) ) opc.close()
except:
print 'OPC communication failed' pass
(:sourceend:) (:divend:)
This example shows how to connect to a Modbus Server with a Modbus Client. The code writes a table of 16 values to 16 individual registers (2 each for a floating point number). The registers start at 40001.
(:toggle hide modbus button show="Show Modbus Example Code":) (:div id=gekko:) (:source lang=python:)
- #######################################
- Modbus write
- #######################################
try:
# import the various server implementations from pymodbus.client.sync import ModbusTcpClient as ModbusClient from pymodbus.constants import Endian from pymodbus.payload import BinaryPayloadBuilder from pymodbus.client.sync import ModbusTcpClient as ModbusClient from pymodbus.payload import BinaryPayloadDecoder # initiate client ##client = ModbusClient('192.168.0.1') client = ModbusClient(host='localhost', port=502) slave_address = 0 # AVG Registers Modbus 40001,3,5,7 # MAX Registers Modbus 40009,11,13,15 # MIN Registers Modbus 40017,19,21,23 # INST Registers Modbus 40025,27,29,31 # registers reg = 0 # AVG, MAX, MIN, INST for i in range(0,4): # Channels for j in range(0,4): builder = BinaryPayloadBuilder(endian=Endian.Little) builder.add_32bit_float(opcm[j][i]) payload = builder.build() result = client.write_registers(int(reg), payload, skip_encode=True, unit=int(slave_address)) # two registers for floating point numbers reg = reg + 2 client.close()
except:
print 'Modbus communication failed' pass
(:sourceend:) (:divend:)
There are many methods to acquire and send data for industrial systems with a range of proprietary and community-based standards that facilitate exchange of information between instruments, a programmable logic controller (PLC), a distributed control system (DCS), and other systems that measure, analyze, or optimize the system. Exchange of information is increasingly important, particularly for optimization solutions, as availability of information is the foundation for many automation and optimization methods. Of particular interest to this course are the methods to retrieve data, run parameter estimation or optimization algorithms, and then either display advisory results or implement a solution back into the process. Common communication standards include:
There are many methods to acquire and send data for industrial systems with a range of proprietary and community-based standards that facilitate exchange of information between instruments, a programmable logic controller (PLC), a distributed control system (DCS), and other systems that measure, analyze, or optimize the system. Exchange of information is increasingly important, particularly for optimization solutions, as availability of information is the foundation for many automation and optimization methods. Of particular interest to this course are the methods to retrieve data, run parameter estimation or optimization algorithms, and then either display advisory results or implement a solution back into the process.
Common Communication Standards
Data Acquisition and Communication
There are several hardware platforms that allow data acquisition and software platforms that facilitate communication.
Robot Operating System (ROS)
ROS is a C++ or Python library that allows inter-process communication. It can be run on anything from a desktop computer to an Arduino (at least in part). Its strength comes from the many different types of ROS packages that already exist and allow for sensors and actuators to communicate.
Micro-processors
Several micro-processors, such as the Raspberry PI and the Beagle Bone Black, have been developed to allow on-board data acquisition and processing. These credit-card sized platforms run a full version of Linux with ARM processors several digital and analog inputs and outputs.
Micro-controllers
The Arduino is a popular micro-controller that allows data acquisition, limited on-board processing, and output capabilities. With a large developer community and supported sensors, this platform is a popular choice for data acquisition and automation applications.
- (MODBUS)
- (OPC-DA / OPC-UA)
- (SQL Server / Client)
- MODBUS
- OPC-DA / OPC-UA
- SQL Server / Client
- MODBUS
- OPC-DA / OPC-UA
- SQL Server / Client
MODBUS (Wikipedia Article)
- (MODBUS)
- (OPC-DA / OPC-UA)
- (SQL Server / Client)
MODBUS
OPC (Wikipedia Article)
OPC
SQL Client/Server (Wikipedia Article)
SQL Client/Server
OPC (Wikipedia Article)
OLE (Object Linking and Embedding) for Process Control (OPC), was developed in 1996 by an industrial automation group based on the need for a common platform for exchanging data. The OPC standard details the communication of real-time or historical plant data between control devices and computers. OPC is sometimes referred to as "Oh, Please Connect" with frequent difficulty in connecting to various computers with incorrect DCOM settings. A recent effort termed OPC-UA or Unified Architecture, is a new implementation of the software that allows communication to devices other than the Windows OS platform. In November 2011, the OPC Foundation (body primarily responsible for the OPC standard) officially renamed OPC to mean "Open Platform Communications".
- MODBUS Toolbox for Matlab
- MODBUS Package for Python
OPC (Wikipedia Article)
OLE (Object Linking and Embedding) for Process Control (OPC), was developed in 1996 by an industrial automation group based on the need for a common platform for exchanging data. The OPC standard details the communication of real-time or historical plant data between control devices and computers. OPC is sometimes referred to as "Oh, Please Connect" with frequent difficulty in connecting to various computers with incorrect DCOM settings. A recent effort termed OPC-UA or Unified Architecture, is a new implementation of the software that allows communication to devices other than the Windows OS platform. In November 2011, the OPC Foundation (body primarily responsible for the OPC standard) officially renamed OPC to mean "Open Platform Communications".
- OPC Toolbox for MATLAB
- OPC Client for Python (OpenOPC)
SQL (Structured Query Language) is a programming language popular with enterprise and web applications that requirement storage and access to historical or real-time data. It is not commonly used for process control or automation like MODBUS or OPC but the large user base of SQL programmers and requirements for big-data suggests that SQL may become a more popular platform in the future.
- SQL Queries from MATLAB
- SQL Queries from Python
(:keywords data acquisition, OLE for process control, OPC, DAQ, dynamic data, simulation, validation, differential, algebraic, tutorial:)
(:keywords data acquisition, OLE for process control, OPC, Modbus, DAQ, dynamic data, simulation, validation, differential, algebraic, tutorial:)
There are many methods to acquire and send data for industrial systems with a range of proprietary and community-based standards that facilitate exchange of information between instruments, a programmable logic controller (PLC), a distributed control system (DCS), and other systems that measure, analyze, or optimize the system. Exchange of information is increasingly important, particularly for optimization solutions, as availability of information is the foundation for many automation and optimization methods. Of particular interest to this course are the methods to retrieve data, run parameter estimation or optimization algorithms, and then either display advisory results or implement a solution back into the process. Common communication standards include:
- MODBUS
- OPC-DA / OPC-UA
- SQL Server / Client
OPC (Wikipedia Article)
OLE (Object Linking and Embedding) for Process Control (OPC), was developed in 1996 by an industrial automation group based on the need for a common platform for exchanging data. The OPC standard details the communication of real-time or historical plant data between control devices and computers. OPC is sometimes referred to as "Oh, Please Connect" with frequent difficulty in connecting to various computers with incorrect DCOM settings. A recent effort termed OPC-UA or Unified Architecture, is a new implementation of the software that allows communication to devices other than the Windows OS platform. In November 2011, the OPC Foundation (body primarily responsible for the OPC standard) officially renamed OPC to mean "Open Platform Communications".
MODBUS (Wikipedia Article)
ASCII or RTU MODBUS is an older standard than OPC and has several limitations that motivated the creation of OPC. Although it is an old standard, several legacy pieces of equipment still support this protocol. There are several serial or network connection possibilities including RS232, RS422, RS485 (serial) or TCP/IP (network).
SQL Client/Server (Wikipedia Article)
(:title Dynamic Data Acquisition:) (:keywords data acquisition, OLE for process control, OPC, DAQ, dynamic data, simulation, validation, differential, algebraic, tutorial:) (:description Python tutorial for collecting data for dynamic simulation, estimation, and control:)