OPC UA Events based plugin

[sjwang90]

Feature Request

Recently merged inputs.opcua plugin currently polls on a certain interval to gather metrics, it does not respond immediately to events. Therefore any event collected would be delayed based on the interval setting.

• will have to be a separate plugin from existing polling plugin - maybe a opcua_events plugin (like snmp vs snmp_trap plugins)
• subscription based plugin that responds to OPC UA events
• supports arrays
• look into using service input for building enhancement. For initial opcua plugin, attempted to implement both input and service input interfaces in the same plugin but it failed and only functioned as an input.

Use case:

OPC UA is subscription based, will be expected by a lot of users

[loonwong]

Hello,

As for the manufacturer of the opc ua server, this is an interesting topic.

I would like to note that in opc ua, events mean a special system (https://reference.opcfoundation.org/v104/Core/docs/Part3/4.6.1/), so I would suggest a different plugin name, such as opcua_subscription.

[gavioto]

I think that "subscription" mode should be a feature of the actual plugin.
In fact, traditionally in OPC the subscription, or polling is only a configuration layer, at the end you want to receive updates and you don't mind how these updates are sent. Take into account that using subscription transparently (something that all OPC servers should support) have better performance, as the system only updates with changes, and also you can set a maximin frequency for updates.

Every other configuration option like tags or items, nodes, types, will be the same in both plugins. I think it makes no sense to have duplicated plugins.

On the other hand, having a plugin to get "OPC UA Events" like @dalekseev-monokot commented probably makes sense on it's own.

[loonwong]

Yes, I also agree with @gavioto , usually polling and subscription are implemented as a OPC UA client configuration layer. I also want to draw your attention to the fact that in addition to improved performance, the subscription mechanism solves two very important problems:

1. Prevents data loss during connection failure with the OPC UA server by buffering data on the OPC UA server side.
2. Allows you to get values that change sporadically. I.e., values that change an arbitrary number of times over an arbitrary time interval. An example is the popular IEC 60870-5-104 in the power industry.

It seems to me that the plugin should be designed to solve these tasks as well.

[sinfergit]

Any update on this? A newer version released with pub/sub method for OPC UA plugin?

[Jayclifford345]

Adding this community query to the support for this feature: https://community.influxdata.com/t/telegraf-opc-ua-data-incoming-with-1ms-speed-but-influxdb-only-showing-10-sec-intervals/22836/4

[mharjula]

I would very much prefer a subscription type model.

Additionally I have problems with Red Lion DA 10 servers, since they do not seem to support registered reads (see #9551 ). Normal reading and subscriptions seem to work in the gopcua examples.

See also gopcua/opcua#478

[LarsStegman]

I have implemented a prototype to connect to a server using OPC UA subscriptions. Currently, I only implemented DataChangeNotification. This is easily extendable to EventNotificationList, but as our team has no need for this, I cannot assess how an event should be mapped to a Metric.

I believe this should not be a separate plugin, but simply an extension to the existing input plugin. For this reason I have refactored the existing input plugin to reuse as much code as possible. This seems to work very well so far. The refactor was quite extensive, as I needed the existing code to be much more modular than it was. The refactor also allows some of the code to be reused for an OPC UA output plugin. In my opinion, the refactor makes this much easier.

@sjwang90 I am planning to create a pull request soon. Do you want me to make one pull request for the refactor and the subscription feature, or should I split this up in separate pull requests?

[R290]

@LarsStegman I agree with your idea of sticking to a single plugin. You could for example extend the NodeSettings structure with a field that describes if it is a request, event or subscription. Feel free to create a pull request and tag me in it for review. I started working on something similar a while ago, so I'm curious what you came up with.

[LarsStegman]

You want to specify per node whether it should use a request, event or subscription. I do not entirely agree with that. My implementation is on a plugin instance level. All nodes specified in the config are either requested or subscribed to.

If you want to subscribe to certain nodes and request others, you can instantiate multiple instances of the plugin: one for subscriptions and one for requests:

• In both cases you need to create 2 TCP connections to the OPC UA server;
• The plugin logic is easier to follow ;
• The code is easier to modularize;
• A connection to Influx (or any other output) will be more predictable.

[R290]

The implementation you describe would also be perfectly acceptable for my use case.

[dremsol]

@LarsStegman I look forward to your PR!

[R290]

@LarsStegman any update on this?

[LarsStegman]

Yes, the PR is ready to go, but I'm still waiting on the go ahead from the legal department.

[sjwang90]

@LarsStegman Let me know if there's any legal logistics I can help with on the Influx side.

[R290]

Should we put a deadline on this? And start parallel development of the subscription feature if it is exceeded? Seems that we now are in a bit of a deadlock for such an in demand feature.

[powersj]

@LarsStegman were you able to work through your legal department? As @sjwang90 mentioned we are happy to help out if you need anything from InlfuxData.

@R290 - yeah let's see if we get a response. Otherwise, is this something you are interested in contributing? Otherwise I was going to add to the team's Q4 list to see where it might land in priorities.

[LarsStegman]

I'm still waiting on confirmation. I'll ask again this week.

[R290]

Shall we set the deadline date on something like the 1st of October? This is about 6 weeks from now and should be enough time to arrive at a conclusion w.r.t. the legal department?

@powersj I should be able to integrate something in the existing plugin. The Go OPC-UA library is quite complete, so it is mostly high level.

[powersj]

Sounds good to me.

@LarsStegman does that work for you? If it does come down to it, I'm sure @R290 and myself would love your help testing something.

[LarsStegman]

I will hear back tomorrow. I will let you know what the results are.

[LarsStegman]

Good news: I have received permission. I will make the pull request somewhere next week.

[LarsStegman]

PR created 😄

[Skr3ms]

Hey together and @LarsStegman
One question. I have the opcua_listener plugin running, but to read the subscribed event correctly i need to declare the event type.

[[inputs.opcua_listener.nodes]]
     name = "node2"
     namespace = "3"
     identifier_type = "i"
     identifier = "1812"

In this case i get no information about the incoming objects itself. I
know the id from basetype is i=2041. Do somebody know how to configure the
conf file the correct way?
Best greetings Alex

[LarsStegman]

[[inputs.opcua_listener.nodes]]
     name = "my_field_name"
     namespace = "x"  # you really need to known this 
     identifier_type = "i"
     identifier = "2041"

should do it

[inselbuch]

I had quite some difficulty in configured the CONF file for the input plugin. Don't know if this is helpful, but here is an example that is working for me. Note that idenitifier_type is not the same as data_type. That just indicates what to expect in the identifier="" Regards, Frank [[inputs.opcua_listener]] name = "opc141" endpoint = "opc.tcp://10.10.0.141:53530/OPCUA/SimulationServer" connect_timeout = "10s" request_timeout = "5s" # subscription_interval = "300s" # if the OPC server has security, it is supported, sample server is open security_policy = "None" security_mode = "None" # auth_method = "UserName" # username = "" # password = "" # namespace="3" nodes = [ {name="value",namespace="3",identifier_type="i",identifier="1001",data_type="integer",tags=[["point","Counter"],["server","${HOSTID}"],["quality","good"]] }, {name="value",namespace="3",identifier_type="i",identifier="1003",data_type="float",tags=[["point","Sawtooth"],["server","${HOSTID}"],["quality","good"]] } ] Frank Inselbuch 713-701-5421 ***@***.*** From: Lars Stegman ***@***.***> Date: Tuesday, November 1, 2022 at 10:57 AM To: influxdata/telegraf ***@***.***> Cc: Frank Inselbuch ***@***.***>, Manual ***@***.***> Subject: Re: [influxdata/telegraf] OPC UA Events based plugin (#8083) [[inputs.opcua_listener.nodes]] name = "my_field_name" namespace = "x" # you really need to known this identifier_type = "i" identifier = "2041" should do it — Reply to this email directly, view it on GitHub<#8083 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFMVC6MRW7NZESIV3AQ57STWGE4XZANCNFSM4QYUNVFA>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Skr3ms]

Hello inselbuch,
the node "1001" and "1003" you subscribe in your sample are singel variables of the the type integer and float?
In my case the object behind my node id is an Alarm Message of the type BaseEventType.
The property "data_type" you write is not explained in any official example about the opc ua config file, where do you know that you can define it?
Im new in InfluxDB and Telegraf sorry, i have one general question. When you write fields or tags of the point, are the values of these properties are completly static? Like in your example the point will have a tag named "quality" and the values is everytime "good"?
Best Greetings
Alex

[JohnArneBirkeland]

It is great that Telegraf now has subscription based OPC-UA input.

But looking at the latest Telegraf documentation, I don't see any way to set the DataChange parameters when monitoring a node.

This is an important part of the OPC-UA subsciption model, and essential for managing logging in systems where you may have thousands of sensors spewing out data.

[LarsStegman]

That's right, this functionality is not implemented. It shouldn't be difficult to add it. Can you open a new issue describing the functionality?

[aq-daniel]

Hello,
I stumbled across the problem of subscribing to opcua array variables. This issue #8475 lead me here. Although the feature request for array support is mentioned by @sjwang90 in the first post, it seems to be not further discussed or worked on?

Unfortunately the lack of support for array variables still persists in the current version of both input.opcua and inputs.opcua_listener plugins.

Here you can see some logs of the inputs.opcua_listener plugin, where the plugin correctly receives the datachange notification of a subscribed array variable - including the values, but doesn't output the values into the generated metric:

Here the same result while using the inputs.opcua plugin. No array values are found in the generated metric for influxdb:

A workaround for the time being would be to convert the array values to a string. This was mentioned by @nicolasj92 in the other issue here.
I hope somebody could help me or point me in the right direction for implementing this workaround. My questions are:

• Where does the functionality to convert received opcua array values into a string need to be implemented? Maybe it could be a selectable workaround for both opcua plugins?
• Would it be possible to also achieve the workaround with a processor plugin? I guess the processor plugins can only work on generated output metrics and therefore don't have access to the array values? So the answer is no?

Any hints to the right place or the right direction are welcome. Thank you!