Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Odd interference issue with other equipment #392

Closed
x52x58 opened this issue Dec 16, 2018 · 4 comments
Closed

Odd interference issue with other equipment #392

x52x58 opened this issue Dec 16, 2018 · 4 comments
Labels
help wanted An issue that needs contributors needs info An issue that needs more information

Comments

@x52x58
Copy link

x52x58 commented Dec 16, 2018

Ok, so I've got a really odd issue with the software. I'm an Industrial Programmer and was at a treatment plant to work on some unrelated system issues. Here's where it gets odd. I had installed GSConnect and been using it without any issues for a few weeks prior. I got on site and connected my laptop (Ubuntu 18.04.1 LTS) to the plant network. A few minutes later an operator calls me to let me know that several PLC's (Programmable Logic Controllers) had lost communications with the SCADA(HMI) system. I take a look and find I cannot see them on the network, nor ping them from the system. Walking out to the offending equipment I find the network modules are locked up in error. Only by cycling power on each unit locked up fixes the issue.

Then I find about 10 minutes or so later the same issue happens. I decide to unplug my laptop just in case it's causing the issue and proceed to reset the equipment again. Wait about an hour(went to lunch) and decide whatever cause the issue must be cleared, as we had a phone company tech working on the DSL Modem. I figure it's ok to plug my computer back in again. A few minutes later, boom everything drops again.

Just by deduction and a good guess I figure it might be GSConnect causing the issue. Since it presumably does a broadcast across the network and was one of the most recent programs added to my laptop since I had been to this site last.

I disconnect my laptop again and reset the plant equipment. I then disable GSConnect and reboot my computer. Reconnect it to the network and thus no issues.

So yeah, it's a odd probably isolated case issue. I'm mostly submitting the error report because I'd hate for a similar issue to happen to someone at a larger plant or somewhere with critical equipment. Luckily in my case it was a small plant and could stand to have equipment shut down and restarted.

To Reproduce
Steps to reproduce the behavior:

  1. Connect laptop with GSConnect running
  2. Plant PLC's error out
  3. Disable GSConnect
  4. Reset PLC's and test.
  5. PLC's ok.

Expected behavior
Connect Laptop with GSConnect enabled and cause PLC network cards to error

Screenshots
No information on screen, all issues happen on plant equipment.

Debug output
If needed I could probably see about running it on a test rig, I'd rather not take the plant PLC's down again. They weren't happy with it the last time.


System Details (please complete the following information):

  • GSConnect version: 17
    • Installed from: extensions.gnome.org
  • GNOME/Shell version: 3.28.2
  • Distro/Release: Ubuntu 18.04

GSConnect environment (if applicable):

  • Paired Device(s): Samsung Galaxy S9+
  • KDE Connect app version: 1.10
  • Plugin(s): N/A

Additional context
Here's links to the type of PLC's and network card in them in case you wanted to know.
PLC:
https://www.automationdirect.com/adc/shopping/catalog/programmable_controllers/directlogic_series_pl
cs_(micro_to_small,brick-a-modular)/directlogic_405(small_modular_plc)/cpus/d4-454

Network adapter for PLC:
https://www.automationdirect.com/adc/shopping/catalog/programmable_controllers/directlogic_series_plcs_(micro_to_small,_brick_-a-_modular)/directlogic_405_(small_modular_plc)/communications_-a-_networking/h4-ecom100

OK, there you go. I'm thinking it might be something like a brodcast issue or something that the PLC Network cards just don't like. So it might be nothing that can be done about it. They're old model PLC's and the newer PLC's that have been installed recently had no issues.

@andyholmes
Copy link
Collaborator

Hmm, the only service I can find that uses UDP port 1716 is xmsg. which is apparently some kind of XML RPC protocol. Is it possible these PLC's use XML RPC for configuration somehow?

@andyholmes
Copy link
Collaborator

Possibly related bugs:

https://bugs.kde.org/show_bug.cgi?id=399694
#389 (comment)

@andyholmes andyholmes added help wanted An issue that needs contributors needs info An issue that needs more information labels Dec 17, 2018
@ferdnyc
Copy link
Member

ferdnyc commented Dec 17, 2018

OK, there you go. I'm thinking it might be something like a brodcast issue or something that the PLC Network cards just don't like. So it might be nothing that can be done about it. They're old model PLC's and the newer PLC's that have been installed recently had no issues.

Older network devices like that unfortunately often don't play well with general network traffic, they do better isolated on their own dedicated subnet with no foreign devices present. Especially when (as is the case with these ECOM modules, apparently) they rely on network broadcasts of their own for discovery and peering.

The protocol docs for the Modbus TCP protocol specify that it communicates bidirectionally over TCP port 502, exclusively. However, the docs for the ECOM modules themselves indicate that Modbus TCP is used primarily for PC-to-PLC communication, and that they also support IPX protocol messaging and PEERLINK TCP broadcasts (10×/second) for PLC-to-PLC communication.

So, their firmware is going to be monitoring network traffic looking for broadcasts of certain types, and that firmware could be buggy and susceptible to disruption by discovery traffic from programs like GSConnect even though all of the network messages GSConnect transmits are correctly formed.

They're also 10/100bT devices, which means if they're connected to a gigabit-or-faster network, they're going to end up flooded with traffic from faster devices, possibly causing network buffer overflows that could trigger other firmware bugs that lead to crashes/lockups.

The fact that the technical docs for the ECOM modules stress this point multiple times makes me think that Host Automation are not entirely unaware of this possibility (though I'm sure their lawyers would make them say this regardless):

WARNING: We recommend against connecting Ethernet modules to the same network that serves as your primary office network. While Ethernet networks can handle a very large number of data transmissions, and normally handle them very quickly, heavy Ethernet traffic can adversely affect the reliability and speed of the network. – src (pdf)

@andyholmes
Copy link
Collaborator

I've added mention of it in the Wiki. Since there's not much we can do about this without breaking protocol, I'm going to close this now. Thanks for reporting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted An issue that needs contributors needs info An issue that needs more information
Projects
None yet
Development

No branches or pull requests

3 participants