Panelview Legacy HMIs dropping connections

[Ed IrelandParticipantKarma: 16Rank: Padawan]

All of the Legacy Panelview HMIs in our network recently started dropping their PLC connections after approximately 30 minutes.  Other Panelview Plus and Panelview 6 HMIs are continuing to work without issue.  We use Cisco Catalyst switches throughout the network, and all process control devices are separated from other traffic by assignment to VLAN30.   The communication problem surfaced at approximately the same time that a new Cisco 3650 switch was installed off of the core.  There are no process control devices connected to the new Catalyst 3650, and none of its ports are configured for VLAN30 traffic.  In troubleshooting we’ve found that the communication problem between the HMIs and PLCs is eliminated by replacing the Cisco 2960 edge switch, (the 2960 connects some of the HMIs and PLCs to the network)  with an unmanaged switch.   On the 2960, we’ve already tried setting speed and duplex on the VLAN30 switch ports and on the PLCs,  and have removed spanning-tree portfast settings as well, but neither have helped.  We’ve also tried replacing the 2960 with another new managed Cisco switch, but this has not helped either.   Any ideas or suggestions would be greatly appreciated!

[mrincredibleParticipantKarma: 53Rank: Padawan]

Hi Ed,

I dont think I’m going to be too much help but I am interested in following this thread. From your post it sounds like you performed all logical steps to try to troubleshoot it.

A couple things that are interesting. The issue only is experienced on legacy panel views…I assume you’re referring to the Panelview line that runs the old Panelbuilder32 software. The thing that’s interesting is that when you put an unmanaged switch the problem goes away. That would indicate to me a Cisco configuration issue/mismatch somewhere…but why effecting only the old Panelviews???

Interesting problem. Please do update when you find a solution.  Hopefully some of the network gurus here can offer more!

[Ken SParticipantKarma: 172Rank: Jedi]

1: you replaced the switch with a NON-Managed switch and all seems good.
2: Now this means you have a bad setting in the managed switch.

Now is that managed switch allowing duplicate IP’S on the network ?
How is routing on that Switch setup  ?
Sometimes your routing will cause this as it will pass through to another network and you could end up duplicating IP’s that way as well.
Are your ports setup to power save after idle period ?
How are those ports configured  – it would seem that for me I would look at one port thoroughly, make changes to that port and see what effects I get on the device connected to that port .. I might even separate the switch from the rest of the LAN to see what happens between a couple HMI’s to isolate out – a port configuration setting versus a LAN setting .

[Ed IrelandParticipantKarma: 16Rank: Padawan]

Thank you Ken S and Mister Incedible for your responses.

I should have mentioned in my initial post that the failing HMIs and PLCs are spread across three separate switches.  The 2960 that was mentioned connected three of the five failing legacy Panelviews to the network.

As a test, today I “shut” the port on the core switch that I had trunked the new C3560 switch into.   For whatever reason, this resolved the problem that the legacy Panelview (built with Panelbuilder32) HMIs had with dropping PLC connections.  Several tests were run to make sure that this wasn’t some sort of fluke;  as long as the C3560 was disconnected from the core, the HMIs had no trouble staying connected,  but after reconnecting the C3560 the HMIs would drop the PLCs within 35 minutes.  Leaving the C3560 disconnected from the network wasn’t a long term fix as it is used to tie a VMWare cluster and a SAN to the network that provide essential services.  Excluding VLAN30 in the trunk configuration between the core and the C3560 appears to be a satisfactory solution until we can either upgrade the HMIs to a newer (supported) version or figure out what the real problem is.

[Author]

Posts