Ethernet Comes with a Brand New Easy Solution: Single Pair Ethernet (SPE)

Ethernet is well known globally as solution for communication. Ethernet was hated and liked for the last 40 years or so ... there have been alternative solutions developed that were marketed as much easier, faster, deterministic, ... think of Tokenbus (IEEE 804), Profibus, ... and many others.

Now we see a new version: Single Pair Ethernet (SPE). SPE can bring fast Ethernet (up to 1 GBit/s) and power to the field level using just one twisted wire pair ... enabling application of protocols using TCP/IP.

Click HERE for a general description.

Click HERE for a nice presentation by IEEE experts (January 2019)

SPE is a new technology to replace CANbus in automobiles (cars, trucks, busses, ... trains) and fieldbusses. SPE is a layer 1 standard ... so it can be used for Profinet, Ethercat, ... and it could run TCP/IP.

SPE is more intended to replace fieldbus systems ... here my dream of the late 80s becomes true:

Fieldbus Standardization - Another Way to Go

http://blog.nettedautomation.com/2017/05/tsn-fieldbus-standardization-another.html

additional posts related to the topic:

http://blog.nettedautomation.com/search?q=another+way

The use of SPE for connecting sensors to the cloud is to follow a trend ... it may increase the sales of component manufacturers.

When I wrote my Diploma Thesis in 1982 (at Siemens) I was asked to analyze Ethernet ... the idea was cancelled because of the very very expensive MAU ... needed two ... each for 23,000 USD ... total of 46,000 USD ... no way to get approval to spend that amount for a "standard" Diploma Thesis ... 

It took some 40 years to get to SPE - likely the real Ethernet ... ;-)

Too late for me ... just retired this year with 67 ... 

One crucial challenge is here: HOW to SECURE a huge number of end nodes (sensors, actuators ...) directly connected to the clouds or data lakes? Compare the situation with Smart(er) Grids: In Smart(er) Grids it is intended to connect millions of smart meters to the entities (clouds!?) that use the data for billing and further applications like controlling millions of inverters or power users. 

In the German power system there is a requirement to use the so-called Smart Meter Gateway (SMG) to provide highly secure communication channels! 

Click HERE to check what has to be implemented ... many published Megabyte pdf documentation of the required specification like: "Protection Profile for the Gateway of a Smart Metering System (Smart-Meter-Gateway PP)" ... by the German BSI.

It took many years before we have seen the first certified Smart Meter Gateway offered at the market. And be aware: The Administration of this infrastructure is very complex and ... far away from cheap and affordable by "everyone".

Many similar huge "security systems" would be required to connect the billions of smart sensors and actuators through Single Pair Ethernet to some centralized entities ... 

SPE is nice - BUT to build secure distributed systems it is required to develop also new security solutions that are as simple as Single Pair Ethernet!!

We have to look at the complete SYSTEM COST - not just at the possibilities of a new physical layers ... the SPE increases the problems of implementing secure systems, because it is easier and cheap to build a huge mashed network of millions of end nodes ... that may not perfectly secured!

Difference between IEC 60870-5-104 and IEC 61850

There seems to be a growing interest to understand what the difference is between IEC 60870-5-104 and IEC 61850. There have been many discussions, complaints, and frustrations ... no wonder.Here is what I have answered to somebody this week:

Dear xxx,

I guess I got it ... you are analyzing the communication inside a station ... to the IEDs (protection, control, ...).

The IEC 60870-5-104 plus a lot of utility or project specific (signal) engineering will do the job – has done it for decades.

The engineering is the key issue when comparing the two standards … if you can compare them at all!!

IEC 61850 offers a lot more than 104 or DNP3 …

From a message overhead point of view, you can say, that both are more or less the same ... because they use both Ethernet and TCP/IP. There is no benefit to use one or the other.

It is likely that IED vendors will mainly focus on IEC 61850 ... and may get rid of 104 in the long run.

I have always said that utilities using 104 in all substations should continue to use it – until they build new substations or do major refurbishments. There is no need to replace a running 104 solution with IEC 61850 ...

Another issue is: To use GOOSE for interlocking … to get rid of copper … or use it for tripping … and use sampled values some time down the road.

Finally there is an issue with manpower: If the utility has senior experts in 104 close to retirement … they should wait until they have retired. Yes! I have seen many old engineers not willing to learn something completely new!!
Click HERE for a detailed comparison written by domain experts.
Hope that helps a bit more.
Best Regards,

Karlheinz

IEC TC 57 published Two Documents Related to Security Measures (IEC 62351)

IEC TC 57 just published the following two documents:

57/1928/NP
IEC 62351-100-3: Conformance test cases for the IEC 62351-3, the secure communication extension for profiles including TCP/IP

The scope is to specify common available procedures and definitions for conformance and/or interoperability testing of the requirements of IEC 62351-3, the security extension for profiles including TCP/IP.

57/1931/DC
Proposed revision of IEC TS 62351-6 ED1 and conversion into an International Standard (Power systems management and associated information exchange - Data and communications security - Part 6: Security for IEC 61850)

Both documents indicate that the security measures defined by the series IEC 62351 are becoming more important! Hope that more experts in the power delivery domain will understand the impact!

Conflicting Use of TCP Port 102 for IEC 61850 and Simatic S7

IEC 61850-8-1 defines how the abstract IEC 61850 services (ACSI) are mapped to MMS (ISO 9506). The MMS protocol runs on ISO/OSI Transport Layer, ISO/OSI Session Layer, ... For IEC 61850 it has been decided to use TCP/IP as transport protocol.

TCP has to be "extended" by some definitions to get the same services and protocol features as provided by ISO/OSI Transport Layer class 0: The IETF RFC 1006 defines how to use TCP for MMS. RFC 1006 defines among other issues to use TCP Port number 102 for the MMS Server role. Any IEC 61850 Server role has to run on port 102 - independent of the platform it is running on: protection device, control device or a Windows PC.

Siemens SIMATIC S7 PLCs use RFC 1006 entitled "ISO Transport Service on top of the TCP" (ISO-on-TCP) as a protocol extension for the TCP protocol for connection between two systems.

RFC 1006 (and thus Port 102) is used for standard connections in the SIMATIC environment.

• STEP 7 remote programming via LAN
• ISO-on-TCP connections
• S7 connections via Industrial Ethernet

I have come across situations where PCs are running SIMATIC S7 tools that are using Port 102! In that case you cannot run an IEC 61850 Server role on the same PC (with the same IP address) - because Port 102 is already in use!!

If you have trouble running an IE 61850 Server role on your computer - check also if Port 102 is already in use. In one case we figured out this situation with a server model (SCL) that we tried to simulate with the Omicron IED Scout! IED Scout reported an error: TCP Port 102 already in use. We stopped the SIMATIC S7 application to free the Port 102.

This is another use case where the IEDScout reports very useful error information!

Here is an example of the command "netstat -a" (may use as well "netstat -a -b") to figure out, if the port 102 is used or not: Waiting for port "102": 


Click HERE for the Server demo (shown on the right).

Click HERE for a list of ports used by Siemens SIMATIC S7.

Draft for Role Based Access Control (RBAC) Published (IEC 62351-90-1)

IEC TC 57 published the IEC TR 62351-90-1 Draft for Role Based Access Control (RBAC) [57/1905/DTR]:

IEC 62351 Data and communications security –
Part 90-1: Guidelines for handling role-based access control in power systems

The voting period closes on 2017-09-29.

"The power system sector is adopting security measures to ensure the reliable delivery of energy. One of these measures comprises Role-based Access Control (RBAC), allowing utility operators, energy brokers and end-users to utilize roles to restrict the access to equipment and energy automation functionalities on a need-to-handle basis. The specific measures to realize this functionality have been defined in the context of IEC 62351-8. It defines 3 profiles for the transmission of RBAC related information. This information is, but not limited to, being contained in public key certificates, attribute certificates, or software tokens. Moreover, especially for IEC 61850, it defines a set of mandatory roles and associated rights. The standard itself also allows the definition of custom roles and associated rights, but this is not specified in a way to ensure interoperability."

Data and communication security is a crucial issue in the communication between multiple IEC 61850 clients and an IED with a single IEC 61850 Server. The administration of the roles and further behavior requires a highly complex (centralized!?) administration and a complex functionality in each IED implementing RBAC.

The following aspects have a big impact on implementations:

1. TCP/IP Networking,
2. General security measures like TLS,
3. RBAC, 
4. MMS,
5. IEC 61850 Services, Models and Configuration, and
6. Power system functionalities (key for the power delivery system) on top

The bulk of resources needed are mainly independent of the MMS protocol and services. People that want to use other protocols cannot really expect that the cost for getting secure communication and data will be lowered - the most efforts are related to non-protocol issues.
The second, third, fifth, and sixth bullet are most crucial.
In addition to the cost of implementing RBAC (including the other required parts of the series IEC 62351) one has to understand that the operation, management, engineering, and configuration of RBAC consumes a relatively huge amount of resources of the embedded controllers or other platforms.
That is one of the crucial reasons why many IEDs installed today cannot (and likely will not) be upgraded for measures defined in the IEC 62351 series.

Recommendation: As soon as possible get started to understand the impact of the measures defined in IEC 62351 and how to implement some or many of these measures.

Related documents of the series IEC 62351 IEC/TS 62351, Power systems management and associated information exchange – Data and communications security – are:

Part 1: Communication network and system security – Introduction to security issues
Part 3: Communication network and system security – Profiles including TCP/IP
Part 4: Profiles including MMS
Part 5: Security for IEC 60870-5 and derivatives
Part 8: Role-based Access Control

CD published: Conformance Test Cases for the IEC 62351-5

IEC TC 57 just published a 110 page crucial document on security testing (57/1852/CD):

IEC TS 62351 - Data and communications security -
Part 100-1: Conformance test cases for the IEC 62351-5 and its companion standards for secure data exchange communication interfaces

Comments are welcome by 2017-06-09

The scope is to specify common available procedures and definitions for conformance and/or interoperability testing of the IEC/TS 62351-5 (Security for IEC 60870-5 and derivatives), the IEC/TS 60870-5-7 and also their recommendations over the IEC 62351-3 for profiles including TCP/IP. These are the security extensions for IEC 60870-5 and derivatives to enable unambiguous and standardised evaluation of IEC/TS 62351-5 and its companion standards protocol implementations.

What is an IEC 61850 Data Model – Come and See

Data or device modeling is a crucial feature of IEC 61850 and IEC 61400-25. You may have seen many different approaches to explain how such a model looks like. Some five years ago I used these Russian dolls (matryoshka doll):

An IED contains a lot of “inner” objects.

 

Today I have thought that another approach may help you to understand the IEC 61850 approach:

What do you think? This and more will be explained in detail during my comprehensive – most liked – courses.

Does IEC 61850 Add Complexity for Technicians in Power Utilities?

This week I was asked the question in the title during an introduction of IEC 61850 to some 15 utility experts. My response was not just yes or no. Initiated by that question I thought it would be of interest to discuss this issue on the blog.

We have to understand that the expected complexity in power system information exchange has at least the following three crucial aspects:

1. Complexity of the network infrastructure (independent of protocols defined and used by standards like IEC 60870-5-104, DNP3, IEC 61850, IEC 61400-25, …). The infrastructure used and discussed these days seems to explode! Compared to dial-up-links and and fixed land lines used usually for remote access of something, the application of Switched Ethernet, Ethertype, VPN, VLAN, TCP/IP, UDP/IP, GSM, UMTS, LTE, … requires a good understanding of your needs and the various solutions that could be used.
2. Complexity of standards (like IEC 60870-5-104, DNP3, IEC 61850, IEC 61400-25, …) that use the above infrastructure.
3. Complexity of communication software and application interfaces between applications and communication software, and complexity of engineering and configuration tools.

In many cases I have experienced that users do have little understanding what they really need! And may even have lesser knowledge about the various solutions, how to use them for their systems, and to understand how they impact the dynamics of the whole system!

I have talked to many people that have complained about the complexity of protocols … but usually we figured out that the complexity was caused by a bit of everything … and mainly by the fact that people tend to NOT TRUST the chain of solutions from, e.g., a control system application to an API of a front-end, front-end application, protocol API, protocol IEC 60870-5-104, TCP/IP, VPN, GPRS, RTU, interface between RTU and remote application, and remote application.

Here is an example I have experienced recently (with the topology based on GPRS as listed above):

1. The control system does not trust that the information exchange with the RTU is reliable and available. Therefore the control system sends Pings every 2 seconds.
2. The front-end application does not trust that the RTU is reliable and available. Therefore the front-end applications issues a 104 control command (toggle bit) every 10 seconds … just to see if the 104 protocol is still alive.
3. The front-end application does not trust (even it figures out that the RTU is available) that the remote application is really receiving a parameter setting for a function in the remote application. Therefore the remote application copies a received setting value to another 104 information object and sends a spontaneous message with the just received setting value.
4. The protocol IEC 60870-5-104 exchanges flow control messages to acknowledge the received messages (in both directions).
5. TCP uses flow control messages and keep alive messages …

So, what do you think about such a bunch of deep mistrusts? Do you think that such a system would work properly and reliable?

I guess that there are many huge GAPS: in the understanding of the NEEDs, the various links in the chain like the dynamics of a system using, e.g., GPRS, … the APIs, the applications …

I recommended to the audience that there is a crucial need for: MORE EDUCATION !! 

A screw driver is not sufficient for future power delivery systems. And: Ignoring IEC 61850 is not sufficient to get the job done! IEC 61850 solutions can be very easy for simple needs.

You can experience it – if you want! Let me know!

Cyber Security in Industrial Control Systems – Is this enough?

Cyber security is more than a hype. Is this enough to reach a secure and stable power system? No!

I found a very good documentation on cyber security measure:

Since February 2013, industrial stakeholders (final users, vendors, integrators, professional organizations, etc.) and French governmental entities have been working together on elaborating concrete and practical proposals to improve the cyber security of critical infrastructures.

The first results of this working group are the following two documents:

• The first document describes a classification method for industrial control systems and the key measures to improve their cyber security.
• The second one gives a more in-depth description of applicable cyber security measures.

Click HERE for the website with the links to the two documents. Nice reading!

These measures (comparable to those listed by many other organizations and groups) will help to improve the cyber security of critical infrastructures. No question.

Do these measures help to keep the power flowing, help to keep a stable and highly available power system? To some extend these measures solve mainly issues that are caused by new control system solutions based on standards like Ethernet and TCP/IP.

But: What’s about the power system stability? Let’s assume that we have a 100 per cent cyber secure ICS managing the power generation, transmission, distribution, storages, and loads. This “secure” systems may be used in many different ways – taking the physical laws seriously into account or ignoring some basic requirements to keep the power system stable.

One very critical impact on the electrical system is the change of power flow. Each change (more or less generation or load) has to be controlled in a bunch of close loop control systems. If the amount of change in a short time (within seconds) is too high, then the systems is likely to black-out.

A highly secure ICS may be used to configure schedules for feeding power into the power system (generator or storage) or drawing power from the system. The power flow change caused by schedules may exceed the maximum value that can be automatically managed by primary power control systems … risking a power outage.

Who is now responsible that the maximum allowable power flow change in an interconnected power system will be taken into account when we have millions of such schedules? Maybe too may schedules are configured to draw power or feed in starting at 14:00 h today. As a consequence the power flow change could be far beyond the maximum amount that can automatically be managed by the primary power control system (as we have them today in all systems).

Cyber security of ICS is one aspect – system stability of the power system is another. Secure ICS’s are important. A high level of power systems stability is more important and requires secure information and communication systems AND the need of understanding of the power system physics. 

We have to make sure that any new ICS approach does not allow a huge sudden power flow change! This is true also for all solutions based on standards like IEC 60870-5-10x, DNP3, IEC 61850, or …

These standards would allow to disseminate immediate control commands or specify schedules.

WHO is in charge to have the big picture in mind – to configure power systems in a way that they do not blackout because of commands and settings communicated by highly secure ICS’s? The power system could not differentiate if these commands or settings are intended or caused by hackers.

It is highly recommended to keep an eye on the power system physics and prevent any ICS action (secure or insecure) to danger the stability of the power system!

Security Standard IEC 62351-3 on its way

The Technical Specification IEC TS 62351-3, First edition, 2007-06 is underway to become an International Standard (57/1319/CDV):

Power systems management and associated information exchange –
Data and communications security –
Part 3: Communication network and system security – Profiles including TCP/IP

The CVD is out for ballot until 2013-07-05.

IEC 62351-3 specifies how to secure TCP/IP-based protocols through constraints on the
specification of the messages, procedures, and algorithms of Transport Layer Security (TLS)
(defined in RFC 5246) so that they are applicable to the telecontrol environment of IEC TC57. It is intended that this standard be referenced as a normative part of other IEC TC57 standards that have the need for providing security for their TCP/IP-based protocol.

The conformance is very strict:

8 Conformance
Conformance to this part shall be determined by the implementation of all parts of clause 5.

The definition of clause 5 could be implemented today already: the content is available in the Technical Specification IEC TS 62351-3.

There is no (and never was an) excuse to not implement quite secure communication.

Tissue Database for IEC 62351 just opened

The Tissue Database for IEC 62351:

Power systems management and associated information exchange – Data and communications security

has been opened for immediate access. Nine parts have been published so far. You may post your feedback (bug reports, …) now.

Access the Tissue Database for IEC 62351.

Video with brief Introduction to IEC 61850 and IEC 61400-25

IEC 61850 and IEC 61400-25 comprise some 25 documents. Part IEC 61850-7-1 contains some basic modeling concepts that may help to get a few ideas what IEC 61850 is about. I guess that just a few people have read that part. In my training courses with almost 3.000 attendees I have gained a lot of experience on how to explain the basic concepts. In 2011 I have conducted more than 30 training sessions (from one to 12 days). Today I am starting a new service to the industry: providing videos that explain basics with animated up-to-date slides.

The first video is a brief presentation of the key concepts of IEC 61850 (one slide): modeling methods, models, configuration language, communication, and mappings. The demonstration shows how these concepts are used to compose a system. Of course, this slide is just showing the basics of a “small system”. This slide is part of the introduction of my commercial training curses.

Please click on the start button to see the video – in order to see it in the full screen, click again on the video and select the full screen button.

I hope you will enjoy this video!
Your feedback to Karlheinz Schwarz would be appreciated.

The four Keep-alives in IEC 61850

IEC 61850 uses several mechanisms to monitor the communication between two devices (Client/Server) or between a publisher and many subscribers and to monitor functions – in order to increase the overall reliability of the information exchange.

The four mechanisms allow to check connected devices and determine whether the connection and the devices are still up and running or not. Reactions on failures are a local matter of applications.

The two mechanisms for Client/Server are:

• Keep-alive in TCP connections (used by MMS) [RFC 1122]
• Reporting of Report Control Block attribute “IntgPd” [ACSI, 7-2]

The two mechanisms for Publisher/Subscriber (Layer 2 multicast) are:

• Time-Allowed-to-Live in GOOSE messages (next message in) [8-1]
• Sample Rate in SMV message (sampled measured value) [ACSI,7-2]

The following figure shows how Integrity Period (communicated in a Report message) could be used to cyclically inform the client that the Reporting mechanism is up and running. Integrity Period is often used in cases where events happen very seldom and where the client wants to have a “heart beat” from the reporting server.

The configuration of the DataSet and the Report Control Block is usually provided by a SCL file. In the case of SystemCorp’s IEC 61850 API it is easily be done by uploading the corresponding SCL File to the IED, e.g., the Beck IPC IEC61850@CHIP.

Click HERE to evaluate the “Keep-Alives” with Reporting and GOOSE and real software running under Windows.

Click HERE in case you are looking for education and training about the possibilities, philosophy, and details of IEC 61850 and IEC 61400-25.

I have educated more than 2.500 people from more than 60 countries and more than 600 companies. More to come … see you soon.

IEC61850@CHIP - Flyer

A new flyer from Beck IPC, SystemCorp and NettedAutomation explains the architecture of the IEC61850@CHIP. The platform is very powerful, offering a lot of integrated functions, modules, and services like TCP/IP, SSL, IPSec, HTTP(S) server, IEC 61850, CAN Bus, IEC 61131-3, ...

Additional components like the following ones are available for EASY integration ... because the integration is already done by Beck IPC:

More to come
Click HERE for the flyer [pdf, 0.9 MB].

Does IEC 61850 require special Ethernet Switches?

NO and Yes! It depends which services you are looking for. The communication profiles for client/server and GOOSE messaging are defined in IEC 61850-8-1.

The services and protocols for client/server communication are defined in the "TCP/IP T-Profile":

This table shows that the mandatory services and protocols are "standard" Ethernet ... you may purchase in a shop round the corner. There is no need for a special Switch etc.

In case you want to run GOOSE (or sampled values) messages, this requires IEEE 802.1Q (VLAN and Priority Tagging):

DataLink: Priority Tagging/ VLAN IEEE 802.1Q  (mandatory)

Ethernet Switches for rugged applications in substations have also to conform to IEC 61850-3 (EMC, EMI, Temp range, ...).

For applications outside substations you may use "standard" Ethernet switches.

How to Secure the Smart Grid Network Infrastructure?

Andrew K. Wright, Paul Kalv, and Rodrick Sibery have published an excellent paper with the title "Interoperability and Security for Converged Smart Grid Networks".

The conclude: " While modern computing and technologies are now widely used throughout control centers and utility enterprise environments, field communications equipment largely uses outdated technologies. By deploying a converged smart grid network, utilities like ... can modernize their communications infrastructure, deploy new applications such as AMI and Distribution Automation, and adopt an architecture that is based on standards and supports interoperability based on Internet Protocol. Interoperability will allow them to replace individual subsystems that become out of date as technology evolves, without requiring forklift upgrades. Converged smart grid networks will require strong logical separation of traffic to ensure security of smart grid applications, and this will be best provided by a defense-in-depth architecture that considers security across all layers of the IP stack."

Click HERE for downloading the excellent paper [pdf, 1.5MB]

Recall the following statement I posted the other day "NAMUR expects that this clear statement and the requirements formulated will enable all those involved in the standardisation process to work together constructively with a view to achieving a converged [added by Karlheinz - Wireless Fieldbus] standard.")

Click HERE for the discussion of the Wireless Fieldbus (NAMUR, ...).

From the view point of information models, configuration Language, information exchange services and (IP-based) protocols we have reached a very high level of convergence with IEC 61850 - including the security measures as defined in IEC 62351.