How To Bring Plant Engineers To The Table When Cyber Issues Are Discussed?

In my career as electrical and IT engineer I have experienced that engineers are quite often not invited to discuss the measures and plans for critical infrastructure protection with IT personnel.

It is completely different compared to the world of electric power system protection - I mean the applications of protection relays. Protection engineers are (in my understanding) the most crucial engineers. They are very important for the reliable delivery of electric power. Protection engineers are likely to attend any meeting when it comes to the reliability of the power flows. Protection engineers know what to do ... software people may help to implement the "what" and the IT personnel may help to solve the communication issues ... but the crucial parts are dominated by protection engineers!

Mr. Vytautas Butrimas, a globally well known engineer involved in cyber security of control systems has briefly discussed the "Berlin wall" between IT personnel and plant engineers.  

Click HERE for the four page paper written by Mr. Butrimas.

Either of the groups involved believes that his or her group is the center of universe. There is little communication between the IT personnel and the engineers. 

There are so many semipermeable walls between, e.g., politicians, company lawyers, economists, IT experts, and plant engineers. There is usually no way that experts from any layer are allowed to talk to the experts from the other layers. In the end: Each layer feels independent of the other layers ... which leads to what we see these days ... and may be even more in the future. Have you heard of a discussion between a power protection engineer and a lawyer or even a medical doctor?

It would help medical doctors to understand the basics of electric power system reliability ... and so on. Because medical doctors (and all other people of a society) depend 100% on available power.

So in the end: (Electrical) Engineers should be honored by the society ... the problem may be that the engineers are not wearing white coats but wear safety boots, safety helmets, goggles,  protective gloves, ... a single doctor may harm a few people ... a protection engineer may harm millions of people during a blackout caused by a misconfiguration of protection equipment.

Power Outage In Frankfurt Area (Germany) - And People That Need A Breathing Ventilator

I just read that in the western part of the city Frankfurt (Main, Germany) the electric power was down for more than 10,000 customers. A current transformer (CT, for measuring the current) blow up ... and produced a lot of smoke. The power went off from 17:15 on Tuesday 2021-10-26. The restoration took some eight hours!

The Hessenschau (de) reported that nine (9!) people that depend on breathing ventilators had been hospitalized. This critical situation tells us, that the ventilators did likely not have battery backup power - either in the devices or external. The devices we use for my wife have both two internal batteries which give (rated!) power for 16 hours for each device.

This brings two questions up in my mind:

1. Why is it not required by law that all breathing ventilators have battery power for at least 24 hours?
2. Why don't we have to have external batteries and inverters that would give power for several days?

Instead of bringing patients with the ambulances to the hospital, it would be much easier (faster and cheaper) to bring an emergency power supply package (batterie plus inverter) to the patients! Or?

There seems to be a wide area of improving the quality of life.

By the way, why did the CT (current transformer) crash? Was it too old or not ... or? I hope that my friend Andrea Bonetti (one of the most experienced protection engineers on this planet) will comment on the importance of CTs!

Any comment?

Add on (2021-10-30): 

First: The utility has told that more than 100 workers are involved in fixing the problem ... the current fix is provisionary only! 100+ workers means: It must be a big problem that needs so many people to fix. 

Second: It was reported that in an elderly care home the nurses had to use their mobile phone's flash light to look for the elderly people ... no emergency light! Hmm ... strange. A few 12V batteries and some 12V LEDs would have done a good job! Cheap and useful ... lifesaving! ... if somebody would care for their maintenance. The management has obviously decided to purchase a hand lamp per floor ... 👍 something is better than nothing. Note: A battery leak (AA or AAA batteries) may damage a flash light that is not used often ... or only in case of emergency. Non leaking batteries are available: Lithium Batteries are the right choice for emergency devices. I have replaced the typical AA and AAA batteries with Lithium batteries for all flashlights and outdoor devices like thermometer ... they withstand cold weather and do not leak ... life time likely 10+ years ...

Click HERE for the extended Hessenschau (de) report.

Basics For The Future Electric Power Systems – Some Thoughts

Smart Grids, Energiewende, renewable energy, ... Digitalisierung (digitization) der Energiewende, ... are buzzwords for some years. What is really needed for the future electric power systems?

Against the background of the ongoing discussions by engineers, lawyers, politicians, other people ... with a degree in the social or natural sciences, graduate engineers, medical doctors, lawyers, and graduates of business studies and economics, it may be helpful to look at the real needs that are related to the physics ...

I have written a one page document:

Basics for the future electric power systems – some thoughts 

It starts (de and en):

"Das zentraleuropäische Verbundnetz UCTE ist eine der größten Maschinen (oder gar bezüglich der Leistungsfähigkeit der Energiebereitstellung weltweit die größte Maschine) auf unserem Planeten. Bezüglich der Ausdehnung wird dieses elektrische Netz nur von dem inzwischen weltweit ausgedehnten Kommunikationsnetz übertroffen. Alle diese technischen Netze folgen dabei strikt physikalischen Gesetzen – egal, von wem sie und zu was sie genutzt werden oder auch welche Ideen am Markt gerade en vogue sind..."

"The central European interconnected network UCTE is one of the largest machines (or even the world's largest machine in terms of the performance of energy supply) on our planet. In terms of expansion, this electrical network is only surpassed by the communications network that has now expanded worldwide. All these technical networks strictly follow physical laws - regardless of who uses them and for what they are used or which ideas are currently in vogue on the market..."

Click HERE to download the above mentioned document [pdf, 108 KB de/en]

Some Information About My Personal Situation

You may be surprised about the fact that I posted very few information on this blog for some time. Let me explain our family situation here at home. I received an email today from a colleague and follower of this blog. This email encouraged me to publish excerpts from the emails we exchanged and to add some thoughts at the end of this post:

Email I received on 2021-10-16:

"Dear Karlheinz,

Perhaps you remember me, I've long been a keen follower of your blog on IEC 61850, and we have exchanged e-mails a few times in the past. ... I am writing to you because 

I remain looking forward to any news from you ... Kind Regards ..."

Email I sent on 2021-10-16:

"Dear ...,

I am sorry to read these lines ... 

I have changed my work (now being retired) to nurse my wife 24/7 … she was diagnosed ALS in January 2017. We are happy that the Lord Jesus has helped us to do that work together with one of our daughters. My wife needs non-invasive ventilation around the clock … we are still together … which is very good.

So, as a consequence I have more or less stopped my business …

Have a great weekend!

Best Regards,
Karlheinz"

Email I received on 2021-10-24:

"Dear Karlheinz,

I am very sorry to learn that you stopped your business because your wife is seriously ill.

To be honest I had realized that the frequency of your posts had declined in the last months... Your blog was a wonderful source of information and it will be missed by the smart grid community. Your posts were real, technical, relevant, useful stuff and not the usual buzzword-laden, over-hyped bullshit we are all used to read these days.

Given the circumstances, I believe no one will say you didn't do what was your duty.

I wish all the best to you and your family."

Some Thoughts to think about:

• Have you ever thought about the electric power in your personal life or the life of the people around you 👪?
  
  
• What could we do to help keeping the power flow? It is more than IEC 61850 ... 
  
  
• Smart(er) Grids are nice ... more important is the need for power in situations where the public grid is down ... In our case we need power for the breathing ventilator 24/7, power for the lifter to lift my wife from chair to wheel chair ... to bed ... power for the nursing bed (move it up and down), power for light, power for heating and cooking, ... and so on. It is more than emergency power ... we need at least some power 24/7 ... we use just PV, batteries and inverters, ... 
  
  
• Did you know that (in Germany) we had in 2019 some 4,100,000 care-dependent people that needed nursing ... 3,310,000 nursed at home (usually in families), 818,000 in nursing homes, ... even in nursing homes it is not required to have emergency power !! ... some have. What about the many homes that need electric power 24/7 to survive ... I guess most of them believe that there is always power ... except for a few minutes per year ... 
  Click HERE for the source of the German nursing situation.
  
  
• One of the most crucial challenges in the future is to provide permanent available power to those homes that nurse people ... 
  
  
• Try to contact your neighborhood, your police station, nearby hospital, fire station, power utility, ... to figure out how and where you could get power to operate the breathing ventilator or other medical devices in case of a blackout! You may be surprised what you get ... almost nothing.
  
  
• It seems to be more important (or interesting) to install hundreds of super charger stations along the German Autobahn ... than to care about a minimum of power to help care-dependent people to survive.
  
  
• We need all-over more electric storages ... small and big ones ... for various applications ... in order to have electric power whenever and wherever we need some ...
  
  
• Let me know what you think ... or report about your experiences ... Thanks.

Updated Mains Frequency Information Service Has Been Launched

Please not that the updated Mains frequency information service has been launched yesterday. 

The mains frequency info service is now active. 

You will now receive messages about unusual mains frequency behaviour.

For information on the messages, please refer to the website www.pc-projekte.de.

The website offers non-commercial information newsletter for the continental European network RG-CE (UCTE)

In this test group, you can register for an automated newsletter "Netzfrequenzinformation Verbundnetz RG-CE (UCTE)", which will provide you with the latest information on network frequency measurements.

Experience has shown that the measuring systems recognize and report approx. 4-6 messages per week on a daily basis.

You can find much more information about the grid frequency HERE

This service is a very huge offer to the power systems communities in Europe. I haven't seen anything comparable ... and all for free!

UPDATE 2021-01-23: EXTREMELY Crucial Frequency Deviation Of The European Electrical Interconnected Grid (UCTE)

 Dear All,

You may have heard that on January 08, 2021 Europe was hit by very serious power problems that entered almost to a very big blackout.

Message time : 08.01.2021 14:05:12 MEZ
Message : UCTE low frequency
Mains frequency : 49,746 Hz
Mains load difference : -3741,4 MW

There are many publications on the Internet ... please search for UCTE blackout January 2021 ...

Click HERE for the official news.

UPDATE 2020-12-10: Crucial Frequency Deviation Of The European Electrical Interconnected Grid (UCTE)

Dear All,

Due to the increasing number of frequency events I have decided to recommend you the following ... it is too much work for me to post the newest events ... sorry for that:

1. Register to receive the event reports directly from one reliable source 

A look at the following private website (offered by an experienced senior electrical engineer) provides very informative and easy to use information (right part on the figure):

www.pc-projekte.de (the content can easily be translated - with Google)

You may register to receive messages like the one above:

https://pc-projekte.lima-city.de/ucte-netzfrequenz-infodienst.html

Note that these messages carry just privately generated information about specific events of the frequency of the ENTSO-E Network in Continental Europe (former UCTE).

2. Check the following website:

https://www.saurugg.net/blackout/risiko-eines-strom-blackouts/aktuelle-situation#netzfrequenz 

There you can find a list of frequency events ... 135 in 2020 already !!

UPDATE 2020-11-10: Crucial Frequency Deviation Of The European Electrical Interconnected Grid (UCTE)

 Please find another frequency deviation detected on 2020-11-10

Message time : 10.11.2020 06:00:52 MEZ
Message : UCTE high frequency
Mains frequency : 50,135 Hz
Mains load difference : 1916,6 MW

UPDATE late 2020-11-05: Crucial Frequency Deviation Of The European Electrical Interconnected Grid (UCTE)

 Please find another frequency deviation detected on late 2020-11-05

Message time : 05.11.2020 20:01:28 MEZ
Message : UCTE low frequency
Mains frequency : 49,889 Hz
Mains load difference : -1548,7 MW

UPDATE 2020-11-05: Crucial Frequency Deviation Of The European Electrical Interconnected Grid (UCTE)

Please find another frequency deviation detected on 2020-11-05

Message time : 05.11.2020 06:01:36 MEZ
Message : UCTE high frequency
Mains frequency : 50,122 Hz
Mains load difference : 1717,4 MW

UPDATE 2020-11-03: Crucial Frequency Deviation Of The European Electrical Interconnected Grid (UCTE)

 Please find another frequency deviation detected on 2020-11-02.

Message time : 02.11.2020 21:01:14 MEZ
Message : UCTE low frequency
Mains frequency : 49,896 Hz
Mains load difference : -1441,3 MW

Ukrainian Power Grids Cyberattack - A Forensic Analysis Based On ISA/IEC 62443

Three Ukrainian power distribution companies sustained a cyberattack in western Ukraine on 23 December 2015. As the forensic information is extensive from a technical point of view, it is an opportunity to put 

ISA/IEC 62443-3-3
In­dus­tri­al com­mu­ni­ca­ti­on net­works - Net­work and sys­tem se­cu­ri­ty
- Part 3-3: Sys­tem se­cu­ri­ty re­qui­re­ments and se­cu­ri­ty le­vels

to the test with a real-life example. Several sources were used for this purpose that, overall, provide unusually detailed information.

Click HERE for the report "Ukrainian Power Grids Cyberattack - A Forensic Analysis Based On ISA/IEC 62443" ... worth to read!

Click HERE for a white paper on the series IEC 62443

Click HERE for a preview of the standard:

Crucial Frequency Deviation Of The European Electrical Interconnected Grid (UCTE)

The frequency of the European Interconnected Electrical Grid (UCTE) is under stress. The frequency should be 50 Hz ... as you know. Click HERE for an UCTE background paper. Click HERE for details and basics of the Nordic Grid.

The network frequency is uniform in a power supply network and, apart from minor deviations from the nominal value, constant over time. The frequency changed today and earlier more than what usually is expected.

Here is some information about todays situation at 11:00 a.m.:

Time : 21.10.2020 11:02:35 MESZ
Message : UCTE low frequency
Mains frequency : 49,886 Hz
Mains load difference : -1594,6 MW

UPDATE 2020-11-03
Message time : 02.11.2020 21:01:14 MEZ
Message : UCTE low frequency
Mains frequency : 49,896 Hz
Mains load difference : -1441,3 MW

UPDATE 2020-11-05

Message time : 05.11.2020 06:01:36 MEZ
Message : UCTE high frequency
Mains frequency : 50,122 Hz
Mains load difference : 1717,4 MW

UPDATE 2020-11-05/2

Message time : 05.11.2020 20:01:28 MEZ
Message : UCTE low frequency
Mains frequency : 49,889 Hz
Mains load difference : -1548,7 MW

UPDATE 2020-11-10

Message time : 10.11.2020 06:00:52 MEZ
Message : UCTE high frequency
Mains frequency : 50,135 Hz
Mains load difference : 1916,6 MW

-------------------------------------------------

A look at the following private website (offered by an experienced senior electrical engineer) provides very informative and easy to use information (right part on the figure):

www.pc-projekte.de (the content can easily be translated - with Google)

You may register to receive messages like the one above:

https://pc-projekte.lima-city.de/ucte-netzfrequenz-infodienst.html

Note that these messages carry just privately generated information about specific events of the frequency of the ENTSO-E Network in Continental Europe (former UCTE).

The diagram on the left part is derived from:

https://gridradar.net/netzfrequenz.html

Hope you will find this post informative.

VDE-Studie "Lösungsansatz für Zellulares Energiesystem"

In dem neuen Papier „Zellulares Energiesystem“ zeigt jetzt der Technologieverband VDE eine effiziente und konsensfähige Lösung für eine erfolgreiche Umsetzung der Energiewende: Die VDE-Experten empfehlen den Strom direkt dort zu verbrauchen, wo er erzeugt wird, nämlich auf lokaler und regionaler Versorgungsebene. Bei diesem „zellularen Ansatz“ erfolgt die Umsetzung der dezentralen Energieversorgung auf Basis zellularer Strukturen.

Hier zum kostenlosen Download der Studie clicken.

Die Studie verdient vollen Respekt! Der holistische Ansatz verbindet viele der bisher versprengten, wenig aufeinander abgestimmten Lösungen zu einem ganz Neuen (dem zellularen Ansatz). Herzlichen Glückwunsch!

Der zellulare Ansatz wirkt teilweise futuristisch ... man darf natürlich auch mal träumen! Konkrete Leitlinien für einen Teil der Bürger unserer Gesellschaft sind allerdings rar und insbesondere der Aspekt der „überlebensdienlichen“ Notwendigkeiten scheinen etwas unterbelichtet zu sein. So heißt es auf Seite 26 : „Der Inselnetzbetrieb im zellularen Energiesystem soll nur im Notfall eine Option sein. Im Normalfall sind der Verbundbetrieb und die Solidarität die Regel.“

Es müssen unbedingt „überlebensdienliche“ Lösungen in Form von kleinen (vorübergehend) autarken, kostengünstigen Systemen (beispielsweise PV-System mit ein paar kWp, einem Batteriewechselrichter und ein paar Batterien) implementiert werden, um im wirklich großen oder kleinen Notfall Überlebenschancen für viele – besonders für Schwache – zu bieten.

Hintergrund für diese Position:

Meine Frau benötigt 24/7 ununterbrochene nicht-invasive Beatmung durch zwei Geräte mit jeweils 4-h-Akku Notfallversorgung. Ein Gerät für den Tag und eines für die Nacht. Ein 10-stündiger Stromausfall würde wahrscheinlich ... unvorstellbar.

Beatmungsgeräte für invasive beziehungsweise nicht-invasive Beatmung sind zig-tausendfach im Einsatz! Manche gehen von mehreren 100.000 aus!

Auf der anderen Seite haben wir eine netzgekoppelte PV-Anlage mit knapp 10 kWp ... soweit so gut. Die nützt uns im Notfall nicht, weil sie nur mit dem öffentlichen Netz zusammen funktioniert.

Bei einem Netzsaufall würden unsere PV-Module nutzlos auf dem Dach liegen und sich „sonnen“ ohne einen Sonnenbrand zu bekommen ... sie würden allerdings weiter Schutz für nistende Tauben bieten.

Momentan überlegen wir, wie wir die Module im Notfall manuell auf einen Batterie-Wechselrichter umschalten könnten. Das würde helfen, einen Teil der für Notfälle benötigte Energie vom Dach zu ernsten ... der Salat und die Tomaten im Garten würden ohnehin auch ohne Strom wachsen. Wasser können wir per Handpumpe aus dem Rheingraben pumpen.

Wir alle sollten mal überlegen, wie wir diese oder eine ähnliche Notfalllösung unter die Leute bringen könnten ... das wäre zwar nicht netzdienlich – aber auf jeden Fall auch gemeinschaftsdienlich! Geladene Batterien und mobile Inverter könnten auch an die Nachbarn oder ... ausgeliehen werden.

In unserem Fall haben wir zwei Notstromaggregate, Benzinvorräte und ein paar Bleiakkus mit zugehörigen Invertern. Für den Notfall würde ich gerne einfach einen Teil der Module manuell auf einen Batteriewechselrichter (vielleicht 2 kWp) umschalten. Wenn das Netz wieder verfügbar ist, dann kann man wieder zurückschalten.

Noch eine kleine Nebenbemerkung zum Thema IEC 61850 auf Seite 38:

„Für die Kommunikation und das Datenmanagement im Prozessnetz haben sich in den letzten Jahren
u.a. die IKT-Standards IEC 60870 (Datenmodelle für Energiemanagement) und IEC 61850 (Kommunikation) etabliert.“

Hier wurden offensichtlich die beiden Normen verwechselt! IEC 61850 bietet Datenmodelle ... u.v.a. mehr und IEC 60870 bietet nur Kommunikation.

Beim Thema „Kommunikationsstandard“ haben wir mittlerweile mit den vielen neuen Ansätzen bei IEC 61850 eine hinreichende Basis, um die meisten notwendigen Informationsaustauschmechanismen ((Funktions)Modelle, Dienste und Protokolle) implementieren zu können. DER-Modelle (61850-7-420) werden zurzeit beispielsweise am KIT bei der Realisierung mehrerer Forschungsprojekte verwendet ... das sind erfolgsversprechende Ansätze!

Ich würde mich freuen, wenn der Aspekt „überlebensdienlich“ im hier beschriebenen Usecase in Zukunft in einfache technische und sinnvolle Lösungen einen zielführenden Niederschlag finden würde!

Dieser Vorschlag, den ich einigen Experten aus dem VDE-Umfeld vorgestellt habe, hat bereits nach zwei Tagen viele positive Rückmeldungen bewirkt!

How to ring the front door bell in case of power outage?

Today I had a phone call with a friend discussing what happens when we suffer a blackout. I was sitting in my home office in the basement when the door bell was ringing. I asked my friend: What would happen, when we would have a power outage here at home?

He sent me a nice picture that I used as an instruction to install a very useful (new) SuperCat bell push at our front door:




If you push the left button ... Auuuuutsch ... we would hear you crying ... haha.

Nice blackout-resisting door bell. I have removed it later ;-)

Additional Information on the EPEX Problems

Please note additional Information on the EPEX problems introduced in:

http://blog.nettedautomation.com/2019/07/1-mwh-electric-energy-may-cost-37856.html

Additional details can be found here:

http://www.epexspot.com/de/presse/news-archive/details/news/Root_cause_for_the_incident_of_07_June_2019

http://www.epexspot.com/de/presse/press-archive/details/press/EPEX_SPOT_assesses_course_of_events_and_lessons_learned_from_the_Day-Ahead_incident_

1 MWh Electric Energy May Cost 37.856 Euro - You Don't believe it?

Belief me, this is true: You could have sold 1 MWh electric power to the German Grid for 37.856 Euro the other day (used for primary frequency control) - if you would have had the opportunity to sell it on the spot market ...

We have our roof PV system some 10 kWp since Sept 2016. The energy produced since then is 25 MWh ... this would sum up to 25 x 37.856 Euro = 946.400 Euro !! WOW !! Unfortunately I did not have the opportunity to influence the German electricity market and to make such a sale ...

See some news in German regarding the situation.

Something went wrong on the 6., 12. and 25. June 2019.

The belief in the Market to fix everything ... may end up in a big blackout.

Add-On (2019-07-03):

Today I found more details on the reasons why we were so close to big trouble:

"Due to a faulty data package, the European electricity
exchange EPEX in Paris decoupled the European
electricity market on June 7, 2019. This caused a great
deal of excitement on the markets. Johannes Päffgen,
Head of Energy Trading at Next Kraftwerke, explains the
causes and consequences in an interview.
Christian Sperling: Johannes - What happened? Why
was there so much trouble at EPEX on the Friday before
the Whitsun holidays?
Johannes Päffgen: Well - in the end it's a computer error... but we should go into that later. At about 11:40 this Friday we noticed that something was wrong at EPEX. We couldn't place any more bids for the day-ahead electricity auction on Saturday. ..."

Clicke HERE for the full report.

I guess it was a human error ... somebody didn't take into account that corrupted data packages will be sent and received ... how could a faulty package have such a dangerous result?!?!

Unbelievable.

Stay tuned for more information once available.

Dangerous Situation in the European Electric Power System Caused by "frozen" Measurements

Measurements of power flow (Watts in export or import) are very crucial for Load Frequency Controller … wrong (i.e., “frozen”!) measurements have caused almost a big blackout in Europe in January 2019.
What happened: the measurement of power of the lines between two transmission systems (Germany – Austria) were frozen when the export value of 723 MW from Germany to Austria was measured (which was a result of 34 GW wind power generation in Germany). Later the wind power generation decreased to 4 GW … and the measurement (as input to the controller) many hours later still used the input value of 723 MW !!! In such a meshed power network it is unlikely that such a value is constant …

Oops … something went absolutely wrong!

Report by exception (on a value change as used for the above measurement) is great … as long as there are changes figured out and reported. A frozen value does not cause a change and thus no new value will be reported … No receiver should expect that the export power is constant (723 MW) for days!! The sensors may have worked fine … but the software and communication failed … on both sides (sender and receiver). A receiver should not trust that the software and communication is working fine all time.
Here are some measures to monitor the communication (by the receiver) to figure out if the communication is OK:

1. Ping (in case of TCP/IP) (if no response after some time: raise flag)
2. TCP Keep-alive (if no keep-alive message in t bigger keep-alive: raise flag)
3. Polling by receiver (if no response after some time: raise flag)
4. Periodic reporting (if no report in t greater period: raise flag)
5. In case of no message received in a configured time period (in case of using IEC 61850 Reporting) the receiver should check if the report control block is enabled and is using the correct configuration values like trigger option, …
6. Check if the sequence of received values are plausible
7. Use redundant systems (comm, …)
8. …

Check out the official Entso-E report (with links to more details):

https://www.entsoe.eu/news/2019/05/28/entso-e-technical-report-on-the-january-2019-significant-frequency-deviations-in-continental-europe/

This reminds me on the Boeing 737 MAX disaster … maybe a programmer left the vendor of the load frequency controller and hired with Boeing … I am kidding.

How many programmer or people that configure power control systems and communication systems that lack experience with complex systems like a plane or a power system. Where are the “grey-hair” experts that would tell you in minutes how to … ? They may enjoy the beach with warm water and sun shine – relax and spend the pension for ...

It is not sufficient to have no ideas – one should also be unable to implement them.

I expect that more of these problems will hit the street once we have far more control, monitoring and communication in the smart(er) grids of medium and low voltage. Note that the problem in January 2019 occurred at transmission level!! … where more resources (higher budgets) are available (in the past).

Have a great weekend – with power.

Could the Utility Industry learn from the Boeing 737 Max Disaster?

The other day I read the IEEE article:

How the Boeing 737 Max Disaster Looks to a Software Developer

https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer 

Everybody working in the power utility world SHOULD study this paper in detail - and take some time thinking about it and plan and implement consequences ... by some means or other.

We are in danger to end up here:

I have slightly modified a quotation from the above paper:

“Long ago there was a joke that in the future control centers would control themselves, and the only thing in the control room would be an operator and a dog. The operator’s job was to make the customers comfortable that someone was in the control room. The dog’s job was to bite the operator if he tried to touch anything.”

This 737 Max problem is a symptom of a trend that is happening in many domains: power production, power delivery, ... even education (from Kindergarten to university).

A senior developer sent me the other day the following link with the subject:

When management thinks 100 trainees can do work of 3 fully qualified senior developers

https://youtu.be/JeB19gvJcxs

This statement and clip is very very true - in my training business with more than 4,500 attendees all over I have experienced that in some cases training (regards IEC 61850 and other subjects) had to be paid by the attendees ... using annual vocation! Unbelievable! HR has a big (often negative) impact on the training of the employees as well. The complexity of, e.g., IEC 61850 is usually totally underestimated! ... no need for a training ... read the myriad of papers and study slides ... that is quite often the recommendation of the management and HR.

This understanding is widespread in the utility domain, too ... students are hired (for low wages) to investigate and figure out how new technologies (especially digitalization) could be used ...

I hope that the utility industry will wake up and do a better job than the people at Boeing - but it will cost a lot of money ... shareholders and customers may not want to spent.

The European electric power system is under more stress since January 2019 ... see the following link (first German and second English text):

Switzerland was on the brink of a blackout on May 20, 2019:

https://gridradar.net/schweiz_blackout_entgangen.html

On the left side bar you can see that we faced three additional critical situations (two in January and one in April 2019)!

One reason behind all this is the "market driven" power delivery ... more and more relying on software that processes a lot of data to get forecasts and set schedules for the energy flow. Hope that this software is better than ...

We all rely 24/7 on electric power. Have you thought about the possibility of local, regional or total blackouts? How would your life change?

I am nursing my wife here at home. She needs ventilation 24/7. The two ventilators have each a battery good for four hours ... so a blackout of 10 hours would mean that my wife ... For that reason I have several batteries, two emergency generators and some 30 liter gasoline.

I hope that not many "old grey-hairs are sitting in the corner" - BUT walking around and helping the young people to understand why we have the electric power system as it is now - developed in more than 130 years. Senior experts like Gregory Travis (author of the IEEE article) are very rare ... and not well understood ... and maybe too expensive for the bean counters.

The cumulative experience of the "old grey-hairs" (many retired years ago) that has been collected between the 60s and 90s and that are still involved one way or the other should have reasonable influence on young engineers ... in order to keep the power flowing.

In some time down the road we may have 200 trainees on the play field ... and no "old grey-hair" left. In this case the "w" in the German "Energiewende" will drop - means Energie-Ende" ... end of energy.

I am one of these "old grey-hair" engineers (66). Just a few young people are listening to me - I have to stay at home 24/7 ... no travel anymore ... One of the young people listening to me is our grand-daughter (20, finishes her bachelor in EE and IT this year). She is really eager to learn from my experience with MAP, Fieldbus, UCA, IEC 61850, Modbus, ... power systems, renewables, ... As an intern at a distribution company she transfers part of my knowledge to the utility - it is cheaper for the utility than to renew the contract with me ... ;-)

Maybe they will contract with me for the second time - but then it may be too late.

What happens during a blackout - Comprehensive Report of the German Parliament's study

The OFFICE OF TECHNOLOGY ASSESSMENT AT THE GERMAN BUNDESTAG
published in 20111 a very comprehensive report:

What happens during a blackout - 
Consequences of a prolonged and wide-ranging power outage

"THE COMMITTEE’ S PREFACE
Infrastructures such as a reliable energy supply, functioning water-supply and wastewater-disposal systems, efficient modes of transport and transport routes and also information technology and telecommunications technology that can be accessed at all times represent the lifeblood of high-technology industrialised nations. The Committee on Education, Research and Technology Assessment therefore commissioned the Office of Technology Assessment at the German Bundestag (TAB) to investigate the possible effects of a prolonged and widespread power blackout on highly critical infrastructures such as drinking water, wastewater, information and communications systems, financial services and health services, especially against a backdrop where the blackout has a cascading effect spanning state and national boundaries.
In Germany, several recent natural disasters and technical malfunctions (Elbe and Oder floods in 2002/2005, power blackout in the Münsterland in 2005, the Kyrill storm in 2007) have highlighted the population’s dependence on such (critical) infrastructures. Supply bottlenecks, public safety problems and disruptions to road and rail transport have revealed the vulnerability of modern societies and made extreme demands on health, emergency and rescue services...."

Click HERE for the 250 page report [English].
Click HERE for the German version.

The report is one of the best descriptions I have seen. It is really worth to read, to understand and to follow.

If you want to understand what power outages could mean to a society (in a warm region - not in c(o)ld Germany), study the following reports:
Click HERE for the report "Puerto Rico 'heartbreaking' five weeks post-storm"
Click HERE for the report "Puerto Rico Struggles With Power Recovery ..."
Click HERE for further information

I hope something like that will not happen during winter time in Germany.
Note that we have more than natural disasters: Man-made aging infrastructures and aging workforce.