http://www.danzigercartoons.com/ |

The cost -
risk trade-off that grid operators need to make is complex. Costly risk
reducing adjustments to the grid need to be weighed against the rise in cost of
operating the network and therefore the rates consumers pay for using the grid.
For making the trade-off, an estimate of the probability of failure of an asset
is required. In most cases, specific analytical models are developed to
estimate these probabilities. Using pipeline characteristics like type of material,
age, and data on the environment the pipeline is in (i.e. soil type,
temperature and humidity) pipeline specific failure rate models can be created.
Results from inspections of the pipeline can be used to further calibrate the model.
Due to the increased analytics maturity of grid operators, these models are
becoming more common. Grid operators are also starting to incorporate these failure
rate models in the creation of their maintenance plans.

#### Averaging the Risk

As you can
probably imagine, there are many ways for constructing failure rate models.
This makes it difficult for a regulator to compare reported asset conditions
from the grid operators, as these estimates could have been based on different
assumptions and modelling techniques.
That is why, in the UK at least, it was agreed between the 4 major gas
distribution networks (GDN), to standardise the approach. In short, the method can be described as
follows.

- Identify the failure modes of each asset category/sub group in the asset base and estimate the probability of failure for each identified failure mode.
- For each failure mode the consequences of the failure are identified, including the probability of the consequence occurring.
- For each consequence the monetary impact is estimated.
- By summing up over all failure modes and consequences, a probability weighted estimate of monetised risk for an asset category/sub group is calculated. Summarising over all asset categories/sub groups gives a total level of monetised risk for the grid.

This new
standardised way of calculating risks makes the performance evaluation much easier,
it also allows for a more in-depth
comparison. See for more details on the method the official documentation.

An
interesting part of this new way of reporting risk is the explicit and
standardised way of modelling asset failure, consequence of asset failure and
cost of the consequence. This is similar to how a consolidated financial
statement of a firm is created. Therefore, you could interpret it as a
consolidated risk statement. But can risks of individual assets or asset groups
be aggregated in the described way and provide a meaningful estimate of the total
actual risk? The above described approach sums the estimated (or weighted
average) risk for each asset category/sub group, so it’s an estimate of the
average risk for the complete asset base. However risk management is not about
looking at the average risk, it’s about extreme values. For those who read Sam
Savage’s The Flaw of Averages or Nassim Taleb’s Black Swan know what I’m
talking about.

#### Risking the Average

Risks are
characterised by extreme outcomes, not averages. To be able to analyse extreme values,
a probability distribution of the outcome you’re interested in is required.
Averaging reduces the distribution of all possible outcomes to a point estimate,
hiding the spread and likelihood of all possible outcomes. Also, averaging
risks ignores the dependence between each of the identified modes of failure or
consequence. To illustrate let’s assume that we have 5 pipelines, each with a
probability of failure of 20%. There is only one consequence (probability =1)
with a monetary impact of €1,000,000. The monetised risk per
pipeline than becomes €200,000 (=0,20*€1,000,000), for the total grid it is equal to €1,000,000. If
we take dependence of the failures into account than there will be a 20%
probability of all pipes failing when these are fully correlated events. There
will be a 0,032% change of all pipes failing if they are fully independent. The
estimated financial impact than ranges from €1,000,000 in
the fully correlated case to €1,600 in the fully independent case.
That’s quite a range which isn’t visible in the monetised risk approach.

Regulators
must assess risk in many different areas. Banking has been top of mind in the
past years, but industries like Pharma and Utilities also had a lot of attention.
How a regulator decides to measure and asses risk is very important. If risks
are underestimated, this could impact society (like a banking crisis, deaths
due to the admission of unsafe drugs or increase of injuries due to pipeline
failures). If risks are overestimated costly mitigation might be imposed,
again impacting society with high costs. The above example shows that the monetised
risk approach is insufficient as it estimates risk with averages, where in risk
mitigation the extreme values are much more important. What than is a better
way of aggregating these uncertainties and risks than just averaging them?

#### Monte Carlo Simulation

The best
way to better understand the financial impact of asset failure is to construct
a probability density function of all possible outcomes using Monte Carlo simulation
and based on that distribution make the trade-off between costs and risk. Monte
Carlo Simulation has proven its value in many industries and in this case will
provide what we need. Using the free tools of Sam Savage’s probabilitymanagement.org
the above hypothetical example of 5 pipe lines can be modelled and the
distribution of financial impact analysed. In just a few minutes the below
cumulative distribution (CDF) of the financial impact for the 5 pipelines case can
be created. Remember that the monetised risk calculation resulted in a risk
level equal to the average, €1,000,000.

From the graph it
immediate follows that P(Financial Impact<=Monetised Risk) = 33%. It implies
that the P(Financial Impact>Monetised Risk) = 1-33%=66%. So, a 66% chance that
the financial impact of pipe failures will be higher than the calculated monetised
risk. Therefore we’re taking a serious risk by using the averaged asset risks. Given
the objective of better comparison of grid operator performance and enabling risk
trading between asset groups, the monetised risk method is to simple I would
say. By averaging the risks, the distribution of financial impact is rolled up
into one number leaving you no clue on what the actual distribution looks like (See
also Sam Savage’s : The Flaw of Averages) A better way would be to set an
acceptable “risk threshold” (say 95%) and use the estimated CDF to determine
the corresponding financial impact.

This
approach would also allow for better comparison of grid operators by creating a
cumulative distribution for all of them and plotting them together into one
graph (See example below). In a similar way risk mitigations can be evaluated and
comparisons made between different asset groups, allowing for better informed
risk trading.

Standardising
the way in which asset failures and consequence of failures are estimated and monetised
definitely is a good step towards a comparable way to measure risk. But risks
should not be averaged in the way the monetised risk approach suggests. There
are better ways, which will provide insight on the whole distribution of risk. Given
the available tools and computing power, there is no reason not to do so. It will
improve our insights on the risks we face and help us find the best mitigation
strategies to reducing public risks.

_{}

^{}