Methodology for identifying hazard scenarios to assess the
resilience of critical infrastructure
Ioanna Ioannou1 Willy Aspinall1 Christian Bouffier2 Elisabete Carreira3 Daniel Honfi4 David Lange 4 Laura Melkunaite5 Nina Kristine Reitan6 Tiziana Rossetto1 Karolina Storesund6 Rui Teixeira 7
1. EPICentre, Department of Civil, Environmental & Geomatic Engineering, UCL, London 2. Ingénieur géotechnique INERIS, France.
3. INOV, Lisbon, Portugal.
4. SP Technical Research Institute of Sweden, Borås, Sweden.
5. Research and Development Department, DBI, Copenhagen, Denmark. 6. SP Fire Research AS, Trondheim, Norway.
7. DAS - Divisão de Águas e Saneamento, Barriero’s Municipality, Portugal.
Deliverable Number: D2.1
Date of delivery: November 30, 2015
Month of delivery: M18
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement no. 653390
Coordinator: David Lange at SP Sveriges Tekniska Forskningsinstitut (SP Technical Research Institute of Sweden)
Table of Contents
1 Executive Summary 4
2 Introduction 5
3 Literature Review 6
3.1 Ports 6
3.2 Potable water network 10
3.3 Motorway network 12
3.4 Determination of hazard scenarios 13
4 Methodology for risk-related scenario identification 17
4.1 Introduction 17
4.2 Proposed Methodology 18
4.2.1 Step 1: Identification of the components of the critical infrastructure. 18
4.2.2 Step 2: Identification of risk-related hazard events 19
4.2.3 Step 3: Draft questionnaire 21
4.2.4 Step 4: Workshop with stakeholders 22
4.2.5 Step 5: Completion of questionnaire 22
4.2.6 Step 6: Analyses of answers 23
4.2.7 Step 7: Feedback workshop 23
4.2.8 Step 8: Report of findings 23
4.3 Concluding remarks 24
5 The Øresund Region 25
5.1 General information 25
5.1.1 Natural hazards 25
5.1.2 Technological hazards 26
5.2 Results 27
5.2.1 Ranking of natural hazards 28
5.2.2 Ranking of operational hazards 30
6 The Port of Oslo 34
6.1 General information 34
6.2 Hazards 35
6.2.1 Ranking of natural hazards 36
6.2.2 Ranking of operational hazards 37
6.3 Results 39
6.3.1 Ranking of natural hazards 40
6.3.2 Ranking of operational hazards 43
7 The A31 Highway, France 46
7.1 General Information 46
7.2 Hazards 47
7.2.1 Natural hazards 48
7.2.2 Operational hazards 50
7.3 Results 54
7.3.1 Ranking of natural hazards 54
7.3.2 Ranking of operational hazards 56
8 Water network in Barreiro, Portugal 58
8.1 General information 58
8.2 Hazards 62
www.improverproject.eu 2
8.3 Results 63
8.3.1 Ranking of natural hazards 63
8.3.2 Ranking of operational hazards 67
9 Concluding Remarks 73
Appendix 1 – Questionnaire for Øresund region 75
1. Introduction 76
2. Expert elicitation method 78
3. Instructions 78
4. Participant’s Statement 82
5. Personal details and experience 83
6. Natural hazards 85
7. Technological/human hazards 88
Appendix 2 – Questionnaire for Port of Oslo 91
1. Introduction 92
2. Expert elicitation method 94
3. Instructions 95
4. Participant’s Statement 98
5. Personal details and experience 99
6. Natural hazards 101
7. Technological/human hazards 104
Appendix 3 – Questionnaire for A31 Highway 107
1. Introduction 108
2. Expert elicitation method 110
3. Instructions 110
4. Participant’s Statement 114
5. Personal details and experience 115
6. Natural hazards 117
7. Technological/human hazards 120
Appendix 4 – Questionnaire for Barreiro’s Water Network 123
1. Introduction 124
2. Expert elicitation method 128
3. Instructions 130
4. Participant’s Statement 133
6. Natural hazards 136 7. Operational hazards at the distribution system of the potable water
network. 139
www.improverproject.eu 4
1
Executive Summary
Critical infrastructure is exposed to a wide range of hazards, capable to disrupt its operations in various degrees. This raises the question of which hazard scenario an operator shall use to assess the resilience of their critical infrastructure asset. Various techniques aiming to prioritize the various risks are commonly used in the literature. This study proposed an 8-step methodology, which aims to rank the risks of pre-defined hazard scenarios by eliciting the opinions of the stakeholders through a structured expert elicitation technique termed paired comparison. The novelty of the proposed technique is its ability to quantify the degree of disagreement regarding the ranking order of the scenarios and thus to capture the uncertainty associated with these risks.
The proposed methodology has been applied to four living labs, namely: the Oresund region, the port of Oslo, the A31 Highway in France and the potable water network in Barreiro. The applications aims to rank scenarios of natural and operational hazards according to their disaster- and emergency-risk. Despite the small number of participants, the results provide an excellent basis for further discussion regarding the most likely disaster or emergency risk scenarios. For most living labs, the ranking of the hazards using paired comparison was successful in identifying the scenarios associated with the highest risk. Overall, ranking the natural hazards according to their disaster- or emergency-risk has been associated with a higher degree of consensus than the ranking of the operational hazards reflecting on the higher complexity and perhaps the limited understanding of the later.
In more detail, snow storm is the hazard with the highest disaster risk for the A31 Highway. Similarly, earthquake is the hazard with the highest disaster risk for the water network in Barreiro. Three meteorological hazards ranked the highest for both the likelihood to occur and to cause disaster to the Øresund region. By contrast, the ranking of the hazards for the port of Oslo identified several scenarios with similar likelihood to cause disaster, which ranked very different in their likelihood to occur in the next 5 years. This raises question as to whether the most of least likely to occur scenarios is most suitable which can be answered in collaboration with the stakeholders.
With regard to the operational hazards, the contamination of the water in the water source or the distribution network due to an accident at the high-risk industrial SEVECO operations has been identified as the single scenario with the highest risk of disaster for the water network in Barreiro. Three events including a multiple day strike and two accidents in the wet bulk terminal have been identified as having the highest disaster risk for the port of Oslo. By contrast, no operational hazards can be identified as having the highest risk of occurrence for the A31 highway and the Øresund region.
2
Introduction
Critical infrastructure consists of the assets or systems, which are essential for the functioning of the society and economy. There is an increasing concern about the security and resilience of critical infrastructure given that their destruction or disruption can affect the well-being and the security of citizens. In general, critical infrastructure is exposed to a wide range of hazards, i.e. events capable to disrupt its operations in various degrees. Some of these hazards are well-understood, and some are unknown. Given this uncertain and complex reality, how can an operator decide on which hazard type to use in order to assess the resilience of a given critical infrastructure? So far, the assessment of the resilience of a critical infrastructure is typically based on a well-defined hazard scenario. The literature often concentrates on natural hazards, such as earthquakes and hurricanes, given the significant disruption that they have caused on critical infrastructure in the recent past. Similarly, in the aftermath of 9/11 and the rise of terrorist attacks in the Western World, there is an increased interest in assessing the resilience of critical infrastructure to such attacks.
This study concentrates on developing a generic rigorous methodology to identify suitable risk-based scenarios for various hazard classes, which can be used to assess the resilience of the critical infrastructure. IMPROVER and therefore this study concentrates on three critical infrastructure systems, namely: ports, potable water and motorway networks. In what follows, a literature review of the main hazards that can affect these three types of critical infrastructure is presented. A thorough review of the literature on procedures for selecting hazard scenarios for resilience assessment is then presented. Finally, a detailed outline of the proposed methodology is provided, followed by its application to the four living labs, namely: the port of Oslo, the Oresund crossing, the water network in Barreiro and the A31 Highway which connects France with Luxembourg.
www.improverproject.eu 6
3
Literature Review
Critical infrastructure is exposed to a wide range of hazards, which can vary in nature. So far, there is no widely accepted way to classify the hazards. Thus, for the needs of this study, hazards are divided into natural and non-natural. Natural hazards refer to the naturally occurring rapid or slow events, which can be geophysical (e.g., earthquakes, volcanoes), hydrological (e.g., floods), climatological (e.g., extreme temperatures, droughts), meteorological (e.g., cyclones, storm surges), astrophysical (e.g., solar surge) and biological (e.g., epidemic). The non-natural hazards are further subdivided into three classes which include malicious human hazards (e.g., terrorists or criminal attacks, sabotage), operational (e.g., technological, organizational hazards, human error) and market/political/economic hazards. The malicious human hazards category is included due to the increased concern regarding the resilience of critical infrastructure to these type of hazards in the aftermath of 9/11. The operational hazards refer to disruptive events caused by the people, processes, system failures etc. The latter hazard class is added as it acknowledges the fact that critical infrastructure is part of the supply chain and wider political events (e.g., war, sanctions) or market shocks (e.g., bankruptcy of an operator) can have a profound impact on their importance and functions. A brief description of the main hazards that can affect the three examined types of critical infrastructure (i.e., ports, potable water ad motorway networks) follows. A review of approaches adopted in the past to determine suitable scenarios for the assessment of the resilience of critical infrastructure is carried out and presented.
3.1 Ports
A port is a location on a coast or a shore which consists of one or more (natural or artificial) harbours where ships can dock in order to transfer people and goods to national and international destinations1. Their location is selected in order to optimize the access to the sea and land. A port is a complex system which consists of the waterfront structures (e.g., breakwaters, piers, etc.), the cargo handling and storage components (e.g., cranes, tanks, tunnels and pipelines etc.), the utility systems (e.g., electric power system, water and waste-water systems, natural gas, etc.) and buildings (e.g., sheds, warehouses, passenger terminals, etc.) and relies on the railway and highway infrastructure to transport goods and passengers to and from the land2. Ports are considered essential for the functioning of a society given that over 90% of the global trade is transported by ship3. Economic globalization has led to the substantial increase in the volume of transported goods in the past two decades4. There are 94
1
Dwarakish G.S., Salim A.M. (2015). ‘Review on the Role of Ports in the Development of a Nation’. International Conference on Water Resources, Coastal and Ocean Engineering, 4: 295-301.
2
SYNER-G (2009). ‘Systemic seismic vulnerability and risk analysis for buildings, lifeline networks and infrastructure safety gain’. Available at: http://www.vce.at/SYNER-G/pdf/deliverables/D2.07_Definition%20of%20system%20components.pdf Accessed: 7 Jun 2016.
3
Maritime Knowledge Centre (2012). ‘International shipping facts and figures – Information resources on trade, safety, security,
environment’. Available at:
http://www.imo.org/en/KnowledgeCentre/ShipsAndShippingFactsAndFigures/TheRoleandImportanceofInternationalShipping/D ocuments/International%20Shipping%20-%20Facts%20and%20Figures.pdf Accessed at: 7 June 2016.
4
main EU ports, and in 20145 these were used by 402 million passengers and handled 3.8 billion tons of seaborne goods. With regards to the type of goods, liquid bulk goods represent 37% of the total tonnage of cargo, dry bulk goods represent 23% of the total cargo tonnage and 21% are containerized goods. Over 50% of the shipped goods as well as 5-10% of the containers handled in ports are hazardous or contain noxious products6, increasing the potential impact of small incidents on safety and the environment.
Ports are exposed to a wide range of hazards which threaten to affect their integrity or disrupt their functions. Ports are increasingly seen as integrated components of the supply chain7,8. This means that the impact of port disruptions propagates into the supply chain. Given their importance, there is a wealth of studies which assess the risk and resilience of ports to a number of hazards.
Natural hazards can disrupt the operation of ports in the short-term and, depending on the intensity of the event, its services can also be reduced in the long-term. The risk to ports posed by severe weather conditions has been the focus of most existing studies9. This could be attributed to the increasing concern regarding the impact of climate change, which is expected to lead to more frequent and severe weather and to rising sea-levels10. In 2012, Hurricane Sandy resulted in widespread damage and disruption to the port of New York and New Jersey, and caused the Port’s closure for nearly one week. Studies9
also concentrated on the seismic risk assessment of ports, in the aftermath of the 1995 earthquake in Kobe, which forced the port to close for 2 years due to repairs and led to the dramatic reduction of its capacity11. The 2011 tsunami in Japan, which affected 319 ports and caused losses of approximately US$12 billion12, led to a number of studies that focus on the tsunami risk assessment of ports. Apart from the aforementioned hazards which have lead to natural disasters in the recent past, additional potential threats to the safe access to the port or its operations13, such as sedimentation, the floating of water debris and the excessive presence of organisms or silts, have also been identified.
5
EUROSTAT (2016). ‘Maritime ports freight and passenger statistics’. Available at: http://ec.europa.eu/eurostat/statistics-explained/index.php/Maritime_ports_freight_and_passenger_statistics#Increase_in_volumes_of_seaborne_goods_and_passenge rs_in_EU_ports Accessed: 4 Jun 2016.
6
Charlier R.H. (2001) "Hazardous goods and their environmental impact". International Journal of Environmental Studies, Vol. 58, pp. 271-285.
7
Carbone V., DE Martino M. [2003]. ‘The changing role of ports in supply-chain management: an empirical analysis’. Maritime Policy and Management, Vol. 30, No. 4, pp. 305–320.
8 Berle Ø., Asbjørnslett B. E., and Rice J. B. (2011). ‘Formal Vulnerability Assessment of a maritime transportation system’. Reliability Engineering & System Safety, Vol. 96 n.6, 2011, pp. 696-705.
9 Curning R.O.S. (2011). ‘Maritime disruptions in the Australian-Indonesian wheat supply chain: An analysis of risk assessment and mitigation strategies’. PhD thesis, Australian Maritime College.
10
Becker A. H., Acciaro M., Asariotis R., Cabrera E., Cretegny L., Crist P., Esteban M., Mather A., Messner S., Naruse S., Ng A. K. Y., Rahmstorf S., Savonis M., Song D.-W., Stenek V., Velegrakis A. F. (2013). ‘A note on climate change adaptation for seaports: a challenge for global ports, a challenge for global society’. Climatic Change, Vol. 120, No. 4, pp. 683-695.
11
Chang S. E. (2000). 'Disaster and transport systems: loss, recovery and competition at the Port of Kobe after the 1995 earthquake'. Journal of Transport Geography, vol. 8, no. 1, pp. 53-65.
12
Muhari A., Charvet I., Tsuyoshi F., Suppasri A., Imamura F. (2015). ‘Assessment of tsunami hazards in ports and their impact on marine vessels derived from tsunami models and the observed damage data’. Natural Hazards, 78:2, 1309-1328.
13 Maritime Safety Authority of New Zealand (2005). Environmental factors affecting safe access and operations within New Zealand ports and harbours’. Available at:
https://www.maritimenz.govt.nz/Publications-and-forms/Commercial-www.improverproject.eu 8
In the post-9/11 era, an increasing interest can be noted regarding the mitigation of malicious human-induced hazards9,14, e.g., terrorist attacks, arson or sabotage. In 2008, Helmick15 highlighted the need for a systematic and coherent research agenda, aiming to enhance the port or maritime security. Nonetheless, von Winterfeldt and Rosoff16 explored the impact of a dirty bomb by constructing three scenarios involving radioactive bombs with three levels of intensity (i.e., low, medium and high). Nair et al.17 developed three terrorist scenarios, accounting for the uncertainty in the exact characteristics of the event. It should be noted that in some studies malicious human-induced hazards are considered of the same importance for the safety of the port as natural hazards 17,14.
Port functions can be disrupted by operational risks, which include the risk from the interaction between the people and the assets of the port or the ships. These risks can be subdivided to technological, human and organisational factors18. Access and network factors have also been identified19. Table 3.1 shows examples for each factor. Disruptions up or down the supply chain can lead to port disruptions. For example Loh and Thai20 examined the impact of rail or road disruptions how they affected the container terminal.
Curning21 produced a detailed and thorough literature review of studies that describe operational hazard events leading to port disruption and affecting the supply chain. The 8 main, more frequently cited, causes of port disruption are, port strikes, port congestion, equipment breakdowns, and customs and administration.
operations/Ports-and-harbours/Environmental-Factors-affecting-safe-access-and-operations-within-NZ-Ports-and-Harbours.pdf Accessed: 7 June 2016.
14
GAO Report to Congressional Committees (2007). ‘Port Risk Management: Additional Federal Guidance would aid Ports in disaster planning and Recovery’. GAO-07-412. Available at: http://www.gao.gov/new.items/d07412.pdf
Accessed 8 June 2016. 15
Helmick J. S. (2008). ‘Port and maritime security: A research perspective’, Journal of Transportation Security, Vol. 1, No. 1, pp. 15-28.
16
von Winterfeldt D. and Rosoff H. (2007). "A Risk and Economic Analysis of Dirty Bomb Attacks on the Ports of Los Angeles and Long Beach". Published Articles & Papers. Paper 39.
17
Nair, R., Avetisyan, H., and Miller-Hooks, E., 2010, Resilience Framework for Ports and other Intermodal Components, Transportation Research Record, Vol 2166/2010, pp 44-65.
18
Mansouri M., Nilchiani R., Mostashari A. (2010). ‘A Risk Management-based Decision Analysis Framework for resilience in Maritime Infrastructure and Transportation Systems’. Marine Policy, Vol. 34, No. 6, pp. 1125-1134.
19
Grainger A., Achuthan K. (2014). ‘Port resilience: A primer’. Nottingham University Business School. Available at:
http://eprints.nottingham.ac.uk/2279/1/PortResilience-primer%281-0%29.pdf Accessed: 8 June 2016. 20
Loh H. S., Van Thai V. (2014) ‘Managing Port-Related Supply Chain Disruptions: A Conceptual Paper’. The Asian Journal of Shipping and Logistics, Vol. 30, No. 1, pp. 97-116.
21 Curning R.O.S. (2011). ‘Maritime disruptions in the Australian-Indonesian wheat supply chain: An analysis of risk assessment and mitigation strategies’. PhD thesis, Australian Maritime College.
Table 3.1 Summary of hazards to ports
Hazard Events
Natural Earthquake Ground motion
Tsunami Volcano Ash Meteorological Ice Fog Snowstorm Storm surge Hurricane Lightning Hydrological Higher tidal level
Higher sea-level Other
Environmental
Sedimentation Water borne debris
Excessive presence of organisms/silt Unexploded WII ordinance
Malicious Human-Induced Arson
Terrorist Attack Sabotage Cyber Attack
Operational Organizational Congestion (berth, gate, storage)
Bureaucracy
Poor quality of training course Communication failure Land or marine assess hazards Official inspections
Technological Port equipment failure Lack of IT maintenance
Lack of navigational system maintenance
Human Strikes of port workers
Decision making error Operating error Market/economy/political Shortage of demand
Shortage of ships Financial crisis
National political unrest Geopolitical crisis
There is significant literature aiming to study the frequency and the causes of accidents in ports and propose ways to reduce their frequency and impact. A historical analysis of 471 accidents in ports worldwide by Darbra & Casal22 identified the release of materials as the most frequent accident followed by fires, explosions and gas clouds. More than half of the accidents occurred during transport (e.g., collision of ships, train/rail accident) and a significant percentage during handling and storing of materials. Human error is also identified as a notable factor to these accidents and maritime accidents in general23. An analytical framework has been determined in order to identify the most important human factors (e.g.,
22
Darbra R.-M., J. Casal (2004). ‘Historical analysis of accidents in seaports’. Safety Science, Vol. 42, No. 2, pp. 85-98. 23 Hetherington C., Flin R., Mearns K. (2006). ‘Safety in shipping: The human element’. Journal of Safety Research, Vol.
www.improverproject.eu 10
supervision error vs decision-making error) contributing to a single accident in detail, using the judgement of the marine accident surveyors and a pairwise comparison approach24. Finally, studies9 also accounted for the port disruptions due to the national or global political events and market events. Political riots, financial or geopolitical crisis and war25 can all have a detrimental effect on port functions. Similarly, piracy26 affects maritime transport. The loss in demand or the shortage of ships27 can also affect port operations.
Overall, there is considerable research on assessing the risks to ports from a large number of hazards. Nonetheless, a recent survey28 identified that port stakeholders prepare mainly for frequent events that can delay or moderately disrupt the operations of the port, and that although they are aware of the perhaps catastrophic consequences for the low-frequency-high consequences events these are not accounted for in their preparedness plans.
3.2 Potable water network
A potable water distribution system is a spatially distributed complex network of nodes, i.e., pipes junctions, pumps, treatment plants and reservoirs connected with pipes. The network can be divided into two subnetworks: the transmission network, which transfers water from the spring to the treatment plants and the distribution network, which transfers the treated water to the consumers for everyday or industrial use. Hazards affecting water networks can interrupt the water supply; deteriorate the drinking water quality or both. In general, hazard events can lead to a number of undesirable consequences such as loss of power, loss of communication, loss of supervisory control and data acquisition, service disruption, reduced workforce, contamination of the water and economic disruptions. The type of water contamination29 has been a subject of special concern in the literature as it determines the response strategy. Four types of water contamination have been identified, namely: microbiological, chemical, physical and radiological.
Natural hazards such as earthquakes, tsunamis, hurricanes or droughts can cause large-scale damage to the water distribution system, which can affect the water supply and quality. For example, in 2005, hurricane Katrina lead to the lack of fresh water and poor planning
24
Celik M., Cebi S. (2009). ‘Analytical HFACS for investigating human errors in shipping accidents’. Accident Analysis & Prevention, Vol. 41, No. 1, pp. 66-75.
25
Novati M., Achurra-Gonzalez P., Foulser-Piggott R., Bowman G., Bell M.G.H., Angeloudis P. (2015). ‘Modeling the effect of port disruptions: Assessment of disaster impacts using a cost-based container flow assignment in liner shipping networks’. Transportation Research Board 94th Annual Meeting.
26
He R. (2009). ‘Coast guards and maritime piracy: sailing past the impediments to cooperation in Asia’. The Pacific Review, Vol. 22, No. 5, pp. 667-689.
27
Curning S., Cahoon S. (2010). ‘The cycles of maritime disruptions in the Australian-Indonesian wheat supply chain’. 10th International Conference ‘Research and Development in Mechanical Industry’, Serbia.
28
Berle, Ø., Asbjørnslett, B. E., Rice J. B. (2011), ‘Formal Vulnerability Assessment of a maritime transportation system’. Reliability Engineering & System Safety, Vol. 96 n.6, pp. 696-705.
29
Dwi (2014). ‘Private Water Supply Risk Assessment - Explanatory Notes for Risk Assessment tool Version 4’. Available at:
hampered the emergency response to reach the affected people30. The pumping stations were out of order due to the power loss or the floodwater that followed the hurricane. Excessive cracks in the pipelines also lead to the contamination of the water and the lack of pressure. Two weeks after the hurricane, only 30% of the potable water system had been restored31. In 2007, a wildfire in San Diego, USA severely damaged remote facilities of the water network32. In 2011, a drought in Houston, Texas, led to 700 breaks of main pipes per day due to the consolidated soil33. Climate change is of major concern regarding the resilience of water systems given that they can be severely affected by the change in the rain patterns and the extreme weather.
With regard to operational factors that can affect the water supply network, so far the most detailed list of scenarios for this type of hazard likely to affect a water supply network is produced by the TECHNEAU project34. The latter also propose a methodology for assessing the risk to water systems from operational hazards and provide guidance on identifying hazard scenarios.
Ensuring the security of water supply networks to terrorist attacks is highly topical in current literature35. A US white paper on water security was published in 201036, which tackled the issue of vulnerabilities to the water supply network when faced with a malicious water contamination scenario aiming to high casualties. The vulnerability of the British water networks to terrorist attack, based on the use of water-soluble biological or chemical contaminants, has also recently been accessed37
.
Other market, economic or political factors can also affect the water network. For example, privatization is seen to have a negative impact on the quality of the water supply network38.
30
EPA (2011). ‘Planning for an emergency drinking water supply.’ United States Environmental Protection Agency. Available at:
https://www.epa.gov/sites/production/files/2015-03/documents/planning_for_an_emergency_drinking_water_supply.pdf
Accessed: 13 June 2016. 31
Tanali I.R., Harrald J.R. (2006). ‘EFFECTS of WATER INFRASTRUCTURE FAILURE ON RESPONSE CAPABILITIES AFTER HURRICANE KATRINA’. Proceedings of 3rd
Magrann conference, Rutgers University, New Brunswick, New Jersey, USA.
32
ASCE. (2008). Recovery Practices Primer for Natural Disasters. Produced under the Water Infrastructure Security Enhancements (WISE) Initiative by the American Society of Civil Engineers (ASCE), American Water Works Association (AWWA), and the Water Environment Foundation (WEF).
33
Llanos M. (2011). 700 water main breaks in Houston - a day. Today – NBC News, 16 August 2011. [Online]. Accessed 6 June 2016, http://www.today.com/id/44160515/ns/today-weather/t/water-main-breaks-houston-day/#.U57mtfldU1I .
34
Beuken R., Sturm S., Kiefer J., Bondelind M., Strom J., Lindhe A., Losen L., Petterson T., Machenbach I et al. (2007). ‘Identification and description of hazards for water supply systems - A catalogue of today's hazards and possible future hazards’. TECHNEAU report Available at: https://www.techneau.org/fileadmin/files/Publications/Publications/Deliverables/D4.1.4.pdf
Accessed: 9 June 2016. 35
EPA (2003). ‘Large Water System Emergency Response Plan Outline: Guidance to Assist Community Water Systems in Complying with the Public Health Security and Bioterrorism Preparedness and Response Act of 2002.’ United States Environmental Protection Agency. Available at: https://www.epa.gov/sites/production/files/2015-03/documents/erp-long-outline.pdf Accessed: 13 June 2016.
36
Kroll D., King K., Engelhardt T., Gibson M., Craig K. (2010). ‚ TERRORISM VULNERABILITIES TO THE WATER SUPPLY AND THE ROLE OF THE CONSUMER: A WATER SECURITY WHITE PAPER‘ Water world. Available at:
http://www.waterworld.com/articles/2010/03/terrorism-vulnerabilities-to-the-water-supply-and-the-role-of-the-consumer.html Accessed 23/11/2016.
37
Utility week (2015). ‘UK water networks ‘vulnerable to terrorist attack'’. Article available at: http://utilityweek.co.uk/news/uk-water-networks-vulnerable-to-terrorist-attack/1150512#.WDVKtvkrJnI. Accessed 23/11/2016
38
Abou-Seada, M., Cooper, C., Ghaffari, F., Jones, R., Kyriacou, O. and Simpson, M. (2004). ‘The Economic Consequences of Accounting in the English and Welsh Water Industry: A non-shareholder perspective’. AMICUS Industry Briefing, August.
www.improverproject.eu 12
Pollution due to practices in agriculture, industry or recreational activities in nature have been responsible for water contamination in the supply network, and scenarios where backflow of an industrial site leads to water supply contamination has been considered in the literature39.
3.3 Motorway network
The road network consists of roads, bridges and tunnels and is essential for the inland transport of passengers and goods. In 2011, the length40 of the EU road network is 5,525,168.5km and includes motorways (1%), main or national roads (5%), secondary roads (28%) and other roads. The density of EU motorways reaches 17km per 1,000 km2 of area. The density of passenger cars is approximately 3,500 per km in the EU. There are 1,693 billion tons per km transported in 2012, a 33% increase since 1995. It should be mentioned that although some studies focus on the risk or resilience assessment of a highway network41 in its totality, others concentrate solely on bridges42 or tunnels43.
Natural hazards can affect the functions of motorway networks and, in some cases, affect their integrity. For example, the 1995 Kobe earthquake caused the collapse of the Fukae (Hanshin Expressway) bridge. The heavy rain that caused extensive flooding at the Greater Manchester on Boxing Day in 2016 also caused a sinkhole to appear on the M62 motorway.
Ooperational hazards44 affecting a motorway network include mainly accidents between vehicles or motorbikes as depicted in Table 3.2. The 1999 Mont Blanc accident is one of the most well-known disaster. A heavy goods accident caught fire in the tunnel which took 3 days to be extinguished and lead to 38 deaths. Bridges can be severely affected by the lack of appropriate maintenance. In 1967, the Point Pleasant road bridge collapsed during a period of heavy traffic killing 46 people. More recently, in 2007, the Interstate -35 westbound bridge over the Mississippi River in Minneapolis during the evening rush hour, killing 13 people.
39
DWI (2005). ‘Drinking water inspectorate- Guardians of drinking water quality. Brief guide to drinking water safety plan’. Available at: http://www.dwi.gov.uk/stakeholders/guidance-and-codes-of-practice/Water%20Safety%20Plans.pdf Accessed: 13 June 2016.
40
European Union Road Federation (2015). Yearbook 2014-2015. Available at: http://www.erf.be/images/Statistics/BAT-AD-Stats-2015Inside-ERF.pdf Accessed: 4 Jun 2016.
41UK Ministry of Transport [2014]. ‘Transport Resilience Review: A review of the resilience of the transport network to extreme weather events.’ Available at: https://www.gov.uk/government/publications/transport-resilience-review-recommendations Accessed: 25 July 2016.
42
Bocchini, P., D. M. Frangopol [2012] "Optimal Resilience- and Cost-Based Postdisaster Intervention Prioritization for Bridges along a Highway Segment", Journal of Bridge Engineering, Vol. 17, No. 1, pp. 117-129.
43 Rinaudo P., Paya-Zaforteza I., Calderón P. A. [2016] "Improving tunnel resilience against fires: A new methodology based on temperature monitoring", Tunnelling and Underground Space Technology, Vol. 52, pp. 71-84.
44
Highway Agency (2015). ‘M4 Junctions 3 to 12 Smart Motorway: Hazard log and hazard log report, Annex E.’ Available at:
https://infrastructure.planninginspectorate.gov.uk/wp-content/ipc/uploads/projects/TR010019/TR010019-000464-7-4-EDR-Annexes_E-Hazard-Log.pdf Accessed: 14 June 2016.
Table 3.2 Summary of operational hazards to motorways
Hazard Events
Operational Vehicles reversing up entry slip roads. Unsafe lane changing in the slip road. A group of vehicles driving too fast. Roadworks: long-term, short-term. Individual vehicle driving too fast. Driver fatigue.
Rapid change of general vehicle speed. Motorcycle filter though lane
Driver loses control of their vehicle. Unsafe lane changing.
Tail gating.
3.4 Determination of hazard scenarios
Critical infrastructure is exposed to a wide range of hazards. Some of these hazards are well understood and some are unknown. Central to the assessment of infrastructure resilience in the literature is the determination of one or more suitable disruptive hazard scenarios. These scenarios are determined either by a hazard-specific or an all-hazards approach. The two approaches briefly described below are complementary.
The hazard-specific approach is favored in the literature and can be used in cases where the hazards are well understood or have emerged as important. According to this approach, the likelihood of occurrence of a given hazard event as well as the scale of its impact are quantified. How does the literature identify which hazard type to consider?
The hazards, in this approach, are pre-specified without a rigorous risk-prioritization methodology, which clearly identified them among other hazards as the most relevant for the critical infrastructure.
At national level, resilience of critical infrastructure is mostly considered as a goal or a part of risk management. For this reason, scenarios from pre-specified hazards are developed. In particular, the resilience of critical infrastructure to natural hazards in UK guidelines45 is based on the development of a plausible worst-case scenario for each natural hazard that is of concern. The Australian government provides guidelines in order to improve the resilience of its critical infrastructure to terrorism46 using a risk-informed approach, which includes the determination of specific threats (e.g. espionage, terrorism etc), the assessment of their likelihood to occur and the identification of critical assets and their vulnerability.
With regard to resilience assessment of specific critical infrastructure assets, the use of pre-specified hazards of interest, mainly natural hazards or terrorism, has been noted. For
45
Cabinet Office [2011]. ‘Keeping the country running: Natural Hazards and critical infrastructure. A guide to improving the resilience of critical infrastructure and essential services’. Available at:
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/61342/natural-hazards-infrastructure.pdf. Accessed: 20 July 2016.
46
Australia-New Zealand counter-terrorism committee [2015]. ‘National guidelines for protecting critical infrastructure from terrorism.’
www.improverproject.eu 14
example, the resilience of critical infrastructure (e.g., water network47, hospitals48,49) to earthquakes has been assessed. Other studies concentrated on the impact of pre-specified hazards and accounted for the uncertainty in the realization of a given scenario with regard to its location, intensity and impact. For example, Nair et al.50 assessed the resilience of ports by developing a flooding scenario and three terrorist attack scenarios accounting for the uncertainty in the nature of each scenario. Similarly, Ouyang et al.51 used two hazard types: operational hazards and hurricanes and accounted for the uncertainty in the realization of each hazard in their resilience methodology.
An alternative approach has been noted in the literature where hazards has been identified among a wealth of scenarios through a risk-prioritization methodology. Key in this approach is the quantification of the risk (i.e., the likelihood of occurrence and the severity of its impact) of a given scenario. For example, the resilience of critical infrastructure framework developed by Biringer et al52 allows for the use of past or plausible hypothetical scenarios and requires the assessment of their risk. Mansouri et al.18 propose a more systematic methodology for the identification of suitable scenarios for the assessment of port resilience. Scenarios based on natural, organizational, technological or human hazards are identified. Then one or more of these scenarios are selected according to the stakeholders’ priorities. The risk-profile of each scenario is determined by a risk factor which is the product of the likelihood of its occurrence times its impact. In their application, the likelihood and the impact are discrete factors in the scale from 1 to 5. Despite the understanding that there is a need for consultation with experts in order to quantify the risk factor, in the application, the risk factors have been determined by the authors. Similarly, TECHNEAU34 propose a procedure to identify suitable hazards for a water network. According to this approach, the main components of the system are determined and hazard scenarios that can affect each component are constructed based on a combination of a ‘bottom-up’ and ‘top-down’ approach. The ‘bottom-up’ approach involves interviews with experts who identify hazard events thought to be significant for the assessed network. Then a workshop is organized and the experts are asked to state their opinions on whether the pre-identified scenarios are important, and comment on their initial judgement. It should be mentioned that the experts invited at the workshop include specialists with deep knowledge of relevant processes and parts of the system, people with overall knowledge and overall view of the system, as well as non-technical participants.
The World Health Organization53 also propose a simplified risk assessment procedure. Central to this procedure is the organization of one or multiple day workshops with
47
Rose A., Liao S.-Y. [2005] "Modeling Regional Economic Resilience to Disasters: A Computable General Equilibrium Analysis of Water Service Disruptions*", Journal of Regional Science, Vol. 45(1), pp. 75-112.
48
Bruneau M., Reinhorn A. [2007]. “Exploring the concept of seismic resilience for acute care facilities.” Earthquake Spectra, Vol. 23(1), pp. 41–62.
49
Cimellaro, G. P., Reinhorn, A. M., and Bruneau, M. (2010). “Seismic resilience of a hospital system.” Structural Infrastructure Engineering, Vol. 6(1), pp. 127–144.
50
Nair R., Avetisyan H., Miller-Hooks E. [2010]. ‘Resilience Framework for Ports and other Intermodal Components’. Transportation Research Record, Vol 2166/2010, pp 44-65.
51
Ouyang M., Dueñas-Osorioa L.,Min X. (2012). ‘A three-stage resilience analysis framework for urban infrastructure systems.’ Structural Safety 36-37: p 23-31.
52
Biringer B., Vugrin E., Warren D. [2013]. ‘Critical infrastructure system security’. CRC Press. 53
Bartram J., Corrales L., Davison A., Deere D., Drury D., Gordon B., Howard G., Rinehold A., Stevens M. [2009]. ‘Water-safety plan manual: step-by-step risk management for drinking-water suppliers.’ World Health Organization, Geneva.
brainstorming sessions aimed at identifying hazard scenarios for a given water network. The scenarios are then ranked according to whether they are considered to be ‘significant’, ‘uncertain’, or ‘insignificant’. Expert elicitation is proposed by Kleindorfer and Saad54
in order to identify scenarios of accidents that can disrupt the normal activities of the supply chain. Identification of hazard scenarios using expert judgement as well as empirical data is also proposed by FEMA55 in order to mitigate the impact of major threats to communities. The impact in the community is also considered in terms of the number of fatalities, injuries, impact on critical infrastructure, the geographical area affected, the number of displaced households, the amount of direct economic impact, and the economic effect due to the disruptions of the supply chain.
Similarly, the Ministry of Public Safety and Solicitor General of British Columbia56 propose a hazard, risk and vulnerability tool kit for assisting communities in their response and emergency planning. Central to this toolkit is the appointment of an advisory committee who, with the aid of experts, are responsible for the identification and ranking of hazard events. The hazards include natural hazards (e.g., volcanic eruptions, hydrological hazards or earthquakes), accidents (e.g., plane crash, oil spills ect.) and malicious human induced hazards (e.g., terrorism or riots). Their ranking is based on a qualitative scheme that first assesses the consequences of each event, and then the likelihood of its occurrence. The consequences concern seven categories: fatalities, injuries, critical facilities, lifelines, property damage, environmental, and economic and social. The assessment of the consequences of a given hazard event includes the assignment of a number ranging from 1 (very low consequences) to 4 (very high).The assessment of the likelihood also includes the assignment of a number 1 for very rare events (every 200-300 years) to 6 for frequent of very likely events (i.e., every 1 to 3 years). The stakeholders are invited to a workshop in order to review the results and comment on the proposed response measures and the emergency planning. A methodology57 to prioritise operational maritime risks has been proposed based on fuzzy logic. The methodology includes the identification of the hazards using brainstorming techniques as well as the evaluation of the consequences of the hazards. Finally, a risk assessment for ports58 involves the construction of a risk matrix which assists in the identification of critical hazard scenarios.
The hazard-specific approach is based on the assumption that the hazards are well understood. However, for the majority of networks and critical infrastructure there exist many unknown hazards and failure modes59. Unlike the assessment of risk, the assessment of resilience aims to prepare the critical infrastructure for expected as well as unexpected hazards60. In the case
54
Kleindorfer P. R., Saad G. H. [2005]. ‘Managing Disruption Risks in Supply Chains’. Production and Operations Management, Vol. 14, No. 1, pp. 53-68.
55
FEMA [2011]. ‘Threat and hazard identification and risk assessment guide. Comprehensive Preparedness guide 201. 56
Ministry of Public Safety and Solicitor General [2004]. ‘Hazard, risk and vulnerability analysis tool kit.’ British Columbia. 57
Dourmas G.N., Nikitakos N.V., Lambrou M.A. [2008]. ‘A methodology for rating and ranking hazards in maritime formal safety assessment using fuzzy logic.’ Reliability and Risk Analysis Theory and Applications.
58
SENES Consultants Limited [2009]. ‘Point Lisas port development risk assessment.’ 59
Park J., Seager T. P., Rao P. S. C., Convertino M., Linkov I. [2013]. ‘Integrating Risk and Resilience Approaches to Catastrophe Management in Engineering Systems’. Risk Analysis, Vol. 33, No. 3, pp. 356-367.
60
Holling C. [1973]. ‘Resilience and Stability of Ecological Systems’. Annual Review of Ecology and Systematics, 4, 1-23. Retrieved from http://www.jstor.org/stable/2096802
www.improverproject.eu 16
of resilience assessment, the literature also shows a preference for an all-hazards approach. According to this approach, the scenario is focused on the consequences and not on the hazard event which has triggered them61. For example, Omer et al.62 developed hypothetical scenarios to assess the resilience of ports. Their scenarios determine the level of capacity reduction of the port’s functions over 3 days.
Both hazard-specific and all-hazards approach are useful in assessing the resilience of critical infrastructure. In IMPROVER, the hazard-specific approach is favored. There is a wide range of hazards of various types that can affect a given critical infrastructure. The risk of a given hazard is typically not well understood given the rarity of these events. Ideally, a critical infrastructure should be tested against every scenario. Time constraints and limited resources, however, often demand the use of a small number of scenarios. IMPROVER adopts a risk-informed approach which can identify the scenarios associated with the highest risk to cause disaster (or any other level of consequence) to a given critical infrastructure. The proposed methodology outlined in the next chapter aims to quantify the uncertainty in the ranking of the scenarios due to the lack of data and the complexity of the problem, which is typically ignored in the literature, despite their reliance in experts.
61
Hughes J.F., Healy K. [2014] ‘Measuring the resilience of transport infrastructure’. New Zealand Transport Agency. Available at:
https://www.nzta.govt.nz/resources/research/reports/546/ Accessed: 21 July 2016. 62
Omer M., Mostashari A., Nilchiani R., Mansouri M. (2012). ‘A framework for assessing resiliency of maritime transportation systems’. Maritime Policy & Management 39(7):1-19.
4
Methodology for risk-related scenario identification
4.1 IntroductionCritical infrastructure (e.g., ports, water or electricity networks) are exposed to a wide range of disruptive events. These hazards include natural, operational, malicious human-induced and market/economy/political events, which are likely to disrupt the function of the infrastructure, affect its integrity and have broader and, perhaps long-lasting, local, national or international consequences.
Given the importance of critical infrastructure for the security and well-being of the citizens, it is important to assess and improve their resilience. The resilience of the critical infrastructure aims to prepare the infrastructure in order to be able to withstand the shock and mitigate its impact. The assessment of its resilience typically relies on the determination of one or more hazard scenarios. There are a plethora of hazards that can affect infrastructure and some of these are well or partially understood and some unknown. Hence, one of the challenges that an analyst has is how they can determine a suitable hazard scenario for use in the assessment of infrastructure resilience.
To date, the literature has focused on the assessment of critical infrastructure resilience to natural hazards, such as earthquakes63,64,65, hurricanes66 and floods, which are low-probability-high-consequence events. In the aftermath of 9/11, terrorism has also been examined. The literature also favors the identification of the worst-case plausible scenario associated with the highest risk. This approach involves the determination of a large number of plausible scenarios and the assessment of their risk predominantly through expert judgment. These studies18 typically adopt a semi-quantitative method that does not account for the level of uncertainty in the opinions of the different experts regarding the likelihood of occurrence of the examined events or their impact. In particular, the most suitable scenario is the one which is both the most likely to occur and the most likely to have the heist impact on the critical infrastructure. IMPROVER draws from the best practices in the field of expert elicitation practice and develops a methodology for identifying suitable risk-based scenarios for the three identified hazard types. The novelty of the developed methodology lies in its use of Structured Expert Judgment (SEJ) elicitation; a formalized process to determine a rational consensus among subject-matter experts on the uncertainties associated with problems where sufficient empirical or historical data is not available to characterize uncertainties statistically. It is particularly suited to complex systems, which are difficult to model or to rare events.
63
Bruneau M., Chang S.E., Eguchi R.T. (2006). ‘A framework to quantitatively assess and enhance the seismic resilience of communities.’ Earthquake Spectra 19(4):737–8.
64
Cimellaro G., Reinhorn A., Bruneau M. (2010). ‘Seismic resilience of a hospital system.’ Structural Infrastructure Engineering 6(1):127–44
65
Chang SE, Shinozuka M. Measuring improvements in the disaster resilience of communities. Earthquake Spectra 2004;20(3):739– 55.
66
Reed DA, Kapur KC, Christie RD. Methodology for assessing the resilience of networked infrastructure. IEEE Syst J 2007;3(2):174–80.
www.improverproject.eu 18 4.2 Proposed Methodology
The proposed methodology consists of the 8 steps outlined in Figure 4.1. Central to this methodology is the role of the stakeholders, who are invited to use their judgement as to which hazard scenario is associated with the highest risk from a list of pre-determined scenarios using the paired comparison elicitation method. This method is selected as it meets all but one of the 5 main attributes that characterize a meaningful elicitation67. Firstly, the adopted method is reproducible, as the results can be reviewed and reproduced by scientific peers. Secondly, the method is accountable as all assessments are recorded and could be checked by a reviewer. Thirdly, it is neutral as the stakeholders are invited to complete the questionnaire individually without being influenced by an in-depth prior discussion of the scenarios. Finally, this approach reduces the potential bias of dominant personalities who may sway the opinions of the group in the direction they consider as more appropriate. In addition, the anonymity of the participants and their organizations in the presentation of the results is ensured. The paired comparison is a fair method, as the participants’ opinions are treated with equal weights. Only the stakeholders who are found to provide very inconsistent responses, such that statistically they appear random (i.e., consistent answers are considered the ones for which if A>B and B>C then A>C is true) are excluded. By contrast, the paired comparison does not meet the empirical control attribute as it does not assess the performance of the participants using empirical data. Nonetheless, this is not considered necessary for this study. It is recognized that the selected sample of hazard scenarios represent a fraction of the wide range of scenarios that can occur. This study aims to assess the level of agreement among the participants, which captures the complexity of the problem as well as the lack of data. It should be mentioned that suitable partisipants represent organisations, which have a vested interest in the critical infrastructure or have expertise in this type of critical infrastructure in which of these risk-related scenarios is more appropriate to be used.
4.2.1 Step 1: Identification of the components of the critical infrastructure.
Critical infrastructure systems are complex, often spatially distributed, networks. As a first step, the facilitator needs to have an in-depth understanding of the studied network and its interaction with other critical infrastructure systems. To do this, a list with the components of the network and their main characteristics (e.g., age, design characteristics, construction materials) is compiled. Apart from the list of components, a map which shows the distribution of the system and the location of its components is also a helpful tool. Information regarding the age and design of the system’s components is important in assessing their vulnerability. Finally, information regarding the functions of the critical infrastructure is necessary in understanding the importance of its services at local, national and international level.
67
4.2.2 Step 2: Identification of risk-related hazard events
In the second step, a list of possible hazard events that can be used for the assessment of resilience is compiled. To assist the facilitator with the construction of hazard scenarios, the methodology proposes the construction of events for four different classes of hazards, broadly agreed in the literature:
Natural hazards include here the geophysical and astrophysical hazards. Geophysical Step 1:
Identification of the main components of the examined
critical infrastructure.
Step 2:
Identification of hazard events which can affect the critical
infrastructure of interest.
Step 3: Draft questionnaire.
Step 4:
Workshop with stakeholders.
Step 5:
Stakeholders are sent the revised questionnaire to complete. Step 6: Analyses of answers. Step 7: Feedback workshop. Step 8: Report the findings.
Figure 4.1 Framework of methodology to identify suitable hazard scenario for assessing the resilience of critical infrastructure.
www.improverproject.eu 20
meteorological hazards (e.g. storm surges, hurricanes and droughts). Astrophysical hazards include hazards such as the solar storm.
Malicious human induced hazards, which include acts such as arson, sabotage and terrorism.
Operational hazards occur during the interaction of humans and machine in the specific critical infrastructure system or other infrastructure systems on which the studied network relies upon.
Market-related/Economy-related/Political hazards, such as financial crisis, austerity, geopolitical crisis or war, can seriously affect the functions or critical infrastructure. The constructed scenarios are based on brainstorming sessions as well as past events, which affected the specific infrastructure system or similar systems in other locations. It is important not to concentrate only on past events that have caused large or disastrous consequences. The incorporation of “near-miss” events is also important, as these events could repeat in the future with catastrophic consequences. The incorporation of scenarios used in the literature for risk or resilience assessment of the critical infrastructure is also recommended.
Table 4.1 Definition of consequence levels.
Consequences level Description
Disaster A catastrophic consequence hazard event which causes major disruption to the infrastructure in the region and which has a severe impact to the cities it serves.
Emergency A medium consequence hazard event which causes severe disruption to the infrastructure in the region and a moderate impact to the cities it serves.
Minor Incident A localized low consequence hazard event which causes the partial disruption to the infrastructure in the region.
The risk of a given hazard scenario is defined as the likelihood of disruption to the normal operation of the infrastructure within the region (and economic loss or societal impact of this interruption), and can be estimated as:
Risk = Likelihood x Consequence (1)
where Likelihood represents the likelihood of occurrence of a technological, natural or human hazard event which can disrupt one or more infrastructure assets; Consequence is measured in terms of the disruption of the service which the infrastructure provides and its impact to the community or economic loss including both direct (i.e. cost of any physical damage caused) and indirect (i.e., due to loss of revenue) costs. In this proposed procedure, rather than explicit quantitative evaluation of the consequences, three levels of consequences are proposed, defined in Table 4.1: disaster, emergency situation and minor incident. This aids comparison of different hazards and comparison across different infrastructure systems. In the applications presented in this study, however, we concentrate on the former two: the disaster and the emergency levels.
Table 4.2 Generic table for paired comparison. There are three similar tables for each level of consequence and type of hazard.
Which of the two hazard scenarios is more likely to occur in the next 5 years?
Or
Which of the two hazard scenarios is more likely to cause a disaster?
Or
Which of the two hazard scenarios is more likely to cause an emergency?
Hazard Scenario 1 Hazard Scenario 2 Hazard Scenario 3 … Hazard Scenario n Hazard Scenario 1 R C = Hazard Scenario 2 Hazard Scenario 3 …. Hazard Scenario n
The method which will be used in this exercise to identify suitable risk-related hazard scenarios is by paired comparison analysis of experts’ ‘contingent evaluations’ (sometimes call ‘stated preference’)68. According to this method, for each hazard class, the scenarios are formatted into three table, as depicted in Table 4.2. In the first table, the stakeholders are invited to compare every two hazards (one in a row and another in the column on the table) and using their judgement to identify which one is more likely to occur. The next two tables invite the participants to identify which of every two hazards is more likely to cause a disaster and an emergency situation, respectively. In more detail, the participating stakeholders are asked to consider the criteria in the top left box of each table (see Table 4.2) and to place in each of the empty boxes on the upper right corner an “R” indicating that the incident on the Row, Hazard Event 1, meets the criteria, or a “C” in the box if they think that the incident in the Column, Hazard Event 2, meets the criteria. In case the experts are of the opinion that the two incidents are equally likely to meet the criteria then an equals sign (“=”) should be put in the box.
4.2.3 Step 3: Draft questionnaire
Having concluded the list of hazard events, a list of potential participants to the paired comparison exercise is identified from the list of stakeholders. This list should include from 6-15 participants. Then a pre-workshop questionnaire is drafted which aims to introduce potential participants to the main objectives of ‘IMPROVER’ and this study in particular. It
68
www.improverproject.eu 22
also collects general information from the participants and includes a consent form and information confirming that the study complies with the data protection act and that the participant responses will remain anonymous. Finally, it provides information regarding the paired comparison method and provides clear and detailed instructions on the role of the participants.
The questionnaires prepared for the 4 living labs as part of IMPROVER can be seen in Appendices 1-4. The reader can note that the natural hazard scenarios are far less in number than the operational hazards.
4.2.4 Step 4: Workshop with stakeholders
The proposed methodology aims to identify a disaster-related and an emergency related hazard scenario for a given infrastructure system and to assess the level of agreement amongst the stakeholders. In order to do this, a workshop with the stakeholders is organized to provide information on the research project, to introduce them to the objectives of the study, to familiarize them with the paired comparison technique and invite their feedback with respect to the constructed hazard scenarios. Ideally, a minimum of 6 and a maximum of 15 participants is recommended. One of the challenges faced by the facilitators of ‘IMPROVER’ was to engage with the stakeholders of each critical infrastructure and organize a well-attended workshop. The latter was overcome by inviting the identified participants of each workshop to invite further participants through their contacts lists.
In the workshop, the stakeholders are asked to air their opinions regarding the usefulness of the study and their confidence in the paired comparison method. They are also presented with the list of the pre-defined hazard scenarios and are invited to agree on the list after discussing the usefulness of the pre-selected hazard scenarios, add scenarios that they consider important and remove the scenarios that they consider as irrelevant or unlikely to cause an emergency or disaster. From the practical application of this procedure it is found that a maximum of 18 scenarios for each hazard type is should be agreed on. The workshop is also used to remove possible ambiguities in the description of hazards, technical jargon or definitions of infrastructure components, consequences etc. in the questionnaire. This process is important as terminology differs across infrastructure sectors, and across different cultures and countries. Hence, the workshop can be considered to be a first point of engagement of the stakeholders but also to provide a means of tailoring and honing the initial questionnaire into an infrastructure-specific, useful and more engaging tool for the stakeholders. It also reduces the risk of major disagreement with the results or questioning of the questionnaire in Step 7.
4.2.5 Step 5: Completion of questionnaire
The updated questionnaire is sent in an electronic form to participants. These include both the stakeholder panel in Step 4 as well as other stakeholders identified by the facilitator or through recommendations from the stakeholder panel. The participants are invited to complete the paired comparisons in their own time without interacting with other participants. The questionnaire developed in this study takes approximately 45min to be completed. The completed questionnaires are returned by email or post to the facilitator.
4.2.6 Step 6: Analyses of answers
The completed tables are analyzed using the free-software ‘UNIBALANCE’69 using the probabilistic inversion technique. This produces a mean score for each hazard event rated by the participants as well as the standard deviation around this mean score which represents the level of disagreement within the expert group. Inverse probability is used to estimate both values. The mean scores are then rescaled in order to vary from zero and one. A zero value is assigned to the hazard event considered least risky and a value of one is assigned to the riskier hazard event.
The degree of consistency of induvial stakeholders as well as the group of stakeholders is also tested. A chi-square test is performed to test whether each individual stakeholder responded at random. This test produces a p-value. For p-values below 0.05, there is enough evidence in the responses to reject the hypothesis that the stakeholder responded at random. The confidence in the results can be improved by removing the response of the participants who are found to have completed the questionnaire at random. The level of agreement among the stakeholders is also examined in three different ways. Firstly, the degree of agreement is estimated by measuring how closely the pattern of the stakeholders pairwise preferences match. Secondly, the degree of concordance is examined by measuring how similar the rank orders are amongst the group of stakeholders. Thirdly, a chi-square test is used to check whether the group ranking preferences are made at random. P-values below 0.05 indicate that the group ranking preferences has a structure and is not random. By contrast, p-values above 0.05 suggest that lack of consensus within the group regarding the ranking preferences. In this case, the values are more likely to be clustered and there is n oclear ranking order for the scenarios.
4.2.7 Step 7: Feedback workshop
Having analyzed the results, a feedback workshop with the stakeholders is held to discuss the results. The stakeholders are invited to express their opinions on whether they agree or disagree with the findings and provide justifications for their disagreements. The facilitator should be prepared for the extreme case of major disagreement with the majority of the stakeholders. In this case, the questionnaire could be revised and sent to the participants for the process to be repeated.
4.2.8 Step 8: Report of findings
Reporting the findings is the final step of the proposed methodology. The report should contain the aggregated data and ensure the anonymity of the participants and their organizations.
Apart from providing a tool for the definition of scenarios, it is believed that this process of paired evaluation, stakeholder engagement, workshops, discussion and feedback provide a means for resilience to be discussed and to raise awareness in stakeholders of potential risks. Furthermore, the bringing together stakeholders from across different sectors and operations of the infrastructure, helps evaluate consequences to multiple systems contributing to the
69
www.improverproject.eu 24
infrastructure operation and identifying hazard scenarios that may not be identified as severe through consideration of single systems. For example, where the consequences to one system are small, but the scenario affects multiple systems such that the overall consequences are large.
4.3 Concluding remarks
An 8-step methodology is proposed in order to identify risk-related scenarios, which can be used to assess the resilience of critical infrastructure. So far, the literature concentrates on a semi-qualitative approach of constructing a risk matrix, where the risk of each scenario is quantified by a value representing its likelihood to occur and the severity of its impact. By contrast, the proposed methodology engages the stakeholders through a structured elicitation technique, called paired comparison. This technique can transform the qualitative responses of the stakeholders to a mean rank score and its standard deviation, which represents their degree of disagreement. The successful application of this method is based on engaging with the stakeholders in developing the list of appropriate hazard scenarios for each hazard type as well as in participating in the elicitation.
