68
The paper presents results from an analysis of all the accidents reported to the MARS system up to mid-2007 regarding the underlying causes and the extent of learning, based on the level of learning.
Both the methods used, the one for analysis of underlying causes and the one for establishing the level of learning, worked very well on the data in the MARS database.
The most important underlying causes were found in weaknesses in process analysis (risk assessment) and in procedures, regardless of industry type. Weaknesses in safety management systems and in safety culture contribute as underlying causes in a very high percentage of the accidents. No major differences in the pattern of the underlying causes were found for the various industry types, neither for the various countries. The quality of reporting, measured in terms of analysis of underlying causes, vary considerably between various countries. The level of learning, as determined from the information in the reports, is found to be in general rather low, especially from some of the countries. In two thirds of the accidents the learning stops at a local level within the sites. This study resulted in ideas of improvement of the MARS system.
69
Figure 6.1 A methodology for assessment of the effectiveness of learning from incidents.
For the sake of clarity and as background for answering research questions RQs 2 and 3, the objectives of study 1 (LINS) are repeated here: to develop methods for assessment of the effectiveness of learning from incidents in the process industry (normal cut of incidents) and apply those in the field. The results from application of the methods should be suitable for correlating with other safety measures in an organisation.
RQ 2
How can methods be constructed for analysing and assessing the effectiveness of the learning from “normal” incidents in a process industry (for company-internal use), considering in particular:
a) the effectiveness in the learning cycle (i.e. the necessary steps and actions from reporting an incident to the implementation and follow-up of the measures taken),
b) the effectiveness in the lesson learned (actual learning versus the potential learning),
c) the efficiency of reporting,
d) that the results from application of the methods should be suitable for correlating with other results of measuring safety in an organisation?
Methodology for assessment of the effectiveness of learning from incidents
Method for assessment of the effectiveness in the learning cycle
Method for assessment of the effectiveness in the lesson learned
Method for assessment of the degree of reporting
“reportable” incidents
70
The general answer to RQ 2 is the methodology above with its three elements using:
the method for assessing the effectiveness in the learning cycle with
individual tools for the different steps, described in section 5.2 and in more detail in Paper I;
the method for assessing the effectiveness of the lesson learned (based on level of learning) with its six steps, including the system for classification into level of learning and the tool for assessing the underlying causes/potential learning as described in section 5.3 and in more detail in Paper II;
the tool for assessing the threshold for reporting and the guideline for assessing the hidden number, as described in section 5.4.
Regarding 2d), all the methods and tools developed give results in quantitative terms.
Therefore, the results will automatically lend themselves to statistical correlations with other quantified results from safety investigations.
RQ 3
How effective is the learning from incidents in a selection of companies in the process industry in Sweden, based on:
a) the learning cycle,
b) the lessons learned (both as actual lessons learned and compared to potential lessons learned)?
The methods were applied to a large number of incidents in six Swedish process industry companies. The results are presented in Papers I and II. The results concerning the effectiveness of the learning varied significantly between the companies. The effectiveness in the learning cycle, 1st loop, was in general found to be
“fair” (4 on the 0-10 scale) or at the best “good” (7 on the 0-10 scale) in some respects, except for one company which was consistently “good” to almost “excellent”
(10 on the 0-10 scale). The 2nd loop received comparatively lower ratings than the 1st loop for all the companies. The actual learning compared to the potential learning (based on the lesson learned) was in general rather poor except for the same company that was best in the effectiveness in the learning cycle. Expressed as the ratio between the “mean values” of actual level of learning and potential level of learning, the values were in the range of 0.36 to 0.86.
The results obtained in this research are a start in assembling a larger reference material for the process industry on which more general conclusions for the Swedish process industry can be drawn.
For the sake of clarity and as background for answering research questions RQ 4- RQ 6, the objectives of study 2 are repeated here. The second study is about learning from the major accidents in the MARS database (Papers III and IV). The objective of this
71
study was primarily to assess the actual level of learning of the accidents reported in the MARS database. The objective was also to assess whether all underlying causes had been found in the original investigation reports. Further objectives were to try to link these underlying causes to issues of safety management systems and safety culture, and to identify weaknesses in the quality of reporting and analysing.
RQ 4
How can a method for analysing and assessing the effectiveness of learning from the major accidents contained in the MARS database be constructed, considering in particular:
a) the actual level of learning;
b) an in-depth analysis of underlying causes to reflect the potential level of learning?
The actual level of learning is assessed by using the method (classification system) described in section 5.3, step 1 and in more detail in Paper IV. The underlying causes are assessed by the method described in section 5.3, step 2 and in more detail in Paper III.
RQ 5
Does the learning from accidents by companies and national authorities – based on results from application of the assessment methods – meet the objectives set for the learning from major accidents in the MARS system?
The objective of MARS is to learn broadly within the entire European process industry. Thus, one would expect most of the lessons learned to be at level IV and V according to the system for classification used. The results from the MARS study are that a large part of the reported accidents are incompletely analysed for causes and for the potential lessons learned. The results from application of the method are that only 17% of the lessons learned were classified as a IV or V level of learning. Considering this, the answer to RQ 5 would be a clear “No”.
RQ 6
Based on results from application of the assessment method, are there any (and if so, what are they):
a) Specific characteristic patterns in the underlying causes per industry type?
b) Specific national characteristic patterns in the underlying causes?
c) Industry specific characteristic patterns in the level of learning?
d) Specific national characteristic patterns in the level of learning?
72
e) Impact of the requirement in the Seveso II legislation of safety management system on the causes of accidents?
The results from application of the methods indicated the following answers to the questions:
6a) There is a similar pattern regardless of the industry type for the most common underlying causes. All industry types have weaknesses in “Process analysis” and
“Procedure” as the two most common underlying causes. The third most common is found in either “Training” or “Design”. Notable is also the high percentage of weaknesses in “Maintenance” and “Inspection” for many industry types.
6b) There is a similar pattern for all countries for the most common underlying causes. All countries (except one) have weaknesses in “Process analysis” as the most common. The second most common weakness is in either “Procedure”, “Training” or
“Design”.
6c) The pattern for level of learning is similar for most industry types. Only the petrochemical industry seemed to have a higher percentage of learning on higher levels (III, IV and V) than most other industry types.
6d) Two countries stood out positively with a high percentage of learning on higher levels (III, IV and V), whereas one country stood out negatively with a low percentage of learning on levels III, IV and V and a very high percentage on level 0 learning. For the rest of the countries, the percentages are more or less equally distributed, with higher percentages on the 0, II and III levels of learning.
6e) No significant impact was found of the introduction in the Seveso II requirements of the safety management system on the causes of accidents.