55
causation picture), one can assume a somewhat lower average level of learning than for the incidents actually reported. Even with all these assumptions, it is considered worthwhile to include this step in the method in this semi-quantitative way. If the purpose is to compare the level of learning between various departments, sites or companies, one needs to have a common baseline defining what a reportable incident is (i.e. the same threshold for reporting should be used). This will, however, also vary between organisations.
By making an adjustment for the hidden number in the manner described above, one will be able to arrive at numerical values of the level of learning (adjusted for non-reported incidents), which is probably a truer picture than the uncorrected values of step 3.
Step 5: Consideration of possible learning from incidents on an aggregated basis – the 2nd loop
The next step in the method considered the possible learning from the incidents when treated on an aggregated basis, if such a 2nd loop really exists and increases the learning. The same tool that was developed for evaluation of the effectiveness of the 2nd loop, and described in section 5.2 Effectiveness in the learning cycle, can be used to judge whether the results from steps 1-4 should be adjusted or not. A good treatment of the incidents in the 2nd loop can compensate in part for poor results from the step 1 evaluation. As of now, no quantitative approach has been tried in this step.
Step 6: Consideration of other mechanisms for learning from incidents
The final step in the method considers learning mechanisms for incident learning outside of the incident learning system proper. Information for such considerations is found in interviews of employees (e.g. in safety audits). No quantitative approach has as yet been tried in this step.
56
of reporting is used to reflect that in this case we mainly consider quantity of reporting.
For effective learning from incidents to take place, there is a need to bring as many of those incidents with a learning potential as possible to the attention of the organisation and deal with them in the incident learning system. In estimating how good an organisation is at this, a tool is proposed for determining the threshold for reporting as an indirect measure. Results from this tool are expected to be a support in estimating how high the hidden number is.
Ideally, the reporting should be based on the learning potential of the incident, but in reality in most organisations it is based on the level of consequence or disturbance (severity). The number of reports as a function of the severity of the incidents will in principle be similar to Figure 5.2, where the number of incidents actually occurring decrease with increasing severity. The number of reported incidents is close to what actually occurs for high severity incidents, but is often only a smaller fraction for low severity incidents. The area between the curves represents the unreported incidents, the hidden number. At some defined low severity, the organisation has set the limit for reporting – the reportable incident.
Figure 5.2 Number of incidents (occurring and reported) as a function of the severity of incidents.
The area between “Actually occurring” and
“Reported” represents the “Hidden number”
Number of reports
Severity of incidents Actually
occurring
“Reportable” incident Company
A
B
C
Reported
57
5.4.1 Threshold for reporting
To be able to judge the actual threshold for reporting in an organisation, one can examine many reports and establish the lowest values on the severity scale or better yet, the learning scale (assuming that the learning potential is on the x-axis) that still result in incident reports. A significant number of incidents would be needed to support the choice of the threshold; a few would not be enough.
The tool was built primarily for the types of events that normally are reported in the process industry – events with actual consequences such as personal injuries (type 1) or loss of containment, LOC, (type 2). But the events reported can also just be deviations from normal conditions where the failure of one or more barriers could have led to an event with more serious consequences. Table 5.4 presents the tool that was developed to help in assessing this threshold. The scale used is based on the same idea as the tool for assessing the effectiveness in the learning cycle (i.e. the concept of capability maturity models). The scale was selected by the author to reflect the reporting efficiency from poor to excellent. A reported event of any of the types 1-5, is given a threshold rating value that corresponds nearest to what is expressed in the table (with the possibility to interpolate).
For instance, a report about a deviation from a normal operating procedure, where the deviation is considered minor in itself, but where the presence of another two circumstances (e.g. two failing safety barriers) would have led to an accident, (e.g. a personal injury, LOC, fire, environmental impact or financial loss) would receive a rating of 7.
Table 5.4 Tool for assessing the threshold for reporting.
Type of event
(Consequence or effect)
Rating
2 (Poor) 4 (Fair) 7 (Good) 10
(Excellent) 1 Personal injury
(actual)
Major personal injury, normally hospitalisation.
LTI (Lost Time Incident) = absence from work 1 day or more.
Medical care.
2 Loss of
containment (LOC) of dangerous substance, fires etc.
(actual)
Major LOC.
Major fire.
Major environmental impact and/or
Small LOC.
Small fire.
Minor environmental impact and/or
LOC, which could possibly have lead to fire,
environmental impact and/or financial cost, if 1
LOC, which could possibly have lead to fire, environmental impact and/or financial cost, if
58
financial cost. financial cost. more barrier had failed.
2 more barriers had failed.
3 The event + one or more
circumstances could have led to
….
Not applicable for this type of event.
The event + 1 more circumstance could have led to a serious accident (major personal injury, major LOC, major fire, major
environmental impact and/or financial loss).
The event + 2 more circumstances could have led to an accident (personal injury, LOC, fire, environmental impact and/or financial loss).
The event + 3 more circumstance could have led to an accident (personal injury, LOC, fire, environmental impact and/or financial loss).
4 Deviations from procedures (without accident)
Not applicable for this type of event.
Major deviation from procedure + 1 more circumstance could have led to a serious accident (major personal injury, major LOC, major fire, major
environmental impact and/or financial loss).
Small deviation from procedure + 2 more circumstances could have led to an accident (personal injury, LOC, fire, environmental impact and/or financial loss).
Minor deviation from procedure + 3 more circumstances could have led to an accident (personal injury, LOC, fire, environmental impact and/or financial loss).
5 Other deviations (without actual accident), such as failing safety equipment*, communication systems, etc.
Not applicable for this type of event.
The deviation + 1 more circumstance could have led to a serious accident (major personal injury, major LOC, major fire, major
environmental impact and/or financial loss).
The deviation + 2 more
circumstances could have led to an accident (personal injury, LOC, fire, environmental impact and/or financial loss).
The deviation + 3 more circumstances could have led to an accident (personal injury, LOC, fire, environmental impact and/or financial loss).
* Examples of safety equipment are interlock systems, safety relief valves, fire fighting equipment, emergency alarms, emergency showers.
59
For the event types 3, 4 and 5, some reasonable engineering judgement is necessary from the person using the tool to assess the possibilities for escalation of the reported event to more serious consequences.
The threshold “value” will be a significant guide to the ambition level of the organisation to try to learn from incidents. The lower the “threshold”, the higher the rating, the better the reporting and probably the possibilities for learning. The actual level could also be compared to the official wording of the company’s definition of what should be reported.
5.4.2 Hidden number
The issue of the hidden number is closely related to the threshold for reporting. The hidden number is an expression of how many incidents that are not reported, but were actually “worth reporting” or at least “reportable” and thus should have been reported. In Figure 5.2, the hidden number is represented by the area between
“Actually occurring” and “Reported” incidents.
Certainly, the number of reportable incidents varies considerably in practice and depends among other things on factors such as:
Size of enterprise – the more employees with exposure to hazardous conditions the more reportable incidents, probably
Type of enterprise – the more complex, the more reportable incidents, probably
Type of activities – the more manual work, the more reportable incidents, probably
Type of plant – the tighter and more congested, the more reportable incidents, probably (to some extent related to age, indoors or outdoors location)
Existing safety culture – the better the safety culture, the fewer reportable incidents, probably
Existing safety culture – the better the safety culture, the larger the proportion of reported incidents of the reportable incidents, probably
The company definition of what is a reportable incident (not necessarily the same as worth reporting)
Unreported incidents with a potential for learning will always occur. This hidden number should be as low as possible. In reality, there will always be a balance between quantity and quality. It is probably better with fewer reports, which are handled well, than many reports, which are handled poorly. Considering the variations in the above mentioned factors, the figure for reportable incidents can vary considerably between different companies and also between departments within the same company.
60
Because the issue of the hidden number is so important, it is essential to evaluate what could be a reasonable figure of reportable incidents in an enterprise, despite all the difficulties and uncertainties. In the LINS project, which covered six process industries, evaluations were made of a reasonable figure for reportable incidents per employee, per year. It was concluded that 3 incidents per employee, per year is reasonable in the process industry. An external, neutral expert panel was asked about this. Their figure was 3 incidents per employee, per year as an average and with a span of 1 to 7.5. The companies in the study estimated a reasonable figure for reportable incidents in their own organisation. Their answers ranged from 0.75 to 5 with an average of 2.3 reports per employee, per year based on employees in some sort of technical jobs. The actual number of reports in the companies ranged from 0.1 to 2.3 per employee, per year. Based on this information, one can estimate that reasonable figures for process industry companies are in the order of magnitude of 1 to 5 reports per employee, per year. If no real estimate of a representative figure exists, it is suggested that the figure of 3 be used as a reasonable estimate.
Figures lower than 1 incident per employee, per year would indicate that the reporting can probably be improved. The results from the evaluation of the threshold of reporting should also be consulted. If there is both a low number of reported incidents and a poor rating (say <= 4) of the threshold for reporting, this is most probably an indication that the hidden number is rather high.
Every company is probably best at making its own evaluation of how many reportable incidents there ought to be in the enterprise. A discussion in the company on how many incidents that in fact are reported compared to how many that are reportable is already a worthwhile exercise. This can result in an increase in reporting frequency.
However, resources must then be secured to take care of the increased flow of reports.
Otherwise, the good ambition to learn more could become a waste of effort, and in the worst case, result in decreased motivation for reporting if employees feel that the organisation is not properly attending to the reports.
61