Comment is mandatory

Link to .pdf
Link to full MJV Good Practice Report "Vol. 6 Assessment of Safety Management Systems of Major Hazard Sites"
Link to Seveso Inspection Series Home Page

MJV Good Practice Report

Assessment of Safety Management Systems on Major Hazard Sites 

 

The safety management system (SMS) is now considered a central component of modern process safety management. It was first adopted into various European national laws in the early 1990s, most notably in the United Kingdom for offshore facilities following the 1988 Piper Alpha disaster in the North Sea. With the advent of the Seveso II Directive in 1996 (Directive 96/82/EC), the concept of the safety management system was enshrined as an essential element in control of sites with major chemical hazards across the European Union. The Cullen  Report that was issued following the Piper Alpha disaster also introduced the operator obligation to “demonstrate” that it has a safety management system and recommended that regulators employ a systematic approach to inspections that was equally focused on compliance with safety management criteria as well as technical standards.

With the entering into force of the Seveso II  Directive) the Member States are required to ensure that the operator of an establishment falling under  the requirements of the Directive draws up a policy  for the prevention of major accidents. According to Article 9 of the Directive, the operator must demonstrate that the MAPP and the SMS have been put into effect consistent with the principles articulated in Annex III. The Directive also clearly states that the level of complexity and detail of the safety management system should be in proportion to the level of risk present on the site.

The SMS as described in Annex III consists of the organisational structure, responsibilities, practices, procedures, processes and resources for the implementation of the MAPP. According to the Annex, the SMS must address the issues:

  • Organisation and personnel
  • Identification and evaluation of major hazards
  • Operational control
  • Management of change
  • Planning for emergencies
  • Monitoring performance
  • Audit and review

Article 18 of the Directive requires conducting a systematic examination of the systems being employed at the establishment, whether of a technical, organisational or managerial nature, so as to ensure in particular:

  • That the operator can demonstrate that he has taken appropriate measures, in connection with the various activities involved in the establishment, to prevent major-accidents,
  • That the operator can demonstrate that he has provided appropriate means for limiting the consequences of major-accidents, on-site and off-site,
  • That the data and information contained in the safety report, or any other report submitted, adequately reflects the conditions in the establishment
  • That information has been supplied to the public pursuant to Article 13.

Public authorities are required to carry out inspections of the establishments which cover not only aspects of the technical but also organisational and managerial systems.

There are still widespread questions among many inspectors as to when the assessment of the SMS  can determine that adequate steps have been taken, in particular:

  • At what point, can the demonstration by the operator be considered sufficient?
  • How can inspectors document their evidence of deficiencies in the SMS in such a way as to be able to derive effective enforcement measures from this?

 

This Seveso Inspection Series short report is a summary of a Seveso Inspection Series expert report of the same name. The full report can be found at: http://ipsc.jrc.ec.europa.eu/index.php/Publications/125/0/

 

Mutual Joint Visit Workshop on Safety Management Systems

For this reason, it was recognised that sharing knowledge and experience among inspectors could be very useful for benchmarking good practice on inspection and control of SMS demonstrations. In addition, this exchange would be of value to identify common priorities for further development of knowledge and tools to aid inspectors in  these efforts.

From 27-29 October 2010, the Regional Council of Darmstadt hosted a Mutual Joint Visit (MJV) workshop for Seveso Inspectors in Fulda, Germany on the topic of Safety Management Systems. Workshop participants consisted of 33 participants from inspection authorities from 17 EU Member States, 2 Candidate Countries and 2 countries of the European Economic Area. In addition a number of representatives from industry participated.

The workshop the following SMS elements:

  • Organisation and Personnel
  • Identification and Evaluation of Major Hazards and Risks
  • Management of Change
  • Monitoring Performance, Audit and Review1

1 The third element (Operating Procedures) and fifth element (Planning for Emergencies) of the SMS, as defined in Annex III of the Seveso Directive, were not discussed in view of time constraints.

Figure 1: Hierarchy of an SMSFigure 1: Hierarchy of an SMS

 

The remaining SMS elements were the focus of the workshops as indicated below:

Participants were allocated to one of the 3 parallel break-out groups, focused on a different type of operator, as described above, but the same SMS inspection topic. Each workshop concluded with a plenary session in which the groups came together to share their results. For each plenary session rapporteurs noted the contents, recommendations and conclusions of the discussions and in the final session at the end of the workshop the compiled results were presented for a final discussion. The discussions, their results together with the introductory presentations generated the basis for this publication.

Assessment of SMS Effectiveness

A large proportion of the inspection activities to-date have  concentrated  on  In  assessing  the  SMS,  the inspector should keep in mind the following essential characteristics of an effective SMS:

  1. Evidence of robust implementation, that is, the establishment of clear objectives and clear requirements that are consistently and rigorously followed.
  2. Qualification of personnel involved in executing the safety management system, facilitating formation of a proper process hazard assessment (team), reliable execution of the management of change process, etc.
  3. Performance monitoring, involving the objectives, reports and reviews for 1. and 2. The identification and dissemination and implementation of lessons learned
  4. Leadership from the top down that supports implementation and anticipates and resolves potential conflicts with other corporate objectives giving equal priority to safety
  5. Self-assessment and auditing processes conducted in a thorough manner with adequate frequency followed by appropriate and timely implementation of resulting recommendations.

Both the inspector and the operator are charged with auditing the SMS. By nature an audit requires a systematic and evidence-based approach. The evaluation generally starts with an overall assessment as to whether the SMS addresses all the necessary elements of Annex III. Then the evaluation should proceed to each element of the SMS and systematically seek to find evidence to determine the degree to which the SMS is known, understood, accepted and followed in the organisation. The following questions may go some way to addressing these aspects:

  • Does the SMS contain the elements from Annex III of the Directive?
  • Are responsibilities defined and assigned?
  • Are procedures defined, implemented and adhered to?
  • Does the operation on-site indicate that the SMS functions?
  • Is safe operation a day-to-day and long term goal of the company?

Two further questions of particular importance within the inspection of the SMS are:

  • How good is the SMS?
  • How good does the operator believe the SMS to be?

Organisation and Personnel

Some key aspects of the safety management system are embedded in the organizational structure, including the assignment of roles and responsibilities to job functions, identifying competency and training needs of the persons assigned to the specific job functions, and establishing the communication mechanisms for providing important information across and up and down the organization. In effect, the safety management system provides the essential infrastructure to support the rest of the system.

When the SMS procedures have been outsourced. It is important to verify implementation of the SMS at the site. In all cases, it is never sufficient to rely on written procedures, but even more so when writing the SMS has been outsourced.

Employee training. The organisation of personnel training is an important issue of the general topic of “organisation and personnel”. Both operator employees and contractor employees need to be aware of process safety issues and companies must monitor whether their procedures for organising and training their employees and organising contractors are functioning.

Some questions an inspector might ask of the operator in this regard could be the following:

  • How are decisions made about who should be trained and what the training should cover?
  • What is the minimum safety training required for all employees? To what extent are employees trained to understand hazards?
  • What additional safety training is offered to some job functions, if any, and for which functions specifically?
  • How is training organised? Is there both routine training provided at regular frequencies as well as ad hoc training? How often is safety training targeted or a part of various training events?
  • Are safety topics regularly included in all types of training? Are there training opportunities for addressing specific safety issues (e.g., hazard awareness) and if so, what are they?

Contractor communication and training. Just as for employees, the operator should proactively provide contractor employees complete information on the hazards associated with their work and control measures to  minimise the risk of accident.

Common Success Factors

The ease with which the company empowers the organisation and its personnel to maintain and continuously improve safety often depends on the following key factors:

  • The size and core activity of the company. Chemical manufacturing sites generally have a better understanding of the need to understand chemical hazards and risks than those industries where the chemical hazard is an ancillary operation to the main economic activity.
  • Sufficient resources allocated to safety critical activities. Such resources include not only financial means, but also time, staffing levels, and empowerment of those tasked with carrying out the activities.
  • The involvement of contractors and temporary workers. When contractors frequently perform work on site, it creates an added challenge for safety management. Contractors are not particularly bound into the company safety culture and there is only a limited degree to which individual performance standards and behaviours can be reformed and adapted to reflect the safety climate onsite more closely.
  • Management commitment must be embedded at the very top of the organisation and be present throughout the whole management chain.
  • Availability and involvement of employee representatives. They can play an important positive role in making the SMS work as it should, particularly in larger organisations, because they have established mechanisms for exchanging and channelling information in both directions between management and the workforce.

What does success look like?

The following are examples volunteered by participants from their inspection experience:

  • Safety is a management agenda item – it appears as a regular and important item at managerial meetings, not just safety meetings.
  • Major hazards are addressed systematically in identifying competency, training, procedures and control measures.
  • Safety critical tasks have been systematically identified and documented.
  • There is sufficient evidence that employees and contractors are involved in the development and delivery of training and procedures.
  • Training records reflect the implementation of training to address the identified needs and testing of competence is routinely conducted as follow-up to training or when replacing staff in a safety critical function.
  • Interviews with employees confirm that procedures described in written documents are understood and followed.
  • Selection and management of contractors and temporary workers reflects competency requirements identified for safety critical tasks (certification, qualifications and experience).
  • Contractor supervision and follow-up is a routine part of company procedure and appropriately includes attention to risk management and safe work practices (the intelligent customer).

 

Identification and Evaluation of Major Hazards and Risks

Risk assessment is the cornerstone of the SMS. It is a continuous process in the global life-cycle of a company. The aim of the identification and evaluation of major hazards and risks is to ensure proper control of low-probability, high consequence events.

The role of management. Since the management is responsible for managing resources, by necessity it plays a role in ensuring adequate resources are allocated to maintain the proper control measures to address the risks.

The relevance of accident lessons learned to the risk assessment. It is useful for the inspector to ask the company whether it has researched past accidents in conducting the risk assessment. Relevant findings from past accidents should be used as input since the lessons learned often influence and provide new information to improve standards and codes of practice.

Common  Success Factors

  • Competence. Large companies often have the advantage of maintaining in-house competence in performing risk assessments.
  • Use of experience and feedback. Onsite sources of feedback include the history of past accidents and near-misses, findings from inspections and audits, and maintenance records. Involvement of site employees in the development of the risk assessment can help ensure that the relevant information is communicated for this purpose. Lessons learned from accidents in the same industry or sites with similar processes should also be taken into account.
  • Ownership of the risk assessment. Site management must take on board the outcomes from the risk assessment, including appropriate follow-up on recommendations.
  • Awareness and communication of risks. The site management must take responsibility to communicate the risks and control measures identified in the risk assessment to all personnel who may have a role in managing risk and ensuring the control measures function, equally covering departments in supportive roles, such as procurement and human resources and also the interfaces with contractors.

What does success look like?

The following are examples volunteered by participants from their inspection experience:

  • Risk assessment drives control processes for managing all of the following:
    • Operating procedures
    • Equipment
    • Training
    • Inspections and maintenance
    • Emergency planning
  • Identification and evaluation of major hazards and risks are clearly proportionate in the site’s risk management approach.
  • Employees and contractors are aware of the risks associated with their work and their role in controlling them.
  • The site risk assessment and individual process risk assessments are fully documented, including the process followed, results and information used to produce the outcome. Control measures and associated actions recommended by the risk assessment should be documented including follow-up (when and how they were implemented).
  • There is a systematic selection and application of risk assessment methods and the consequence analysis was conducted by a competentexpert.
  • The off-site risk is communicated transparently to senior management and all stakeholders.

 

Management of Change (MoC)

Seveso site operators often are not sufficiently aware that failure in the management of change is one of the most common causes of accidents.  Every accident that occurs is proof that the safety management system is not 100% working to control the risks as it should.  Sometimes the accident may be caused by latent errors, that is, from a change that was implemented many years ago but never communicated or documented or assessed in any way, and the associated risk only became evident when the accident occurred.

The MoC policy should address all the following elements:

  • Each responsibility, that is each step of the procedure, should be assigned to specific job functions. These responsibilities should include authorisation, initiation, and approval of the risk assessment process and also for the change process selected following the risk assessment. The policy should also include a process to verify that the change was implemented as recommended by the risk assessment with the recommended control measures in place (if any) and that safe operation can take place.
  • The entire process should be transparent from the point that the change has been identified as a potentially safety relevant change all the way to the final step which should consist of verification that the change has been implemented correctly.
  • The required competencies of all involved in the MoC process should also be specified.
  • The system should address whether permanent changes and temporary changes are handled differently – often permanent changes are documented better than temporary ones.
  • The policy should require documentation of the change and verification that the change has been documented. All relevant written operating procedures should be modified as necessary to reflect the change.
  • Changes to process drawings as a result of the MoC should be considered as part of the documentation that may need modification. Often accidents have occurred due to work being carried out using an incorrect drawing.
  • The process for communicating changes should be outlined, including the specific job functions that should be informed and for what purpose.
  • The MoC process needs to clarify the point at which the change is considered as completed, that is, when should the proper authorization of a completed change take place, verifying that the change physically conforms to the intended change and that it has been documented.

Aging of installations. Once a piece of equipment changes the operating process, this is an operational change. Replacement of parts is often simply not exchange of one piece of one piece of equipment for another. It may be an upgrade that imposes changes on interfacing parts of the process or it may even require a process re-design.

Alternatively, the material composition may have changed with subsequent effect on downstream processes.

Organizational change.  The process of managing organization of change should include identification of safety critical roles and the workload, competences and specialised training associated with each role. The risk analysis should serve as the basis for determining whether additional competency, training or a different workload distribution is required.

Involving human resources. The human resources department may be important in assessing the implications of the change, projecting it out over the short and medium-term and communicating it to management and other staff as might be appropriate.

Common  Success Factors

  • Size of the company. Small, simply structured companies with a limited number of hazardous substances and processes may have very few significant changes in the whole lifetime of the company. However, they should be attentive to change events, few as they may be, that could affect their process risks.
  • Complexity and severity of risk. As sites increase in size, they can accommodate larger volumes of substances and more processes that may increase complexity and severity of potential accident consequences Other enterprises may by their nature have rapidly changing processes and chemicals and thus management of change is an essential aspect of doing business.
  • Clear and correct definition of safety relevant changes. One of the greatest challenges of management of change is recognising a safety relevant change. The definition should take account of organisational, personnel and technical changes, including progressive change and temporary changes.
  • Clear procedures for assessing risks associated with change. The risk assessment should be proportionate to the dimension or complexity of the change. It is critical to involve personnel who have experience and are knowledgeable about the process or processes affected by the change.
  • Attention to control of temporary changes. Temporary changes should be managed to ensure they are not forgotten and become permanent by default.
  • Documentation of change and maintenance of corporate memory. Precise information on changes should be recorded in all relevant documentation, process plans, diagrams, and operational procedures, in such a way that it is clear why a particular modification was made.

What does success look like?

  • Within company policy a safety relevant change is clearly defined.
  • The MoC process has a systematic hazard identification and evaluation prizes.
  • MoC procedures are known by all personnel and applied systematically.
  • Initiated changes are tracked all the way through to close-out and all changes are documented in procedures, piping and instrumentation diagrams (P&ID), etc.
  • Temporary changes are closed out and do not become permanent by default.
  • Responsibilities are defined for initiating and authorising changes as well as approval on completion.
  • The MoC process is led by management.

 

Monitoring Performance, Audit and Review

Whether the company has an audit team for process safety (at company or corporate level) is one of the key questions for the assessment of the SMS suitability for monitoring, auditing and reviewing performance. The team should have responsibility for planning and conducting audits, setting audit intervals, determining the content of the audit and ensuring that actions are tracked. Of importance is that the audit team is independent of the operations section which is being audited.  Sometimes a company will not have a formal audit or monitoring system but other audits and routine offer feedback on safety performance.

The role played by Seveso-inspectors in raising awareness of the top-level management for the need to monitor and evaluate safety performance and to provide resources to do so is possibly significant.

Questions that inspectors can pose that address the roles played by the company management include:

  • How does the company monitor its safety performance? Have the figures changed? If so, why? It is important that the management shows commitment to monitoring performance and that practical follow-up takes place and is not just looking at figures.
  • Are near-miss reporting procedures and processes in place to make use of the opportunity to learn?
  • Do processes for collecting and assessing improvement proposals by staff exist?
  • Does a positive failure culture exist (is failure an opportunity for improvement or punishment)?
  • Are aspects of a learning organisation part of the performance monitoring, audit and review processes?
  • Are there regular meetings to follow-up on incidents?
  • Are maintenance tasks on schedule or are they lagging behind?
  • Is training up-to date and appropriate?
  • Does the company use checklists and if so, are they appropriate? How often are they reviewed?
  • Is the lack of accidents and near-misses over a period of time appropriate?
  • What is the quality and systematic approach to following-up on accidents and near misses?
  • How are near-misses taken into account?
  • How does the company follow up on recommendations from the competent authority, from internal audits and others?

For the question of audit and review, the inspector should try to understand how plant safety is integrated into the existing system of evaluating company performance, e.g. annual review. This process should be a documented procedure and note should be taken of the role of a parent company or corporation where existing.

Figure 2: Model of SMS based on ISO 14001

Figure 2:  Model of SMS based on ISO 14001

Safety  performance indicators.

There is a need for objective and consistent measures which address safety critical activities. One possible approach is the use of (Process) Safety Performance Indicators SPIs. If the SMS is effective then the operator should be able to demonstrate that the values within the SPIs are improving or at least constant, that the improvements are maintained over time and that the spot checks by inspectors validate the situation as described by the indicators.

Many inspectors noted that the inspection should include a review of the quality of the safety performance indicators, if the company formally maintains such a feedback system. They offered a number of suggestions to other inspectors on evaluating such systems as part of SMS inspection:

  • The company must use indicators based on its own operations and experience with them. Inspectors should also question why the companies have chosen particular topics for indicators and how the management has determined that they are important.
  • Inspection of the SMS should be based on more than just the output from the indicators. Qualitative feedback, e.g., from audits, occurrence of near misses and accidents, should also be regularly reported with lessons and recommendations extracted and incorporated into the safety management system.
  • Companies should report on competency and training in their indicators. Several examples of measures of training are provided in various guidance that has been published by industry and government on safety performance indicators.
  • Are the right questions being asked? When collecting data on near misses a high collection rate should make the operator proud, at least in the early stage of the programme. There is a need to compare smaller incidents (near misses) to the number of accidents.
  • The quality of the analysis of feedback is important. To evaluate analytical quality, inspectors can inquire about the analytical process, e.g., who performs the analysis, the methods used, and how feedback is selected for analysis (for example, if a dataset is large or certain data are generated continuously). They may also ask to see an example of a report summarising results of an analysis and associated recommendations for follow-up.

A number of publications exist which provide guidance on developing safety indicators:

Responsibility for the SMS. Responsibility for the SMS should be distributed over a number of positions within the hierarchy and involve the whole of the line management. There should be a process embedded in the SMS to check periodically that assigned personnel understand and are performing the tasks allocated in a competent and timely manner. It may be that small site might have one person responsible for the SMS, but for most sites it is not recommended.

Common Success Factors

  • Focus on relevant processes and functions. The audit should be targeted to those aspects of operations that which influence major accident prevention and preparedness. The audit process should be also based on a clear understanding of the role of studied activities in safety performance and their performance expectations.
  • Availability of resources. The use of trained and experienced auditors, as well as making adequate time for the audit, will determine the credibility and reliability of the final results. When internal audits are outsourced, the quality of the outcome will depend on having adequate funding to buy the necessary time and competence to perform the task properly.
  • Management commitment. A successful audit requires support from management throughout all phases, particularly to ensure that action items generated from the audit are adequately addressed. A constructive management attitude also encourages a level of attention and rigour, improving the quality of the audit.
  • Quality of audits and monitoring. Audits themselves should require performance standards. Criteria for judging the quality of an audit include:
    • evidence of procedures for controlling risks,
    • evaluation of how successfully procedures have been implemented,
    • evaluation of effectiveness of procedures achieving safety performance targets (if appropriate),
    • evidence of procedures to identify and reduce problems,
    • observations on non-conformities and substandard practices,
    • observations highlighting examples of good practice.
  • Appropriate selection of process safety performance criteria and indicators. Some characteristics that should be considered include:
    • Tangibility (able to communicate a tangible measure of performance, either qualitative or numeric ,
    • Validity (has validity as a safety performance measure),
    • Reliability (gives consistent feedback on the same underlying conditions) ,
    • Sensitivity (can detect changes in time for corrective action),
    • Transparency (is readily understandable by users).
  • Use of findings to drive improvement. The audit findings should normally include recommendations for immediate corrective actions but also recommendations to explore potentially systemic problems.

What does success look like?

In identifying success the inspector needs to look for:

  • Evidence, via documentation, observation and interviews, that the appropriate behaviours and activities have taken place within the company.
  • Senior management views the audit as an important activity contributing to continuous improvement rather than just a compliance activity.
  • Management is involved in meetings to prepare for audits and discuss results and follow-up.
  • The audit process completes the entire feedback loop of the so-called Deming-Cycle, i.e., Plan-Do- Check Act.
  • All elements of the SMS are reviewed and results of the audit are fed back into the SMS system as a whole. 

 

PDF Version for printing

Vol. 6 Assessment of Safety Management Systems of Major Hazard Sites
Short version(current report in pdf)

Go back to the Common Inspection Criteria website

 

 

Latest update: 5/12/2023 | Top | About this site | Legal notice | Privacy Statement | Accessibility | Cookies | Contacts | Search