Link to .pdf
Link to Seveso Inspection Series Home Page
CIC - Process Safety Performance Monitoring
DEFINITION AND SCOPE
Process Safety Performance Monitoring (PSPM) is a continuous monitoring of compliance with the objectives set by the operator's major accident prevention policy and safety management system. The PSPM is part of the Safety Management System (SMS) (see Figure 1). PSPM generally consists of a composite of both qualitative and quantitative inputs that collectively provide feedback on how well the plant is meeting these objectives. Together with the processes of Internal Audit and Review, performance monitoring is one of the core activities undertaken by the operator to verify the effectiveness of the SMS and implement improvements to correct any identified weaknesses. Figure 2 shows the relationship between the three activities.
Figure 1. Safety performance monitoring is an important part of the safety management system
PSPM centers on the risk management priorities that guide the safety management system. Through the process of hazard identification, the operator should have identified the critical accident scenarios and the control measures necessary to achieve appropriate risk levels. There are likely to be important measures associated with each element of the safety management system, and feedback from the PSPM can suggest improvements to any part of the system, including the PSPM itself.
The senior management should also directly link the performance in process safety to the performance management framework of the entire organisation. Indeed, the results of the PSPM should have the power to influence the strategic goals of the organisation and mechanisms for achieving those goals. In this way, senior management confirms that safety performance monitoring is an important activity contributing to continuous improvement rather than just a compliance activity. This level of commitment provides strong assurance that results of the PSPM receive appropriate attention and that the action items it generates are adequately addressed.
Fig. 2. Relationship between Monitoring, Auditing and Review in respect to scope, frequency and prescriptive nature
FEATURES OF SAFETY PERFORMANCE MONITORING
PSPM is a part of the continuous improvement cycle in the SMS. Performance monitoring should in particular focus on relevant processes and functions and be targeted to those aspects of operations which influence major accident prevention and preparedness. The selection of monitored parameters and the analyses of them should be based on a clear understanding of the role of studied activities in safety performance and their performance expectations.
Logically then, the PSPM process can be viewed as consisting of these essential phases in a continuous Plan-Do-Check-Act cycle:
- Design: Selection of monitoring objectives and feedback mechanisms and parameters to assess performance against the objectives
- Implementation: Monitoring and analysing of the input from the feedback mechanisms
- Corrective action: Communication of findings and implementation of corrective actions to address non-compliance with performance objectives
DESIGN
Selecting the monitoring objectives
The selection of monitoring objectives should be a systematic process, based on established priorities and criteria. All indicators in the PSPM should measure a relevant aspect of process safety. In particular, the PSPM should be targeted to monitor functional elements that are critical to safety, e.g., specific MAPP/SMS objectives and risk control systems associated with major chemical incident scenarios. To identify critical elements of control systems, the operator can consult existing information sources including results of Hazop studies, information on past performance and non-conformances, past incident records, results of internal audits, discussion with operating staff, and other sources with insights on performance and failure potential. The PSPM can include information from all the risk control systems of operational, technical and organisational character, that are put in place to guard against major chemical accidents.
Selection of monitoring targets and parameters
The establishment of monitoring objectives guides the selection of feedback mechanisms and monitoring parameters. To make an appropriate selection of processes and functions to target in the safety performance monitoring, some characteristics to be considered include:
- Validity: The measure should be considered relevant for process safety, justifiable in the context of the MAPP/SMS and the site’s major accident scenarios.
- Reliability: The measure gives consistent feedback with respect to the same underlying conditions over time.
- Sensitivity: The measure can detect changes that are meaningful and in time for corrective action.
- Transparency: The measure is readily understandable for users.
- Tangibility: The measure communicates a clear qualitative or quantitative value relative to performance.
There is no established standard regarding which measures should be part of a site’s process safety performance monitoring system. The composition of the PSPM largely relies on the processes, systems, procedures and practices that are critical to the organization’s ability to achieve its process safety objectives. A good performance measurement system gives valuable information to a site in regard to how well it is meeting its own expectations, and where there needs to progress and how it can be reasonably achieved, so that it can prioritise resources effectively.
Sources of monitoring information
There are a number of information sources that can provide feedback to evaluate performance of objectives against established parameters. Some typical sources include:
- Results of inspections of safety critical plant, equipment and instrumentation
- Documentation of compliance with training, instructions and safe working practices
- Data from automated controls and instrumentation on process deviations and nonconformance
- Records of deviations from and nonconformance with operations and maintenance procedures
- Findings and recommendations from internal audits of SMS elements
- Conformance with design and engineering standards
- Maintenance records
- Analysis of past incidents and recommendations
- Recommendations from process hazard analyses
- Findings from safety tours and observations
- Results of emergency planning exercises
Safety performance indicators in the PSPM
Some of the monitored parameters, or a composite of several similar parameters, can be used as Safety Performance Indicators (SPI), if they represent a good picture of a certain safety state or condition of a plant. There exist a number of publications (HSE, CCPS, OECD, etc.) regarding the development of such indicators. The benefit of such indicators is that they can easily be communicated in the organization and be reported to top management level at frequent intervals. However, SPIs are a part of the PSPM and usually can provide helpful feedback on certain aspects of the safety management system, but not necessarily the whole system. They should be reviewed in the context of feedback from the entire PSPM system, and periodically updated along with the rest of the system.
IMPLEMENTATION
Ideally, a PSPM should be an integral part of the safety management system such that it is embedded in the routine functioning of operations. For this purpose, the system should be practical to implement and have ownership of staff who are involved in production, analysis and use of the information. Moreover, its sustainability and effectiveness over the long term requires management commitment.
Collecting the data
PSPM are more likely to be sustainable when data collection are generated automatically as part of routine operations and procedures. In most cases, implementation of the PSPM is based on data that are already recorded and available at a central source. Companies may create a specific application for collecting PSPM data, linking it to existing data collection systems and forms.
Implementation should identify how the data are collected, who is responsible for assembling the data, and at what intervals. Different data may require collection at different intervals, for example, daily versus monthly intervals, or at intervals based on process conditions, depending on the purpose and character of the parameter.
Involvement of operations staff in implementation decisions can often contribute to a more robust and credible system. For example, they may have better knowledge of data availability and limitations and give insights on avoiding opportunities for data manipulation. Moreover, involvement at this stage facilitates additional contributions of operations staff in analysis and interpretation of results.
Analysing the data
The implementation process requires a process of analysis that designates who analyses the data and at what intervals. The initial review usually may involve selected staff associated with health and safety, operations, maintenance, and other relevant areas, or an existing committee, such as the health safety and environment committee, or inspections or maintenance committees. PSPM responsibilities should normally be distributed over a number of positions within the hierarchy and involve the whole of the line management. There also should be a core of staff involved that have been trained and experienced personnel for monitoring duties.
Analysis should be conducted at reasonable intervals so that there are sufficient data to establish trends but also to allow timely communication of results to prevent incidents. The selection of intervals may be driven by other practical factors, including existing operations and reporting schedules, or the nature of the PSPM parameters themselves. It is also important that monitoring results are evaluated separately from results of internal audits, since PSPM represents a raw picture of the safety condition of the plant that has not been filtrated through the lens of an internal audit.
Analytical protocols should also be established. The analytical process should identify negative and positive performance trends associated with each parameter and provide their interpretation of the significance of these trends in terms of the monitoring objectives and process safety on the site in general. Important changes to or escalation of negative trends should be particularly noted. As much as possible interpretations should be accompanied by insights into causes and influencing factors. There may be additional layers of review contingent on the findings, bringing in perspectives of operations or specialist staff, such as design or safety engineers.
Communicating findings and implementing follow-up actions
Communicating and acting on findings and recommendations represents the last phase of the PSPM cycle. The analysis should normally generate a specific conclusion concerning each parameter, indicating whether the performance level is acceptable and what, if any, action should be taken. Based on this information, outcomes of the analysis should be communicated to senior management accompanied by any actions that need a decision from them or require their approval. Additional communication about findings should also be directed to various functions and levels of management associated with specific findings and recommendations.
Communicating safety performance with senior management should take into consideration:
- What is communicated, e.g., good vs. bad news, only numbers and graphs or with explanations?
- How are the data presented, e.g., is this in a form that managers are used to reading?
- How do managers respond to the data? How is it fed into the decision making process, e.g., budget, manpower, organisation, procedures, contracts, production projects, etc.?
- How is the management response communicated back down the management chain, e.g., rewards, decisions, changes, etc.?
It is particularly important that senior management is periodically informed about the way the SMS is working. Communication of the results of this assessment should routinely travel through the management chain right up to the most senior level. There should be evidence that the indicators have a direct influence on the decision making process at all levels of the management chain. Some typical signs that communication channels are functioning properly include:
- There is evidence that problems needing management attention have been communicated and addressed at the appropriate level
- Documentation (e.g., logbooks, etc.), observation and interviews, confirms that appropriate behaviors and activities have taken place within the company.
- Direct line managers and top management are informed on a periodic basis as to whether the schedule and the defined tasks have been respected
- Improvements recommended for the MAPP and SMS have been adopted
- Plant safety is integrated into the existing system of evaluating company performance, e.g., the annual review. This process should be a documented procedure and note should be taken of the role of a parent company or corporation, where existing.
Despite best efforts, it may be challenging in some organisations to communicate negative findings from the PSPM. It can be evidence of a leadership issue if top management, in particular, the site manager, does not take an active interest in the PSPM, asking for information in a systematic way, rather than only good news. This situation may signal a necessity for corporate management or regulators to look for opportunities to intervene and promote a change in attitude.
Annex of potential inspection questions, verification items, and other advice when inspecting
Download Annex of potential inspection questions, verification items, and other advice when inspecting Safety Performance Monitoring
A. Questions that inspectors can pose addressing Safety Performance Monitoring
General questions
- Does the organization have a policy document relating to Safety Performance Monitoring?
- Does the policy refer to the key risk control systems for prevention of major accidents?
- How does the company monitor its safety performance? Have the figures changed? If so, why?
- Are procedures in place to monitor, record and disseminate the Safety Performance data?
- Are there also procedures in place to make corrective actions in case of non-compliance?
- Is the composition of the team working with Safety Performance Monitoring documented and adequate to the task?
- Are persons with operational expertise involved?
- Does the company use checklists and if so, are they appropriate? How often are they reviewed?
- Has the company ensured that the quantity of data is not overwhelming?
- How is the SPM, the internal audit and the review process of the plant SMS integrated in the existing system of evaluation of the company performance, e.g. the annual review?
Monitoring the progress of planned activities
- How are feed-back on how different plans (for e.g. maintenance and training, see 11 and 12 below and also above for more examples) are being met, reported to the senior management?
- Is training up-to date and appropriate?
- Are maintenance tasks on schedule or are they lagging behind?
- How is the data presented, also compared to other information the top management receives?
- How do managers respond to the data? How is it fed into the decision making process?
- How is this then communicated back down the management chain?
- Does the policy aim to link process safety performance with the remuneration/reward of senior managers?
Reactive Monitoring - learning from accidents
- Are aspects of a learning organisation part of the performance monitoring, audit and review processes?
- Are incidents and near-miss reporting procedures and processes in place to make use of the opportunity to learn?
- What is the quality and systematic approach to following-up on accidents and near misses and analyse the causes, and are these procedures documented?
- Are there regular meetings to follow-up on incidents?
- Does a positive failure culture exist (is failure an opportunity for improvement or punishment)?
- Do processes for collecting and assessing improvement proposals by staff exist?
Shall these questions be removed?
- Is the lack of accidents and near-misses over a period of time appropriate?
- How does the company follow up on recommendations from the competent authority, from internal audits and others?
B. Topics that inspectors could lift for discussion to address the quality of Safety Performance Indicators, if the company formally maintains such a feed-back system [MJV Fulda]
- Ask the operator to give some examples of their Safety Performance Indicators
- Inspectors should question why the companies have chosen particular topics for indicators and how the management has determined that they are important. The company should use indicators based on its own operations and experience.
- Inspection of the SMS should be based on more than just the output from the indicators. See section A above.
- Is the company reporting on competency and training in their indicators. (There are several examples of measurements of training, in guidance on Safety Performance Indicators, published by industry and governments).
- Are the right criteria being used? E.g. when collecting data on near misses a high collection rate should e.g. make the operator proud, at least in the early stage of the program.
- The quality of the analysis of feedback is important. To evaluate analytical quality, inspectors can inquire about the analytical process, e.g., who performs the analysis, the methods used, and how feedback is selected for analysis (for example, if a dataset is large or certain data are generated continuously). They may also ask to see an example of a report summarising results of an analysis and the associated recommendations for follow-up and decision for action.
- Has the company reflected over possible negative side effects of monitoring the specific SPIs that have been chosen?
- Does the policy aim to link process safety performance with the remuneration/reward of senior managers?
References
- Assessment of Safety Management of Major Hazard Sites, Short Report from MJV Fulda Germany 2010
- Another inspection guide for inspection of the use of Safety Performance Indicators is: Assessment of Safety Management System in Relation to Key Risk Control Systems - 1. Process Safety Performance Indicators; Ireland; September 2013
- A number of publications exist which provide guidance on developing safety indicators:
PDF versions for printing
Go back to the Common Inspection Criteria website
