Avoid the 5 Most Common Pitfalls when Implementing Real-Time Industrial Applications

With more than 25 years experienced and involvement in the deployment and implementation of industrial projects related to advanced process control and real time optimization in processing industries around the world, Soteica has identified the following issues as occurring most frequently:

1. Many applications of Process Systems Engineering related technologies are extremely successful when used offline by Engineers (in their offices), but few succeed when transferred to the Operators to be used routinely in real time (tied to online field data) under unsupervised automatic execution
2. The use of certain technologies, deployed under well engineered, successful real time projects, received enthusiastically by Engineers and Operators are no longer used just a few months after project completion
3. Real-time systems with very good up-time figures at commissioning begin to degrade and finally became unused after the Engineer in charge is promoted to a new position - as a result of being so successful with the project
4. Well proven sound technologies fail when applied at a given site, while they succeed in a different location owned by the same company
5. After a failure with a given advanced technology, the site or the Company becomes “immunized” against similar technologies. It is only after several years that, with a renewal of the Engineering staff (young blood that hasn't been exposed to the past failure) that the technology has a chance to be considered again

We will try to identify and address some of the root causes of these issues and discuss the best practices on how to avoid failures in the real-time industrial application. Projects from where our experience has been acquired range from individual pieces of equipment and isolated Units (e.g. a Furnace or a Crude Distillation Unit), to site-wide systems (e.g. a Refinery wide utilities system comprising the steam, fuels, boiler feed water and power networks). All of these were simulated, optimized and/or controlled with real time, online, industrially well-established technologies.

Main industrial project steps and critical details to be taken into account when dealing with real time data and online automatic executions will be explained further. The objective being to achieve a successful technology deployment, avoiding project pitfalls and ensuring the continuous use of the applications based on the proper technology transfer and maintenance.

Real-Time Process Systems

Control rooms worldwide are facing shortages of people; operators are concentrating more and more on the Units under his/her responsibility. DCS screens flourished and hundreds or thousands of tags are available per Unit and a multitude of process diagrams and trends can be projected to the walls and even the ceilings. But only a few employees are available, trained or aware about the strong relation between process operability and economics.

Industrial sites are becoming increasingly complex and inter-related, not only the productive Units but also the power and utilities systems. Tighter and increasingly restrictive regulations related to emissions are also imposing constraints and adding complexity to their proper management. Deregulated feedstock, electric and fuels markets with varying contracted prices (seasonal or daily) introduce additional challenges to Operating personnel. Industrial facilities like Refineries and Petrochemicals are becoming increasingly aware that process and utilities systems need to be optimally operated and managed together.

Real-time process systems engineering applications came to the rescue. Engineers have always attempted to produce tools that improved the way industrial production systems were designed, controlled and managed. The evolution from having plant information scattered throughout many islands of automation to a unified and centralized Plant Information System was an enabling layer for such a work.

However, to successfully implement a real-time process engineering system there are still a few additional issues that need to be addressed and certain implementation practices that could be useful to be taken into account:

Proper Technology Selection

Select the technology that fits with the problem solution rather than force the problem to be solved with an a priority chosen technology. For example, if a simple linear regression with two input variables is sufficient to infer the needed process quality variable, do not force the use of a several inputs neural network. But always high rank systems based on sound process engineering principles (i.e., mass and energy balances and thermodynamics) than those based on correlations or non-physical sense solutions (Friedman, 1999).

Avoid “Computerization”

To solve a given problem, engineers need to apply the EBC approach suggested by Richalet, 2000, rather than the straight use of C. Follow the EBC procedure implies:

• First, use your Eye to inspect trends, data, process diagrams, DCS screens...
• Second, use your Brain to carefully understand and design the real time system and select or design the proper software tool
• Finally, only at the end, use the Computer to solve the problem

The enthusiastic but inexperienced engineers tend to go straight to the Computer, forgetting or minimizing to analyze and perfectly understand the problem to be solved, their real handles and constraints.

Balance Model Complexity against Expected Results

Model complexity must be balanced against expected benefits. Remember: “the simplest is generally the best”. Engineering judgment and experience plays a key role in this task. Operators and Engineers will appreciate any efforts towards simplification, especially for future re-training and system maintenance.

Real Time Data seldom can be used “as is”

Carefully designed and engineered “safety envelops” must be built around real time online data, in order to validate all the signals, in and out of the applications. Minimum and maximum limits for each variable, controller windup identification and associated valve openings and controllers statuses must be combined to properly identify and isolate measurement problems. Steady state detection procedures must also be considered to warn operators or directly skip an optimization run if the process is unstable.

What should a system do if bad data is detected? If the bad data feeds “critical” variables of the system, then its execution should be suspended and Operators warned. If bad data is related to “noncritical” variables then some kind of backup or default values could be used but such an action needs to be defined on each individual variable. The implementation of data validation procedures is a key activity for every real time project.

Pricing Information must be updated

When economics are the main driver to apply advanced process engineering real time systems, prices used for the optimization or objective function calculations must be diligently updated. When possible, entering data manually should be avoided. Automatic pricing information must be implemented acquiring the prices directly from process information systems or web pages (some examples we have used in the past include the sites of ERCOT in Texas, USA and OMEL in Spain, for electricity market spot prices).

Always have operators in mind

From the very beginning try to involve operators, inform them about the project objectives and solve their concerns. A common question is about being replaced by the new system and fired after completion. Second in rank is their worries regarding if the new system will increase their burden. First question is usually outside our control but we never have seen a single Operator being fired due to our projects, most of them very successful. With respect to the second concern, an effort needs to be done to not increase Operator’s burden, implementing automatic data updates or procedures to minimize his/her manual intervention.

Clear and easy operating interfaces need also to be built. Operators must be properly trained and assisted right by the DCS console after being left alone with the new system. It is always a good idea to spend many hours at the DCS console with the operators to discuss the proposed solutions and technology, explaining to them what is needed so they will accept the introduction of the new technology. Properly designed operating manuals must be provided and updated as frequently as needed.

Do not “Prototype” - instead, deploy well engineered systems

Do not expose operators nor final users to “interim or prototyping versions” but only to finished, very well built models, including the operating interface and true handles limits and constraints. Operator’s general attitude is to closely adhere to their first impression. There is a single “silver bullet” to use with them: they will never forget if early disappointed or frustrated when trying to use the system for the first time. Moreover, many of them will became “vaccinated” against the technology and refuse to use it, subtle or frontally, any more.

Document the Application and maintain it “Forever Green”

Engineering documentation must be provided and updated as soon as changes are implemented in the model or to the Operating interface. The application needs to be maintained “forever green”, reflecting continuously the current Process handles and constraints. Changes on the process side must be immediately reflected in the model. Any process change needs to be immediately taken into account because they may impact the way the online variables are considered into the models.

Local resources need to be properly trained and technology providers need to assist and remotely maintain the applications.

As a side note, it is important to mention the huge impact of remote access capabilities on systems performance. Only a few years ago just a minor model adjustment would take a lot of time for the new information to be exchanged back and forth. Nowadays, VPN or Webex like secure remote access can be easily implemented and solutions can be provided in minutes, instead of hours or even days.

Industrial Projects Execution

To cite only a few examples with which we have explored, the following technologies are widely used in the Process Industry and heavily rely on real time data:

• Model Based Predictive Control (MPBC), for example A. Sanz et al., 2005
• Process Monitoring, for example A. Heins et al., 2007
• Abnormal Situation Management, for example D. Ruiz et al., 2002
• Energy Systems Real Time Optimization, for example

1.  M. Kihn et al. for open loop energy management system (EMS), 2008;
2. D. Ruiz et al., 2008 for closed loop EMS applications;
3. D. Ruiz et al., 2011, describing EMS projects and providing industrial application examples;
4. M. Ershaid et al., 2014 for EMS in refining industry and
5. J. P. Ruiz et al., 2015 for open loop EMS in a petrochemical complex

All include software where very efficient and robust numerical methods must deal with real time, online data and cope with the unavoidable uncertainties and errors always present in them. However, even if the piece of software is soundly implemented this by itself, this is not a guarantee of success. Only a well-executed engineering project, carried on by experienced Application Engineers will guarantee the success.

Most of the projects involve activities that are described below:

Collect the Required Information

A document is submitted to the site with all the informational requirements for the real time project. It is very important to define a single project owner from the site side, which will act as a single interface to provide the needed information and coordinate all the project steps.

Kick-Off Meeting

Prior to the Kick-Off Meeting, the provided information will be reviewed to have a better understanding of the site process, main constraints and economic position. Additional questions or clarifications are sent to the site regarding particular issues, as required.  The operating procedures, optimization and control strategies should also be discussed. Operators must be introduced to the project goals and to the technology to be deployed.

Software Installation and Real Time Data Gathering

The software is configured and licensed on the application server PC. It will also be connected to the OPC data server. Remote access to the model also needs to be configured at this time and needs to be available throughout the rest of the project, including maintenance and sustainability periods.

Functional Design Specification

Based on the information obtained during the Kick-Off meeting, a Functional Design Specification document is prepared and approved by the Site. This document becomes the blueprint for the entire project.

Model Building and Configuration

During this stage, the model and the report are built working remotely on the application server. The model is built and grows with continuous access to online real time data. Eventually, a second trip to the facility would occur during this stage and would be used for mid-term review of the model and optimization.

The real time Application will run routinely, but optimization recommendations will still not be implemented but reviewed by the Engineering staff.

During this stage, experienced operators can help to review the system results.

Operators Training and Commissioning

Site engineers will train the operations staff to start and stop the application, understand issues and potential problems and on how to implement the recommendations, when applicable. At the end of this period, engineering and operations staff begin with the daily use of the application, which is transferred to its final owners.

Project documentation will be provided and a benefits report should be prepared based on the current results. It should be submitted to the appropriate management level, to provide the proper feedback about the implemented technology and project execution quality.

Support and Sustainability

After project completion, usually a routine remote check of the health of the online system is performed with the purpose of ensuring the site continues to benefit from its implementation over time. As a part of the program, a regular review regarding the status of the model and the optimization and simulation solutions is proactively done on a periodic basis. A regular report can then be generated and distributed to the appropriate focal point.

Conclusions

Current data in Plant Information systems, which is widely available in process industries, provides the foundation on top of which one can build applications to better operate, control and optimize these processes.

Inherent problems of real-time data need to be addressed by carefully building a safety net around them, in order to avoid the use of unreliable and bad information.

The success of an industrial real time project deployment can be highly increased by following a set of simple rules and ensuring a proper knowledge transfer to both, operators and engineers.

Application maintenance and support helps to sustain the economic benefits after commissioning, allowing them to always be in good shape, continually used, and appreciated by the owners several years after handing the project over to them.