OGE Header

How machine learning contributes to smarter pipeline maintenance

Machine learning can allow oil and gas companies to make better use of the enormous amounts of data as they try to maintain their pipelines.


Before machine learning and pattern detection, sections showing greater than 25% corrosion levels are highlighted in red and only those areas were excavated.  Courtesy: OneBridge SolutionsLast January, a major oil and gas company ran routine inspections of its thousands of miles of pipeline, using the same basic robotic device—the pig—that the industry has used for decades. However, this time, instead of sending data from the pig to a roomful of analysts and waiting months for results, the company applied a solution based on  machine learning and data visualization.

The process identified a point of failure so severe that the pipe gave way as soon as, after excavating to investigate, crews began sandblasting to clean the pipe. Moreover, the pipe ran through an almond grove, where each tree is valued at more than $100,000. If the new machine-learning solution had not discovered the problem, the company estimated its total exposure in the event of a failure at more than $10 million. But with the right tools, they averted disaster.

A system at risk 

That’s just one story. Now consider that there are an estimated 2.7 million miles of oil and gas pipes in North America, running next to factories and farmland, schools and hospitals, mansions and nursing homes. Nearly two-thirds of Americans live within 600 ft. of a pipeline. And in most cases, no one—not the landowners, local authorities, or even the company managing the pipes—knows exactly where it is. National regulators only require map accuracies of plus-or-minus 500 ft. Ask a crew chief how useful that level of precision is when they are out on a dig.

The Pipeline and Hazardous Materials Safety Administration (PHMSA) reported that significant pipeline incidents grew 26.8% from 2006 to 2015. Those are incidents that involved death or serious injury, property damage in excess of $50,000, or a fire or explosion. In 2015, there were 326 reported incidents, nearly one per day.

No one understands this problem better than the industry leaders, who have invested billions of dollars and millions of hours of manpower in efforts to insure pipeline integrity. But despite the best efforts of dedicated professionals, experts are fighting an uphill battle against a relentless foe: time.

Figure 2 - Machine learning and pattern detection – because of machine learning and the ability to ingest and analyze entire data sets, corrosion patterns were visible along the entire section of pipeline. Courtesy: OneBridge SolutionsSome 55% of liquefied natural gas (LNG) pipes in the US are more than 45 years old (see Figure 2). Many of them are quite a bit older and far beyond their projected service life. Each passing year puts more systems at risk of failure. Unfortunately, the technology to support integrity management efforts has not kept pace with the scale of the issue, and that’s putting intense pressure on the teams responsible for preventing incidents.

How big is the gap between efforts and results? A recent Reuters report found only 22% of 466 incidents tracked by Reuters were discovered using advanced detection systems. Most were discovered by members of the public who smelled gas or noticed a fluid leak.

The data bottleneck

The problem with current integrity management solutions isn’t the detection equipment. Pigs have earned their place at the front lines of defense in terms of identifying anomalies. However, some anomalies cannot be identified by current technologies. And there’s no shortfall in the effort and skills of the people tasked to analyze that data. Many have been on the job for decades and have developed the insight and instinct to spot a problem in a sea of data.

The task, however, is overwhelming. Based on 25 years of field experience, one of the authors of this article estimates that only 4% of collected data was being used to manage the pipeline, and even analyzing that 4% was is a gargantuan challenge. 

Why are the numbers so low? Because the data is a huge pain to analyze. The pig produces a blizzard of numbers presented as a spreadsheet—a format that is now more than 30 years old, and better suited to the days of green-screen terminals and floppy disks than today’s era of intelligent, connected systems. It can take months just to break down one report, even under the skilled eyes of senior analysts.

Chart 1: Pipeline incidents 20-year trend. Courtesy: OneBridge SolutionsNow imagine what will happen when the industry’s most seasoned experts retire. The skills they have are not easy to replace, and employers are unlikely to tempt the rising generation of younger workers with a job that requires looking for needles in haystacks using archaic technology.

Despite advances in computing technology, this has been a harder problem to solve than might be thought. Spotting specific patterns and anomalies in a vast sea of barely structured data is a tough task for computers that are great at making a lot of calculations fast, but not so good at tasks that require experience, judgment, learning ability, and observational skills to visualize real-world situations from columns of numbers.

Recent advances

Most recently, machine learning advances are helping to close the gap between what people and what computers can do.  

OneBridge Solutions has been applying emergent technologies to the problems of the oil and gas industry since 2014, building on Microsoft’s Azure: a cloud platform, including services, for cloud-based development and advanced data analytics. 

Machine learning and the vast computing power of cloud technology render the opaque mounds of pig data into clear visualizations within a cognitive integrity management SaaS solution. Features are aligned against the “state” of the pipeline system, creating a new “state,” and then visualized on a 3-D image replicating the actual pipe over the life of the asset, versus run over run. With cognitive integrity management, an entire pipeline system can be analyzed in seconds or minutes rather than days or weeks.

Users drag-and-drop inline inspection tally sheets into the solution, where it accurately ingests and normalizes them into a standard taxonomy in minutes. Machine learning allows for integrity management teams to gather insights into the entire pipeline with ease and simplicity. New recruits learn to use the system and analyze the results in a much more intuitive, visual format. 

It’s not just about speed. Increased precision and confidence in data makes a difference in operational effectiveness, efficiency, and enhanced threat detection. By delivering fast, complete, and accurate results each time the pig is run, there is time to analyze beyond thresholds. An anomaly growth can be tracked over time, before disaster strikes. Rather than single data points along a pipeline, interacting threats can be modeled. 

Integrity managers needn’t fight a constant uphill battle against data, time, and doubt. Decades of unreliable results have led people to question corrosion depth data, even on excavation. Now, with more precise ability to visualize anomalies and validate results before digging, depth uncertainty becomes a thing of the past. Crews armed with better data move faster and cover more ground, with less disruption at the worksite.

Getting the entire picture 

Cognitive integrity management enables integrity management teams to gain insight and visibility into patterns and trends that would otherwise be impossible to detect due to the sheer volume of data presented to the user. Pattern detection provides some of the greatest value to integrity management since it unlocks to power to optimize dig operations. As shown in Figure 1, an integrity team would have determined that the areas highlighted in red were the only sections along the pipe with levels of corrosion that pose an immediate leak threat. As a result, these were the only sections of the pipe that were repaired. Currently within the industry, the thought process is to respond to corrosion growth and simply target immediate threats rather than conducting a comprehensive analysis of all threats, both immanent and trending. 

As shown in Figure 2, leveraging machine learning and OneBridge Solutions’ pattern detection algorithm, the solution analyzed corrosion growth along the entire section of pipeline. When comparing Figures 1 and 2, corrosion was prevalent throughout the pipeline. The sole reason for the anomalies not being analyzed or detected when the pipe was excavated was that corrosion levels had not reached minimum threshold levels. Now, through advanced technology, it became clear the entire pipe was affected. Further analysis determined that the corrosion growth was due largely to faulty tape-coating, something that would have been missed altogether in previous integrity management practices. 

Figure 3 – Cognitive integrity management solution shows a 3-D visualization of the patterns detected along the pipeline with capability to drill down into specific areas along the pipe. Courtesy: OneBridge SolutionsFigure 3 shows the pipeline through 3-D visualization, where the pattern became much clearer and more defined. Pipe zones, specific target areas consistent with individual threats, were identified and searched for threats. In this illustration, corrosion growth associated with tape coat failure, in an aerobic environment, commonly called “tape bagging,” is visible. The image was enhanced by adding interacting threat components, such as crack and dent data. Further development of this technology may advance the detection of SCC

In the situation outlined above, OneBridge Solutions was able to clearly point-out scenarios where pipeline operators could further optimize dig management through greater visibility into their pipes.  With the advancement of machine learning, pipeline operators can be better-equipped to optimize their pipeline integrity management efforts. Dig management becomes a straightforward task, with the confidence to know that the entire sections of pipe that are affected are repaired, rather than just sections that pose imminent risk. 

Finally, machine learning makes it easy for information officers and business decision-makers to say yes to the tools integrity management teams need. This solution meets rigorous enterprise standards for data security, compliance, and manageability. It won’t tax the resources of a stretched-thin IT staff. The goal is to make tedious data mining a thing of the past and free business leaders to make intelligent decisions on solid data with little effort.

Put innovation in the pipeline 

Every time a pipeline fails, lives, property, and the environment are at risk, and so is the reputation of the entire industry. For years, integrity managers have been the last line of defense, pitting their ingenuity, skills, and experience against a rising tide of aging systems and a growing mountain of data. 

Machine learning provides a way to give an organization a competitive advantage in the drive toward zero failures. It enlists the most powerful data and computing technologies on the market to help operators predict pipeline failures, increase safety, reduce operational costs, and increase overall confidence in the industry’s ability to manage its assets responsibly.

Leveraging the power of machine learning, a costly and damaging failure of the pipeline running through the almond grove was avoided. But that’s just the start of a journey where others across the industry embrace smarter solutions that address the looming crisis of maintaining aging pipes. Modern cloud technology and emerging machine learning solutions, delivered as a service, enable industries of all types to embrace an impending digital transformation. Applying these technologies to pipeline integrity management provides much more: it unlocks the possibility of zero pipeline failures. 

Tim Edward is co-founder and president of OneBridge Solutions, and a 30-year veteran of the oil and gas industry. He has spent the past 22 years pioneering software applications for the pipeline industry. 

Rob Salkowitz is an author, educator, and consultant whose work focuses on the social and business impact of technology innovation. He writes regularly for Forbes and serves on the faculty of the University of Washington CommLead program. 


click me
click me