Search

When Investigating Equipment Failures, Don't Stop at the Failure Mode: BlueDragon's Tip of the Week

A recent article on the US Naval Institute News web site by Sam LaGrone, Editor of USNI News, grabbed my attention. It discusses a serious design flaw that is causing propulsion failures in Freedom-class Littoral Combat Ships like the USS Detroit (LCS-7) and USS Little Rock (LCS-9).


A joint team consisting of the Navy, Lockheed Martin (the shipbuilder) and the original equipment manufacturer, conducted a root cause analysis and identified a latent engineering defect in the bearings system that links the ship’s Rolls Royce MT30 gas turbines and the ship’s Colt-Pielstick diesel engines, which power the main drive shaft to achieve the ship’s 40-knot top speed.


Naval Sea Systems Command announced that a design fix was developed by the shipbuilder and is in production, to be followed by factory and sea-based testing and eventual installation in other ships of the same class. In the interim, the Navy will not accept delivery of additional Freedom-class hulls, and they intend to pass along the cost of the fix to Lockheed Martin. (Score one for taxpayers!)


It's an excellent story and you can read the entire article here: https://news.usni.org/2021/01/19/navy-calls-freedom-lcs-propulsion-problem-class-wide-defect-wont-take-new-ships-until-fixed


What grabbed my attention and the focal point of this blog, is that the root cause team from the Navy, Lockheed Martin and the manufacturer, stopped once they identified the design issue causing the equipment failures. However, it is likely that those design flaws are not the deepest-seated causes of the propulsion failures. Consider the huge number of Quality Program requirements, rules, regulations and implementing procedures to design, fabricate, store, transport, install, test, operate and maintain the equipment. Were these elements also evaluated? Stopping a root cause analysis too soon is why so many recurring events take place in government and regulated industries such as Energy, Aerospace, Pharma and Healthcare. The recurring events are expensive, costing those industries hundreds of billions of dollars per year.


As an example, let's look at how much the less-than-adequate implementation of Quality Assurance programs processes and procedures is costing the Medical Devices sector of the Healthcare industry. In a business case report from McKinsey, it calculated the cost of non-routine quality events such as major observations, recalls, warning letters, and consent decrees, along with associated warranties and lawsuits, to be between $2.5 billion and $5 billion per year on average. You can read the detailed McKinsey report here:

https://www.mckinsey.com/~/media/McKinsey/dotcom/client_service/Public%20Sector/Regulatory%20excellence/The_business_case_for_medical_device_quality.ashx

From the article, two things are clear:

  1. The root cause analysis conducted by the team was in line with the majority of other root causes in most industries; they stop as soon as the root causes for the specific event are identified. It is not sufficient to identify the design defect that is causing the failures. The analysis should continue with lines of inquiry that explore how the design defects were introduced. These can lead us to deeper root causes that remain undetected and unaddressed, such as latent weaknesses in the manufacturing process or in the programs and procedures used to fabricate, store, assemble, transport, install, test, operate and maintain the components. The deepest-seated causes often remain undetected, which is why these industries experience recurring events.

  2. The client, in this case the US Navy, does not appear to not know enough about root cause analysis to challenge the shipbuilder and the manufacturer on the depth of their analysis. The skill set needed to solve these complex, human-centric problems extend beyond the knowledge of the manufacturer, Engineers and Scientists. The skills needed to solve these problems are not taught in mainstream courses. To get to the deepest-seated cause requires years of experience and a comprehensive approach that goes beyond the initial equipment failures, to consider other critical areas to investigate such as at-risk behaviors, error-likely situations, latent weaknesses in policies, processes programs and procedures, and weaknesses in management and oversight at all affected organizations (the Navy, the shipbuilder, the manufacturer). A shortage of root cause analysts trained in solving complex, human-centric problems would explain why the Navy did not challenge the results of the root cause.

BlueDragon Tip of the Week: When investigating equipment failures or malfunctions, do not to stop at the immediate cause of the equipment failure (i.e. the failure modes). Continue the investigation until you find what caused those failure modes.


For example, let's say you investigate the cause of a failed motor by using a Fault Tree Analysis (an excellent tool for troubleshooting and identifying equipment failure modes). The analysis identifies that the bearings overheated due to a lack of grease, and that the shaft was not balanced. We can solve those two problems and return the motor to service, with 100% success. However, the deepest-seated causes will continue to cause issues that expose workers to serious injuries and cost the organization time, money and resources for years to come. If we continued our analysis, we would identify the deepest-seated causes that will actually prevent recurrence of future events, once corrective and preventive actions are taken.

  • In the case of the bearings, the deepest-seated cause was the practice of assigning new Engineers the dual role of being the Vendor Technical Reps for contracts, without proper training and experience. The new Engineer/Tech Rep wrote an email to the vendor stating that he would take care of greasing the bearings and that they did not have to, but he only greased the bearings one time. Because there was no Preventive Maintenance schedule developed, the bearings went without greasing for the following 3.5 years. As an inexperienced Vendor Tech Rep, the new Engineer did not realize his email just trumped a contract agreement with the vendor to grease the bearings annually.

  • In the case of the shaft being unbalanced, the deepest-seated cause was that the motor vendor's assembly procedure was flawed and left out key steps. The missing steps could have an adverse effect on other components that are being manufactured, assembled and delivered to other clients. Again, causing problems that would continue to plague their industry for years and at great cost.

As far as the propulsion failures in Freedom-class Littoral Combat Ships, as a former Navy nuclear submarine officer and a taxpayer, I hope the Navy presses the shipbuilder to go beyond the design flaw sand identify the latent organizational and programmatic weaknesses that resulted in those design flaws. Those same deeper-seated causes are likely to be causing other (preventable) problems that are not linked back to the shipbuilder and being fixed at the expense of the taxpayers.


For more information on how to solve equipment issues caused by complex, human-centric problems, visit us on the web at: https://www.dle-services.com/


12 views0 comments