Moneyball for Manufacturing

I'm quite behind the times when it comes to watching movies. The last movie I saw was The Dark Knight Rises...

at a matinee...

so I don't get shot.

A few nights ago, I finally sat down and watched Moneyball, the movie with Brad Pitt and six Oscar nods. It is a "based on a true story" of how the perennially under-budgeted Oakland A's baseball club builds a near-championship team only to lose not only playoff games, but also their best players to big-money baseball clubs when the players' contract expire.

The Oakland A's general manager, Billy Beane, realizes his underfunded system will continue to produce good-enough results that will never win the championship. And to continue running his system the same way is insanity:

Doing the same thing over and over again and expecting different results. - Albert Einstein

To win, Beane decides to do something different, and that something different is focusing on the key performance indicators (KPIs) of winning and getting players that contribute positively to those KPIs... applying statistics and math to baseball is what they call, "Moneyball."

How many of us are in the same boat as this Oakland A's GM?

• How many of us are getting by with under-funded budgets?
• How many of us are managing our systems the same way they've been managed for years?
• How many of us can improve our systems by applying data-driven statistics?

Moneyball is to baseball what Manufacturing Sciences is to manufacturing:

• Having the data
• Understanding the system (which factors impact your KPIs)
• Applying the understanding to increase productivity or system capability.

Biotech and pharma manufacturing is in a period of static or diminishing budgets. Do more with the same or make do with less is the general mantra as the dollars go towards R&D or to acquisitions. To make matters worse, biosimilars are coming on-line to drive revenues even farther down.

by Oliver Yu

Questions I'm getting these days are:

What systems do I need to collect the right data?

What KPIs should I be monitoring?

What routine and non-routine analysis capabilities should I have?

Let's Play

p.s. - Watch the movie if you haven't seen it.  It's as good a movie as it is a good business case study.

How to Make IR Control Charts

by Oliver Yu

Suppose you support a batch process. The way you likely measure performance is to sample each batch and measure different parameters. These measurements are ideal for plotting on an IR control chart - one control chart for each parameter and each batch would be represented by one point on the control chart.

If you have statistical software like JMP, then you can just click around on the menu



...and...



control charts appear like magic:



But suppose Wall Street bankers crashed the economy by securitizing AAA-rated subprime mortgages and you are the collateral damage; forking over $1,250 for a single-user annual license or $1,895 for a single-user perpetual license of JMP isn't in the cards. What do you do?

Good news. William Shewhart developed control charting principles long before computers so if worse comes to worse, you could probably create a control chart from graph paper and a grease pencil.

Here's what you do:

1. Get the data into a column
2. Compute moving range
3. Multiply MR by 3 and divide by 1.128

We're not going to do it with a grease pencil and graph paper. We're going to do it with a spreadsheet.

Step 1: Get the data into a column

We haven't talked about this yet, but data for analysis needs to be structured. If you look at the numbers in a column and they represent what the column headers describe, then you got it right.

Step 2: Compute the Moving Range

This is where you take the absolute value of the difference between measurements. =B3-B2 would be the formula that you'd drag in column C. The average of the moving range is used to determine the width of the control limits.

Step 3: Compute distance to control limits

To get the distance to each control limit, compute 3 * Average( MovingRange ) / 1.128.



In this case, the average of the moving range is 3.90. Take 3.90 * 3 / 1.128 = 10.37.

The Upper Control Limit (UCL) is the 296 + 10.37 = 306

The Lower Control Limit (LCL) is the 296 - 10.37 = 286

What you do is calculate limits for every parameter you measure; apply it to a steady process and lock the limits and monitor the process against the locked-down limits to detect drift.

Get Control Chart Experts

Continuous Process Improvement and SPC

by Oliver Yu

Buzzwords are aplenty in this line of work: Lean manufacturing, lean six sigma, value stream mapping, business process management, Class A. But at the end of the day, we're talking about exactly one thing: continuous process improvement:

How to get your manufacturing processes (as well as your business process) to be better each day.

And to that, I say: "Pick your weapon and let's get to work." For me, I prefer statistical process control because SPC was invented in the days before continuous process improvement collided with information technology.

Back in those days, things had to be done by hand, concepts had to be boiled down in simple terms: special cause vs. common cause variability could simplify what was going on and clarify decision making. And having just Excel spreadsheets is a vast technological improvement to paper and pencil. In those days, there was no time for complexity of words and thought.

If we say words from the slower days of yesteryear, but use tools from today, we can solve a lot of problems and make a lot of good decisions.

Companies like Zymergi are third-party consultants who can help develop your in-house continuous process improvement strategy; especially for cell culture and fermentation companies. We focus applying statistical process control as well as knowledge management so that once we reduce process variability and increase reliability.

The technology is there to institutionalize the tribal knowledge so that when people leave, your high-paid consultants leave, the continuous process improvement know-how stays.

We use SPC and statistical analysis because it has been proven by others and it is proven by us. Data-driven decisions deliver real results.

7 Tools of SPC

1. Control Charts
2. Histograms
3. Correlations
4. Ishikawa Diagrams
5. Run Charts
6. Process Flow Diagrams
7. Pareto Charts

Control Chart Limits vs. 3 StDev

by Oliver Yu

While control limits are approximations of 3 standard deviations, they are not 3 standard deviations.

In thermodynamics, we talk about state variables and path variables. State variables - like internal energy (U) … "is what it is." Other variables like work (w) are path variables… "its value depends on how you got there."

Standard deviation is a "state"-like parameter… if you have a set of points, the standard deviation is the standard deviation; it does not matter the order in which the data happened.



Using the same data from our previous control charting example, we see the standard deviation is 2.9 and a mean of 295. The 3 standard deviations around the average is 286 - 303.

Control limits, on the other hand, are path-like parameters that depend on the order in which it was received, and in the case of pretty random data, the control limits are 285 - 306... which is pretty close to the 3 standard devations, but not exact.




Viewing the control chart, it's obvious there are no special cause signals and there are no patterns in the data that indicate the data is out of the ordinary.

But suppose we got the same exact measurements... except this time, we found that each observed value was equal to or higher than the previous:




The standard deviation remains the same and therefore average +/- 3 standard deviations remains the same: 286 - 303. But look at the control limits... they have tightened significantly to 292 - 298.

This is because the control limits are computed from the moving range, and is when the same data shows an ascending pattern, the control limits are able to shrink and flag special cause signals where the standard deviations are not.

Apply 3 standard deviations where they are applicable; they are not applicable when identifying special cause signals of stable processes.

Control Charts for Bioprocesses

by Oliver Yu

A control chart is a graphical tool that helps you visualize process performance. Specifically, control charts help you visualize the expected variability of a process and unambiguously tells you what is normal (a.k.a. "common cause variability") and what is abnormal (a.k.a. "special cause variability").

Discerning common-cause from special-cause variability is crucial because responding to low results that are within expectation often induces more variability.

So up to this point, we know that low process variability allows us to detect changes to the process sooner. We also know that low process variability enables processes with higher capability.

Below is the control chart of the buffer osmo data from a previous blog post on reducing process variability.



The -green- horizontal line is the average of the population and the -red- lines are the control limits (upper control limit and lower control limit). Points that are within the UCL and LCL are expected (a.k.a. "common"). Points outside of the limits are unexpected (a.k.a. "special"). From the control chart, you can immediately see that the latest value of 301 mOsm/kg is "normal" or "common", and that no response is necessary.

Below, you see the control chart for the second set of data and how a reading of 297 mOsm/kg after 8 consecutive readings of 295 mOms/kg is anomalous and certainly worth an extra look.



There are all kinds of control charts and they have a rich history - worth reading if you're into that kind of thing. In batch/biologics processes, each data point corresponds with exactly one batch and so the type of control chart used is the IR chart.

It is important to know that the control limits are not computed from standard deviations - they are computed from the moving range... without going full nerd, the reason behind this is that control limits are sensitive to the order in which the points were observed and narrow when there is a trending pattern in the data.

Control charts for key process performance indicators are a must for any organization serious about reducing process variability. Firstly, control charts quantify variability. Secondly, control charts are easy to undertand. Lastly - and most importantly, control charts help marshall scarce resources by identifying common vs. special cause.

SPC - Control Charting

by Oliver Yu

No book on statistical process control is worth its salt if it fails to mention control charting; and true to the form of being solid on SPC, so does this pocket book:


Readers of this blog know well the necessity of control charting for process/campaign monitoring.

So it ought not to be surprising that we have yet another blog post about control charting. If you're really serious about reducing process variability, control charting is the highest impact, lowest cost method for establishing a baseline and understanding your status-quo process.

Everything that falls inside of the upper and lower control limits is expected variability (i.e. "common"). Since it is expected - don't do anything with it. Resist management tampering and don't waste resources investigating that which is expected.

Any point that falls outside of the upper and lower control limits is unexpected variability (i.e. "special"). Save your resources to investigate these points: chances are, you'll find something.

What hasn't been discussed here is within-control-limit patterns that can be considered special-cause. For example, 7-in-a-row on the same side of the centerline is a special cause even if no point has exceeded the control limit. Here are 4 other rules detailed later in the pocketbook:


And even farther on in the book are pages telling you how to compute control charts:


In this age with fast computers and JMP, it isn't a good use of your engineers' time to go back that far to derive the control charts limits.

Related posts:

• How to make IR control charts
• Control charts for Bioprocesses
• Control limits vs 3 standard deviations

Process Troubleshooting using Patterns

The variability in your process output is caused by variability from your process inputs.

This means that patterns you observe in your process output (as measured by your key performance indicators, or KPIs) are caused by patterns in your process inputs.

Recognizing which pattern you're dealing with can, hopefully, lead you quickly to the source of variability so you can eliminate it.

Stable

Boring processes that do the same thing day in and day out are stable processes. Everyday you show up for work and the process is doing exactly what you expected. Control charts of your KPIs look like this:


Boring is good: it is predictable, you can count on it (like Maytag products) so you can plan around it. Well-defined, well-understood, well-controlled processes often take this form. The only thing you really have to worry about is job security (like the Maytag repairman).

Periodic

Processes where special-cause signals show up at a fixed interval exhibit a "periodic" pattern.


This pattern is extremely common because in reality, many things in life are periodic:

• Every day is a cycle.
• Manufacturing shift structures repeat every 7-days.
• The rotation of equipment being used is cyclical
• Plant run-rates

On one occasion, we had a rash of production bioreactor contaminations. By the end of it all, we had five contaminations over the course of seven weeks and they all happened late Sunday/early Monday. On Fridays going into the weekend, people would bet whether or not we'd see something by Monday of the following week. Here, the frequency is once-per-week and ultimately, the root cause was found to be related to manufacturing shifts, which cycle once-per-week.
All these naturally occurring cycles at varying intervals and the key to solving a the periodic pattern is identifying the periodic process input that cycles at the same frequency.

Step-change

A step-change pattern is when, one day, your process output changes and doesn't go back to the way it was... not exactly "irreversible", but at least "difficult to go back."


Step patterns are also commonly observed in manufacturing because many manufacturing activities, "can't be taken back." For example:

• After a plant shutdown when projects get implemented.
• After equipment maintenance.
• When the current lot of material is depleted and a new lot is used.

One time coming out of shutdown, we had a rash of contamination: every single 500L* bioreactor came down contaminated. It turns out that a project securing the media filter executed during changeover for safety reasons changed the system mechanics and caused the media filter to be shaken loose. Filter stability was restored with another project so that the safety modifications would remain.

Step pattern is harder to troubleshoot than the periodic pattern because the irreversibility makes the system untestable. The key to solving a step pattern is to focus on "irreversible changes" of process inputs that happen prior to the observed step change.

Sporadic

A sporadic pattern is basically a random pattern.


Sporadic patterns are unpredictable and difficult to troubleshoot because at its core, the special-causes in-process outputs are often caused by two or more process inputs coming together. When two or more process inputs combine to cause a different result than if either two inputs alone, this is called an interaction.

A good example is the Ford Explorer/Firestone tires debacle that happened in the early 2000's. At the time, they observed a higher frequency of Ford Explorer SUVs rolling over than other SUVs. After further investigation, the rolled-over Ford Explorers had tires mainly made by Firestone. Ford Explorer owners using other tires weren't rolling over. Other SUV drivers using Firestone tires weren't rolling over. It was only when Firestone tires AND Ford Explorers used in combination that caused the failure.

To be blunt, troubleshooting sporadic patterns basically sucks. The best thing about a sporadic pattern is that it tells you is to look for more complex interactions within your process inputs.

Summary

Because the categories of patterns are not well defined (i.e. "I know it when I see it"), identifying the pattern is subject to debate. But know that the true root cause of the pattern must - itself - have the same pattern.