Controllable Parameters of Mammalian Cell Culture

Einstein famously said:

Insanity [is] doing the same thing over and over again and expecting different results.

Which got me thinking...

Question: Is there such a thing as doing the same thing over and over again and getting different results?

Answer: Biotech Manufacturing

Rob Caren once asked: "How hard can (large-scale) cell culture be? It's ONE unit operation."

He also observed that large-scale chromatography shouldn't be that hard either:

• Send the pool through that fixed-bed reactor.
• When the optical density is not zero shut this valve and open that valve.
• Try not to send product to drain.

On both counts, he's right, but I can only speak to the cell culture side.

When running bioreactors, there are a few parameters that are within your control. These parameters are sometimes called "knobs" because the production team can literally go to the control system and "turn a knob" (or click a few buttons on the SCADA) to change the parameter.

For the bioreactor or fermentor, those knobs are:

pH. The intracellular pH is known to change the activity of enzymes that run the rates of metabolic reactions. Since cells regulate intracellular pH, the best you can do is to control the extracellular pH. In reality, the pH for the process is specified. And if specified well, the specification will come with a target range and a proven acceptable range. At commercial scale, you ought to be able to operate within the proven acceptable range. For more information on pH control in mammalian cultures, see this.

Dissolved oxygen. Maximizing cellular productivity means aerobic metabolism. From university biochemistry, we know that anaerobic metabolism produces far less energy (2 ATP) than aerobic metabolism (38 ATP). We also know from chemistry that the solubility of oxygen in water is quite low (<10 mg/L when temperature > 15 degC). Therefore, it is important that the bioreactor supplies oxygen. Some bioreactors supply air and others supplement with oxygen. While the dissolved oxygen range is typically specified by the process, the air supply, air/oxygen mix, flow rates and sparge-type can be determined by the facility. As previously discussed, dissolved oxygen depends on other parameters such as agitation and temperature and can be changed within the specified range.

Temperature. Temperature control happens with the bioreactor jacket where water is flowed around the outside surface of the bioreactor. When the temperature gets too hot, the control system sends cooler water; and when the temperature of the culture gets too cold, the control system sends hotter water. While temperature is typically specified, there are processes that will intentionally cool the culture to reduce the rate of metabolic reactions and extend culture viability. Also, since temperature is defined in a range, the setpoint is a turnable knob.

Agitation. Agitation is typically not specified by the process, just that the cells must be suspended (i.e. not settled on the bottom of the bioreactor).  In practice agitation rate is determined by power-to-volume calculations and stays constant for the bioreactor, nonetheless, this is a manipulatable parameter when running bioreactors.

Timing of Inoculation. Inoculation density is often specified by the process. But there's no way to "dial down" or "dial up" inoc density in a control system somewhere like you can with pH, dissolved oxygen, temperature and agitation. When cells grow, the cell density naturally increases, so the way to control inoculation density is to time it (i.e. wait vs. not wait).

Timing of feeds. In fed-batch cultures, additional nutrients are added to the culture. The additional nutrients tend to increase the osmolality and the additional volume can help dilute the cellular waste (like ammonium). Not in all processes, but in some processes, the timing of feeds have been shown to impact culture productivity.

Timing of shifts. Some processes are specified with changes in setpoints of the aforementioned parameters (e.g. pH or Temperature). The shift specifications come in the form of: "When the culture duration reaches X hours" or "when the cell density reaches Y x 10⁶," then change the set point up/down.

When building multivariate models, it is crucial that controllable parameters are modeled as factors and here's why:

When your model shows can correlate significant main effects and interactions to some process output (e.g. titer or quality attribute), you can actually step out of theory and prove it in practice.

ZOOMS 2 - Fastest Way to View Trends

So in addition to biologics manufacturing commentary, it turns out that Zymergi is actually a for-profit business that provides software, consulting and technical support.

For 2014, we're pretty much booked on the technical support side, but I wanted to take some time to talk about our software products.

Our flagship product is ZOOMS, which is an acronym for Zymergi Object-Oriented Metadata Search.  And in 2014 - thanks to Edward Snowden, more people know what metadata is than in 2008 when we started ZOOMS (v1.0)

So why are we interested in searching metadata? Well, let's take a step back. When working the front lines of campaign monitoring and process support, we noticed that viewing trend data (i.e. tag values plotted against time) was the principal component of process understanding. The more trends a person reviewed, the more process knowledge they gained and the more they understood normal from abnormal.

And in all that time, very few people actually learn to speak "Automationese."

"Hey, did you see that weird thing going on with T100.GLOBAL.CLX.AI05.PV?"
- No one ever

In the automation world, everything is a tag. In the Manufacturing Sciences world, everything is about a measurable parameter within the process. So when you listen to the process scientists and engineers talk, it's always about some parameter (e.g. "Optical Density") in some unit (e.g. "Tank 100"). That right there is the metadata of a tag.

The tag takes care of the Y-axis on the trend. What about the X-axis?

The X-axis deals with time-windows: start times and end times and the metadata around the time-windows are called, "batches." Specifically using S88 terminology, people involved with campaign support are interested in Unit Procedures, a.k.a. "unitbatches."

I'll leave the formal definition of "unit procedure" up to the automation gurus, but to a Manufacturing Sciences data engineer, a unit procedure is a process that happens on a piece of process equipment.

So say you're making rituximab in production bioreactor T100 using the version 1.1 process and batch R2D201 ran from 20-Dec-2013 to 28-Dec-2013... that there is a unit procedure:

batchid	unit	product	procedure	start time	end time
R2D201	T100	rituximab	production culture version 1.1	20-Dec-13	28-Dec-13

The metadata around this time-window (i.e 12/20/2013 to 12/28/2013) are as follows:

• R2D201
• T100
• rituximab
• production culture version 1.1

So it stands to reason that if an internet user who knows nothing about a subject can type keywords into Google and get the most relevant results on that subject; that in 2014, a process citizen who doesn't know too much about the process ought to be able to type some keywords into a webpage and get some process trends.

And now they can: Introducing ZOOMS 2:



Learn More

Multivariate Analysis: Pick Actionable Factors Redux

When performing multivariate analysis, say multiple linear regression, there's typically an objective (like "higher yields" or "troubleshoot campaign titers"). And there's typically a finite set of parameters that are within control of the production group (a.k.a. operators/supervisors/front-line managers).

This finite parameter set is what I call, "actionable factors," or "process knobs." For biologics manufacturing, parameters like

• Inoculation density
• pH/temperature setpoint
• Timing of shifts
• Timing of feeds
• Everything your process flow diagram says is important

are actionable factors.

Examples of non-actionable parameters include:

• Peak cell density
• Peak lactate concentration
• Final ammonium
• etc.

In essence, non-actionable parameters are generally measured and cannot be changed during the course of the process.

Why does this matter to multivariate analysis? I pick on this one study I saw where someone built a model against a commercial CHO process and proved that final NH4+ levels inversely correlates with final titer.



What are we to do now?  Reach into the bioreactor with our ammonium-sponge and sop up the extra NH4+ ion?

With the output of this model, I can do absolutely nothing to fix the lagging production campaign. Since NH4+ is evolved as a byproduct of glutamine metabolism, this curious finding may lead you down the path of further examining CHO metabolism and perhaps some media experiments, but there's no immediate action nor medium-term action I can take.

On the other hand, had I discovered that initial cell density of the culture correlates with capacity-based volumetric productivity, I could radio into either the seed train group or scheduling and make higher inoc densities happen.

by Oliver Yu

Contamination Control of Cell Culture Bioreactors

by Oliver Yu

"Contamination Control"

A misnomer. I can see how they got that name... from "Pest Control," but I still hate it.

"Bioreactor control" makes sense as cell culture manufacturers try to direct the behavior of pH, dissolved oxygen, temperature, agitation, pressure...

But "contamination control"? No one is trying to direct the behavior of bioreactor contamination: Everyone tries to abolish bioreactor contaminations.

The abolition of bioreactor contamination in a large-scale setting is generally a team sport. It can take just one person to solve the contamination. But usually, the person who figures out what went wrong (the brains) is unlikely the same as the person who implements the fix (the hands). And in a GMP environment, the change implementation is a coordinated process involving many minds, personalities, and motivations. With all those people come an inordinate amount of politics for a goal that everyone seems to want to reach: no contams.

Immediate-/Medium-term Fixes

The first thing to realize is that operations management is usually the customer when it comes to solving bioreactor contaminations. Everyone's butt is on the line, but no group burns more resources responding to bioreactor contaminations than them. And in my experience, there is no "short-term" vs. "long-term" solution.

To operations management, there's just immediate solution and medium-term solution.

• Immediate solution :: what are you going to do for me today?
• Medium-term solution :: what lasting solution are you going to implement after the immediate solution?

Science... if it fits

The second thing to realize is that there's no room for science. The prime objective is to stop the contaminations. The secondary objective is to find the root cause. If identifying the root cause helps stop the contamination, that's a bonus; but root cause or not, you still must stop the contaminations.

For example, if your contamination investigation team thinks that there are five contamination risks, the directive from management will be to implement all CAPAs necessary to address all five risks. If the fixes work, "Great! You met the objective." Do you know what the true root cause was? Not a clue (it was one of those five, but you'll never know which one).

Political

The third thing to realize is that contamination response is as much political as it is technical.

• You can have the right solution, but present it the wrong way - and it's the wrong solution.
• You can formulate the right solution that is not immediately actionable, no one wants to hear about it.
• You can irrefutably identify the true root cause (thereby shining light on GMP deficiencies), and run against resistance.

Being right is different than being effective. And "bioreactor contamination control" at large-scale requires effectiveness. For in-house resources, it requires a keen understanding of interpersonal dynamics. For organizations that are at either a technical or political impasse, there are external bioreactor consultants who understand how to effectively troubleshoot and abolish bioreactor contaminations.

Abolish Bioreactor Contaminations

Post-Licensure Cell Culture Process Improvements

There's a great article out in GEN on cell culture process improvement, in particular, the Dr. Yuval Shimoni segment on the "low hanging fruits" of post-licensure improvements.

From the article:

At the CHI conference, Dr. Shimoni demonstrated how changes to cell culture media can make a difference by increasing production capacity through greater cellular productivity.

As I didn't go to the conference, I am left thinking that his feat was pretty impressive.  Changing media components post-licensure is quite daring.

The biologics license agreement (BLA) will call out the exact ingredients +/- percentages on each media component.  And changing a single component can (and has shown to) alter product quality.

Changing several media components, if in fact, that's what he did, is quite the feat and would take testicular fortitude of magnitude 10 on the Moh's scale: Any adverse impact on product quality - no matter the cell productivity improvements - is unwelcome.

Pulling off a media-change post-licensure is not only a technical accomplishment, but a political one as well.  

Unnecessary Testing in cGMP World

by Oliver Yu

In the world of soaring medical costs, we have unnecessary medical tests and procedures sapping precious healthcare dollars.

The thing about these medical tests is that they are necessary for someone under some circumstances, just not for most people under most circumstances.

So is the case in the world of GMP biologics manufacturing. There are

Image courtesy of the Chemical Heritage Foundation Collections.

plenty of tests that need to happen to produce a releasable lot. There are in-process tests; there of Certificate of Analysis (CofA) tests.  There are analytical tests you for engineering runs; there are tests you perform on contaminated lots, but not others.

But regardless of what test you are performing, the litmus test for performing the test or analysis is:

Does the result of the test help me make a decision?

Consider the following snippet from a computer program:

   if ( testResult == PASS )
   {
      forwardProcessBatch();
   }
   else
   {
      forwardProcessBatch();
   }

In this case, if I run a test and pass, I get to forward process the batch. If the test fails, I still get to forward process the batch. So if in either case, I get to forward process the batch, why should I bother doing the test? The result of the test does not do anything to serve the outcome!

Another good way to approach the question of whether or not to perform a test is to see if you can write down a plan for what to do with the test result. If you can write down a reasonable plan and stick with the plan prior to getting the test results, then there's a good reason to perform the test; otherwise, you're simply on a fishing expedition and making it up as you go along.

FIO Samples

There exists "For Information Only" samples that are specified into the process.  For example, concentrations of ammonium (NH₄⁺), sodium (Na⁺), pO₂ and pCO₂ are measures of cell culture metabolism that are useful for long-term process understanding.  They likely never be used to make a forward-process decision, though they can be used for retrospective justification of discrepancies or as variables during multivariate data analysis.

In my experience, these routine FIO samples are contentious.  On one hand, they serve the purpose of long-term, large-scale process understanding as well as sporadic justification for discrepancies.  On the other hand, if FIO samples get used enough to close discrepancies and release lots, over time the FDA and other agencies will pressure you into making these FIO tests into in-process or lot-release tests.

Defensibility

In the end, your actions in deciding to perform a test need to be defensible.  You need to defend the costs to do the test to management.  You need to defend not doing the test to the FDA.  And your situation may be different than the biologics manufacturer down the street.

That defense ought to rest on whether or not you can do something with the result of the test.

Who Are You Guys, Anyway?

So, I asked for a report to study Zymergi blog readers, and here's where the biotech/pharma readers are coming from:

This is a veritable who's who of the biotech world.  Obviously, you aren't all customers, but when it comes to large-scale biologics support, cell culture and bioreactor contaminations, readers and customers find themselves in good company.

Thanks for reading.

Note: All logos/trademarks belong to the trademark holder and inclusion on this list is not an endorsement of Zymergi or vice versa.

Pros and Cons for Outsourcing Process Development

by Oliver Yu

I visited a contract development and manufacturing organization (CDMO) a few months ago.

What CDMOs do is they develop the process for you. And if you choose, they'll also execute the manufacturing process for clinical or cGMP material.

In short, if you're outsourcing either process development or manufacturing, CDMOs are the people you outsource to (pardon the preposition-ending sentence).

Dan Stark, currently senior director in the Global MSAT group at Genentech, once said (paraphrased):

The winners of the age of biotechnology will be the people who can translate research discoveries into commercial product and those who already have the infrastructure will have a head start.

What he's saying is if you're just a research outfit and you come up with new molecular entities, you're not in the position to extract the full value of that NME. Say you're IDEC pharmaceuticals and you come up with Rituximab... unless you can get the product into the hands of consumers, you're just a research outfit.

Likewise, if you are a contract manufacturer like Lonza where you have the manufacturing know-how and can produce clinical or commercial material, you also are not in the position to extract the full value of your know-how simply because you rely on someone else for your pipeline. In a lot of ways, you're a commodity (see Samsung getting into the biologics manufacturing space with Samsung Biologics).

Dan is saying that the people... the companies who can take a NME through the clinic... through the FDA approval process and be able to make the drug product are the ones who can extract the full value.

So why would anyone with a NME outsource the crown jewels of their business and go with external process development?

It turns out there are a lot of reasons.  To be sure that we got it right, Zymergi has collaborated with Dr. KC Carswell Ph.D of Carswell Bioprocess Consulting to produce a whitepaper on outsourcing process development.

If you're thinking about sending your process development into the hands of a 3rd-party, you definitely need to read the pros and cons of outsourcing.

Get FREE Whitepaper on Outsourcing PD

Rebuttal to Atmospheric Breaks on Drains

Here's some feedback from an industry colleague regarding air-breaks on drains:

For one thing, BSL-2+ areas like [highly toxic] bacterial fermentation suites require the facility to have closed piping to avoid or minimize aerosol effects and biohazard contamination of people and the environment around the fermentor.  The BMBL 5th Edition (basically the biosafety bible) requires it for BSL-2 and above organisms.

Clearly we know that protecting workers is clearly paramount to safety.  But we also know that not every process uses "BSL-2 and above organisms."  A lot of facilities are designed "just in case" the expression system produces biohazard.

Closed-pipe drain headers requires a deep understanding of SIP cycle design and implementing a robust recipe to get rid of vacuum.

So in closing:

FDA's Metadata is Public. FOIA through 3rd-Parties.

A month ago, former-NSA-employee turned whistleblower: Ed Snowden, revealed far-reaching surveillance capabilities of the National Security Agency, specifically: metadata collection.

What is Metadata?

Metadata refers to data about the data.

Sort of weird to refer to something that way, but here's a simple example:


In fact, our OSI PI historian search engine: ZOOMS stands for Zymergi Object-Oriented Metadata Search.

• The data itself is time-series data.  
• The metadata is all the information that describes it:
  "V7410 pH" is metadata that our search engine archives.

What the US federal courts are saying is that the data (content of phone calls, content of emails) is protected by the 4th Amendment; but that the metadata (sender, receiver, time of call, duration of call, etc.) is not and therefore available for archival by the NSA for "fighting terrorists."

What does this have to do with biotech manufacturing?

Well, our market is regulated by a federal agency called the FDA, and when you contact them up to request information (called an FOIA request "Freedom of Information Act"), they don't just serve up the documents, charge you and be on your respective ways:

Your FOIA request is logged in a database and your FOIA request can be requested like any other FDA document.

This means:

Your business dealings as they pertain to the FDA are as public as your personal life is to the NSA.

• If you think Amgen wants to buy Onyx but you don't have access to insider information?  Send an FOIA to the FDA asking for all recent documents requested by Amgen.  If Amgen is doing due diligence on the deal, they may leave a trail there.
• If you thought Allergan was going to buy MAP Pharmaceuticals and wanted to test your hypothesis, send an FOIA to request the Allergan metadata.

If you're requesting actual documents, the data will be redacted; however, the log of the requests (the metadata) is public and available in sans redaction.

This is why our customer, FDAzilla, built the world's largest 483 store.  When you buy 483s from FDAzilla, you get the product without having to give up who you are and therefore business information you'd rather not have shared.

And if you're interested in more than just 483s, they have a compliance monitoring service that's built to suit your needs.

by Oliver Yu

Once you purchase through FDAzilla, it is true that they now have a record of your information; but the difference is that they are not compelled by law to share it to the public as the FDA is through the Freedom of Information Act.

On top of anonymously getting information, you also get it instantly...(which we all know from FDA FOIA experience, isn't necessarily on your timeline).

4 out of 5 Best-Selling Medicines for 2013 are Biologics

by Oliver Yu

According to the business/investing website The Motley Fool, the best selling drugs for 2013 are:

1. Humira (4.8B) - biologic
2. Advair
3. Enbrel (4.1B) - biologic
4. Lantus (3.5B) - biologic
5. Avastin (3.3B) - biologic

The combined sales of the top 5 drugs come in near 20 billion dollars.  With 15.6 billion (79%) from the sales of biologics.

Using 2012 as baseline, Humira, Enbrel and Lantus were on the list of top selling biologics.  But Remicade, Rituxan and Herceptin all placed higher than Avastin.

Which gets me thinking... did the fool.com author work off incomplete data?

• 5 of 8 Top-Selling Drugs for 2012 were biologics
• Top-Selling Drugs for 2011

What IS Peptone Anyway?

According to the free dictionary,

pep·tone

n.

Any of various water-soluble protein derivatives obtained by partial hydrolysis of a protein by an acid or enzyme during digestion and used in culture media in bacteriology.

Question: what is the source of protein?

Answer: Do you ever wonder what happens to the parts of the animal that humans DON'T eat or use?

Peptone vendors will get the animal scraps and make peptone by "dissolving" them with acid or digesting them with enzymes and eventually make them into a powder which gets sold to cell culture manufacturers who use them.

I know of bovine- and porcine-derived peptones... (aka "beef" or "pork").  With the mad cow scare from several years back, Process Development departments were moving away from bovine- to porcine.  And since then processes that use peptones have tried to move towards non-animal derived (aka "veggie") peptone.

As I've said before, peptone is that je ne sai quoi that the cells like and boosts their productivity.  A process development department that continues to use peptones do so at the risk of increasing manufacturing variability in favor of higher small-scale cell culture productivity.

And doing so risks making the process susceptible to peptone lot variability, which ultimately diminishes process robustness.

tl:dr - peptone is dissolved cow/pig/veggie parts ground into fine powder used by some biologics manufacturers to increase cell culture productivity.

OSI PI Historian Software is Not only for Compliance

by Oliver Yu

In October 1999, I joined a soon-to-be licensed biotech facility as Associate (Fermentation) Engineer. They had just got done solving some tough large-scale cell culture start-up problems and were on their way to getting FDA licensure (which happened in April 2000).

As the Manufacturing Sciences crew hadn't yet bulked up to support full-time commercial operations, there were 4 individuals from Process Sciences supporting the inoculum and production stages.

My job was to take over for these 4 individuals so they could resume their Process Science duties. And it's safe to say that taking over for 4 individuals would've not been possible were it not for the PI Historian.

The control system had an embedded PI system with diminished functionality: its primary goal in life was to serve trend data to HMIs. And because this was a GMP facility and because this embedded PI was an element of the validated system, the more access restrictions you could place on the embedded PI, the better it is for GMP and compliance.

Restricting access to process trends is good for GMP, but very bad for immediate-term process troubleshooting and long-term process understanding, thus Automation had created corporate PI: a full-functioned PI server on the corporate network that would handle data requests from the general cGMP citizen without impacting the control system.

Back in the early-2000's, this corporate PI system was not validated... and it didn't need to be as it was not used to make GMP forward-processing decisions.

If you think about it: PI is a historian. In addition to capturing real-time data, it primarily serves up historical data from the PI Archives. Making process decisions involves real-time, data, which was available from the validated embedded PI system viewed from the HMI.

Nonetheless, the powers that be moved towards a validating the corporate PI system, which appears to be the standard as of the late-2000's. 

Today, the success for PI system installations in the biotech/pharma sector is measured by how flawlessly the IQ and OQ documents were executed.   Little consideration is really given to the usability of the system in terms of solving process issues or Manufacturing Sciences efficiency until bioreactor sterility issues come knocking and executive heads start rolling over microbial contamination.

Most PI installations I run into try to solve the compliance problem, not a manufacturing problem, and I think this largely the case because automation engineers have been sucked into the CYA-focus rather than value-focus of this process information:

• Trends are created with "whatever" pen colors.  
• Tags are named the same as the instrumenttag that came from the control system.  
• Tag descriptors don't follow a nomenclature
• Data compression settings do not reflect reality...
• PI Batch/EventFrames is not deployed
• PI ModuleDB/ AF is minimally configured

The key to efficiencies that allow 1 Associate Engineer to take over the process monitoring and troubleshooting duties of 4 seasoned PD scientists/engineers lie precisely having a lot of freedom in using and improving the PI Historian.  

If said freedom is not palatable to the QA folks (despite the fact that hundreds of lots were compliantly released when manufacturing plants allowed the use of unvalidated PI data for non-GMP decision), the answer is to bring process troubleshooters and data scientists in on at the system specification phase of your automation implementation.

If your process troubleshooters don't know what to ask for upfront, there are seasoned consultants with years of experience that you can bring onto your team to help.

Let's be clear: I'm not downplaying the value of a validated PI system; I'm saying to get user input on system design upfront.

Examples of Cell Culture Productivity KPIs

Let's apply what we know of cell culture productivity KPIs.

Below is a control chart of a process that produces a stable, albeit variable titer:

The titer is a very simple data point to collect. QC measures it following their procedures and they spit out this one number every time a sample gets submitted.

Time required at the production culture stage to achieve this 2g/L is 10 days give or take a few hours.
The culture duration is also relatively easy to determine since we know the timestamp of inoculation and we know the harvest time. An arithmetic subtraction is all that is required to find this number.

Culture Volumetric Productivity

The culture volumetric productivity is computed by dividing titer by culture duration.
It stands to reason that control chart of culture volumetric productivity shows a stable, in-control KPI.

It turns out, there was a scheduled facility shutdown after Run 8. And starting with Run 9, there was a mis-specified parameter that determines the fermentor volume.
Our control chart shows special cause signals from Run 9 - 12 indicating that it took 4 batches before the QA Change Control system was able to push through the change.

Capacity-based Volumetric Productivity

By including bioreactor volume - which is determined by load cells or radar and known to the control system's process historian - we can compute capacity-based volumetric productivity:

If you look at it, the control chart for capacity-based VP doesn't look that different from the culture VP. Even with runs 9 - 12 at 83% capacity, there is still no obvious, control-limit-violating-special-cause signals.

The shutdown lasted seven days  and you can see that even though the bioreactors were cleaned and sterilized, they were left fallow for several days before the plant went back into production. The turn time spikes

Let's have a look at plant-based volumetric productivity.

Plant-based Volumetric Productivity

Again, we see here that runs 9-11 show a depressed plant-volumetric productivity, but still no obvious control chart violations. Plant-based volumetric productivity is a lot harder to compute since you're talking non-row data (in the SQL sense).

Typically, your manufacturing control system (MCS) is enumerating UnitProcedures and storing each UnitProcedure in their own row. To compute the turn time, you actually have to list out the previous several UnitProcedures and find the previous harvest and reliably getting this data is a pain in the butt.

Plant-based volumetric productivity violates Principle #2 of MSAT data:

The benefits derived from collecting the data needs to out-weight the costs.

In this case, for this operation where variability in other parameters are relatively high, all this extra work doesn't give you much that much benefit.

Conclusion

In the perfect world, data is easy to get and KPIs tell you a lot. In reality, it may tell you that you need to reduce your process variability before your KPIs are worth collecting.

Alls I'm saying is that you need not forge ahead and apply every KPI that you learn about. In some cases, getting the data may cost more than the data is worth.

Related articles:

• How Manufacturing Sciences Works
• Local MSAT campaign monitoring
• Manufacturing Sciences Consulting