The whole is not always the same as its parts

You are going to buy a new home.  The house is 2000 square feet on a 3/4 acre lot.  You hire Rich (the termite inspector) to check it out before you buy. After all, no one wants to buy a house with termites. 

  • Good news!  The house passed.  No termites.  Thus, you buy the house.
  • Bad news!  A month after the sale closes you discover - termites.  

What?  How could this happen?

 
You go back and look a little deeper in the method of inspection Rich relied upon.  You find out his methodology was to only check "one square inch" of the floor in the house.  When he did not find anything wrong within the "one inch" he assumed everything else was also termite free.
 

How do you feel now?
 
A part of something does not always represent the whole. Determining how many termites are in "one square inch" of a house does not really answer the question whether you have a termite problem.
 
The termite inspector committed what logicians call the all things are equal fallacy.  This occurs when when it is assumed, without justification, that conditions have remained the same at different times and places.
 
The same danger is present when attempting a forensic measurement.  For example, in a typical DUI case where a blood sample is taken, the lab will test less than a M&M size sample of blood.  However, in Arizona the legal definition of an alcohol concentration is grams per 100 micro-liters. Translation, the legal definition of an alcohol concentration requires multiplying the results of the "one inch" by about 1000 (assuming the M&M is about 100 micro-liters).
 
The danger is assuming the rest of 1000 micro-liters (or 100 milliliters) has a proportional amount of ethanol in it.  Small errors multiplied by 1000 can easily mislead you to believe that a person's alcohol concentration is above a legal limit when it is not.
 
Like the termite inspection, it is up to the crime laboratory to prove their justification for assuming using such a tiny amount below the legal definition of an alcohol concentration answers the question - is the person above the legal limit?  After all, no one wants termites...or people being wrongfully convicted.
 
 

Measuring and Counting

 MEASURING

Measuring is the assignment of a number, and all the uncertainties of that of that number, to something.  The purpose of assigning a number is to give meaning to the object measured.

  • Uncertainty: A bag placed upon a scale shows its weight to be 41 pounds.  If the bag must be less than 50 pounds, then the number produced by the scale indicates it meets this requirement.  However, you must know how far from its true value might the 41 pound number be off by?  Uncertainty is the amount of doubt (e.g. the amount of possible variation) you should expect that number might be off.
  • Fit for Purpose: Assume there are two scales.  The same bag weighing 41 pounds is place on both scales.  However, it was determined that Scale A produces numbers that can be off by as much as 30 pounds.  It was also determined that the number produced by Scale B merely off by as much as 3 pounds.  Knowing the amount of uncertainty contained in the number helps distinguish counting from measuring.  Knowing the uncertainty allows you determine if the measurement is fit for the purpose of determine if the object exceeds 50 pounds.

Measuring relies upon estimation.  The choice of data, the methodologies employed, and level of quality measures used tells you how confident you can be in the estimation.  Once you have a reliable estimation of how close a number may be (or not be) to the true value, you can make informed decision as to what purposes the number can be used - and not used.  

 

COUNTING

Counting is not the same as measuring.  However, the two are often confused.  Counting is usually a technique within a measuring process (methodology).  Counting can result in an exact number.  However, measurement will never claim to represent a true value. Measurements are merely estimations.

Counting an exact amount of something is often not possible or practical.  The thing you are intending to measure (the measurand), the matrix it is found in, or the level of accuracy required may make counting impossible.   Thus a system is needed to provide a reliable estimation which you can rely upon.  

Some things to take into account when making an estimation:

  • Distinguishing: Some molecules are so similar to others that it is often impossible continuously distinguish them from each other.  Thus, they cannot be easily counted.
  • Location: Some substances are contained in places we cannot practically enter to count them.  The best way to know how much alcohol is affecting a person's brain at a particular time would be to take a sample of brain tissue.  However, society has not yet determined such a procedure falls outside the protections of a person's 4th Amendment rights.
  • Gas Chromatographs: The results of a gas chromatograph are often used to determine whether a person's alcohol concentration is above a legal limit in DUI cases.  However, the machine does not measure a person's blood alcohol concentration.  If properly used, the machine merely counts the number ethanol molecules in a gas portion of a headspace vial.  Thus, it indirectly counts a microscopic amount ethanol from a tiny sample.  

A measurement based upon a machine's indirect count of a substance results from combining it with algorithms, numerous assumptions, and historical data regarding the past performance of the machine (and software) used in the process.  This is known as an uncertainty calculation.

In this manner, measuring requires much more than counting.  Measuring requires more than merely assigning a number to an object.  More importantly, one can assign a number to an object but not create a measurement.  When this occurs it is not a measurement.  It is a misrepresentation.

 

Counting is what you do to get a number.  Measuring is what you do if you want to know the truth about the number.

You have a bandwidth problem

An analyst from a crime lab testifies that a defendant, who is charged with DUI, has a blood alcohol concentration of .120.  Despite the legal requirements that the state must prove the test is trustworthy, most jurors have made a blink judgement the that test is correct.  As is often the case, the appearance of science is a powerful tool of persuasion.  This is true  even when the opinion is based upon junk science.
 
Here, despite the claims of the analyst and unbeknownst to the jury, the test result was done using unreliable equipment relying on defective software.  Your challenge: undo the jury's initial judgments, demonstrate the analyst is too biased and lacking the qualifications to understand the severity of the equipment's defects, and show the result can't be trusted.  This is no small task.
 
This task will take time.  It requires a thorough understanding of the many underlying scientific disciplines involved.  Adequately educating the jury will require information from several different sources.  Each piece of evidence will present a different evidentiary challenge.  In short, beyond the inherent difficulties of such cases, you also have a bandwidth problem.
 
Bandwidth is the amount of data that can be transmitted in a fixed amount of time.  DUI trials have time and evidentiary limitations.  There are not intended to be semester long science classes.  There are practical realities inhibiting you from properly educating a jury with the knowledge they need to debunk these unsound claims.  If left unaddressed, a court may not even recognize this bandwidth dilemma.
 
Consider the problem in the following terms.  A presentation that does not reach the audience persuades no one.  If Netflix creates next years best new drama, but there is not enough bandwidth to stream it, then what was the point of creating it.  No one pays a subscription fee to see a "buffering" message.  Quality is meaningless without bandwidth.
 
Being right is does not convince a jury without an adequate opportunity to present it to a jury.   In these cases, you don't have a right or wrong problem - you have a bandwidth problem.  Accordingly, neglecting the bandwidth argument can be fatal.  If you don't sufficiently address this issue, then no one may hear how right you are.

What warrants a warrant?

After the Supreme Court decided the case of Missouri v. McNeely, the question of when a warrant is required, before law enforcement may draw a person's blood became more interesting to say the least.  On one side of the issue was the position that a blood alcohol concentrations is constantly changing, thus, there is a justification for law enforcement to bypass the traditional warrant requirement.

The contrary, and as it turns out the prevailing position, is that our Constitution does not allow law enforcement unfettered discretion to decided if they can stick a needle in your arm without a warrant (i.e. probable cause presented to a judge who issues a warrant).  The reality of modern technology is that a telephonic warrant can be obtained in about 15 minutes for most cases.  Accordingly, the exigent circumstances reasoning for bypassing the warrant requirement is unsound.  As the U.S. Supreme Court stated in their rejection of such a per se rule in DUI cases:

But it does not follow that we should depart from careful case-by-case assessment of exigency and adopt the categorical rule proposed by the State and its amici.  In those drunk-driving investigations where police officers can reasonably obtain a warrant before a blood sample can be drawn without significantly undermining the efficacy of the search, the Fourth Amendment mandates that they do so. See McDonald v. United States, 335 U.S. 451, 456, 69 S.Ct. 191, 93 L.Ed 153 (1948).

Missouri v. McNeely, 133 S.Ct. at 1555 (2013).

Is it really so surprising that what warrants a warrant is what is reasonable under the circumstances?

The anatomy of a gas chromatograph

 

The results produced by a gas chromatograph are usually the difference between innocence and guilt in a DUI case.  The prosecution’s purported blood alcohol concentration (BAC) is typically the “end-all be-all” of their case. Let’s take a look at how this machine creates such a critical measurement.

Big Pictures Thoughts

  • If done properly, gas chromatography is a reliable way to measure the amount of blood in an alcohol sample.  However, automobiles are also reliable, but there are still thousands of car wrecks every day.  There is no presumption of reliability simply because a gas chromatograph was used.
  • The measurement process has both human components and machine components.  All steps in the process must be done correctly for the measurement to be trusted.
  • The goal of is to produce a measurement, which is both accurate and reliable.

General Principles

  • Gas chromatography is an indirect measurement.  The machine does not test liquid portion of a blood sample.  In headspace gas chromatography, the machine converts substances to a gas, and then it must separate the different types of molecules in the sample.  After separation, a microscopic amount of the gas is measured by software.
  • The machine must demonstrate it is able to separate different types of molecules before it can measure them.  If it cannot properly separate different categories of molecules, then its measurements may be artificially higher.
  • Gas chromatography is done in manner like a production line.  Multiple samples (usually over 100 vials) are being processed in a “batch.”  It is essential to safeguard against the wrong information being assigned to the wrong sample.

Vocabulary

  • Gas chromatograph - a machine that separates molecules, and then measures, the amount of the various components in a sample.
  • Gas Chromatography - the scientific process performed by a gas chromatograph.
  • Chromatogram – the graphical representation of the data produced by the gas chromatograph.  This is where you will find the final measurement.  A chromatogram is the machine’s conclusion.

The Human Part

The measurement process starts long before the gas chromatograph is actually turned on.  The blood must be collected, identified, stored and transferred properly before the sample is put into the gas chromatograph.  Even the best machines cannot account for, or identify, that a sample has been corrupted.  The principle of garbage in garbage out must be kept in mind.  That is, incorrect (or poor quality) input will always produce wrong output.  

Human are also responsible for teaching the machine a specific alcohol concentration.  The machine does not come out of the box knowing any specific alcohol concentration.  Typically a lab will purchase approximately four (4) different alcohol concentrations from a vendor.  For example, .01, .10, .20, .40 are often used to calibrate the machine. 

These samples are put into the machine and the analyst programs the machine’s software to use these values.  If the analyst tells the machine a sample is a .40 but it is really a .30, the machine cannot tell the difference.  Ensuring a calibrator is what it purports to be is known as traceability.

The Machine Part

The machine starts its analysis after a small portion (less than the size of the a single M&M) of each blood sample is put into a headspace vial. The headspace vials (usually over 100) containing the samples are loaded into a part of the machine called the autosampler.  

The samples are then heated (in a headspace tube) forcing molecules in the liquid portion of the sample to rise.  After the molecules are vaporized, a needle punctures the top of the headspace vial and extracts a microscopic portion of the gas above the liquid.

These vaporized molecules are pushed through long thin columns by a carrier gas (hydrogen or helium).  These thin columns have a chemical coating inside them designed to interact with the molecules passing through them.  The carrier gas moves at a constant pressure.  This results in different molecules in the gas to group together (e.g. ethanol with ethanol, methanol with methanol).  Each molecule group, such as ethanol, has a unique rate of speed.  This accounts for the separation of the each substance in the columns.

After each molecule group is pushed out of the column, they will be pushed to a detector.  The time when is substance exits the column is called the time it elutes.  The detector’s software has been programmed to identify different substances by the time they elute from the column.  The Flame Ionization Detector, as the name implies, then burns each molecule group and then measures how much is burned.

The software gathers the “raw data” and then processes it.  The “process data” is graphically represented in something called a chromatogram.    The measurement is found here.

The above summary just scratches the surface of the measurement process using a gas chromatograph.  If you are going to rely upon the measurement produced by this technique, then every step in process (both the human and machine) must be shown to have been done correctly.

Read the "Warning Label" of your blood test

When should you trust a blood test result that claims to measures an alcohol concentration?  Start by reading the test's "Warning Label."  Here is an explanation on my legalcoffee blog.

A reported result versus a complete result

 

In DUI cases, a machine called a gas chromatograph is often used to measure an alcohol concentration in a blood sample.   The measurement, which the machine prints at the end of the process, is called a reported result.  We are finally at the point in Arizona, where courts are starting to recognize that merely providing a reported result is not sufficient evidence.  The law is coming to the same realization that science did many years ago: a reported result from a machine is an incomplete measurement.

A complete measurement includes more than just a reported result.  As a matter of fact, simply providing a reported result is often misleading.  A reported result is only complete when accompanied by a “statement of its uncertainty.” See NIST Technical Note 1297, 1994 Edition.  No measurement is perfect.  The result of any measurement is only an estimation of its value.  A “statement of uncertainty” is the range of doubt that exists regarding a measurement.

A complete test result, must also include:

  • a “Range of Uncertainty” and;
  • “Confidence Interval.” 

To illustrate, let’s assume that a blood test result was .100.  Let’s also assume, based on a review of the machine’s prior performance, a “range of uncertainty” was determined to be ± 5%, with a “confidence interval” of 100%.  This means, the reported result could be as low as a .095 and as high as a .105.  Moreover, this also means, if the same blood sample were repeatedly tested on this equipment, the result would only be outside of the ± 5% range 1 out of a million times.  If this statistic were true, this would certainty be a reported result that you could trust.

On the other hand, what if for the same reported result of .100 the range was ± 30%, with a confidence interval of 50%?  Here, this means the reported result could be as low as .070 or as high as .130.   Furthermore, if you continued to test this sample on the same equipment, 300,000 times of out of a million, the reported result would be outside the range stated above.

When comparing the two complete test results, you can see that providing a mere reported result does not tell us the whole story.    Merely telling us the reported result can actually tell us a very misleading story.  Science will not accept incomplete measurements.  Why should the law?

 

A Person Accused Of DUI Deserves A True Second Opinion

Arizona DUI cases almost always involve a chemical test. There is a movement in Arizona towards the exclusive use of blood testing. It is well settled that blood testing is more accurate than breath testing. But how accurate is blood testing? Can it truly determine a person's blood alcohol concentration? I believe that these are reasonable questions to be asked by a person accused of DUI and facing 30 to 45 days in jail.

Under our system of justice we should demand better answers from the government than "because we said so" or "because our lab has the highest standards." Simple conclusions are not good enough in science and they should not be good enough in justice.

So who checks the government's test results in Arizona? The simple answer is the government. They merely claim to check themselves. I have yet to see one Arizona crime lab that conducts blind testing by an independent party.

Here is how the actual process works. In Arizona DUI cases, The government expert will tell juries that they do double check their work. That is, they have a quality assurance program to make sure the blood alcohol test results are accurate. But the government's oversight of their work is not what you my think. Instead retesting every sample, or randomly retesting a portion of the samples, the lab merely does a technical review.

A "technical review" is not retesting. The Government toxicologist usually puts between 30 to 40 blood samples into a blood tester. Then they turn the blood tester on and leave. The blood tester commonly runs overnight and the printed results are reviewed by the government lab person.

 

This review of the printed documents is what is known as a technical review. The actual documents are called chromatograms. They are simply pieces of paper with graphs and data on them (retention times, area counts, etc...) If the run has the usual 30 to 40 samples there could be well over 100 pages of data.

In court, the government will tell the jury that they "double check" all the tests. What they actually do is take those 80 to 100 pieces of paper and give it to another employee of the same lab.  That person then reviews the data. As long as the data appears to be consistent, they conclude it is an accurate test. In Court, they tell the jury they got a second opinon from another analyst to ensure the accuaracy of the tests.  

A nationally recognized expert, Dr. A.W. Jones, has opined that in forensic testing of blood samples retesting should be done, not a mere a "technical review." It is his opinion, that a mere technical review is inadequate to ensure the accuracy of the test results.

This standard makes perfect sense in DUI cases because the level of punishment is often dependent upon the test result. If a person's blood alcohol concentration result is .165 the person is facing a minimum of 30 days in jail. If he was under a .150 then he is merely facing 1 day in jail. Is it too much to ask the government to check their work when 29 days of jail is at stake? I guess the answer depends on how important the result is to you.

If you were at the doctor and she said "the blood tests were back," "we need to operate," let's "prep for the mastectomy." Most people would seek a second opinion. Why...because the issue is too important to merely rely on the first test. When it matter to you - you retest.

P.S. I will address the Defendant's ability to do their own retesting in my next entry.

 

Everyone's Blood is Not the Same

Law enforcement's primary method for determining if a person is driving under the influence of alcohol is a chemical test.  That is, a police officer will take a sample of a susect's blood  or breath.  The chemical test assumes that the composition of everyone's blood is the same.  Specifically, the test assumes that all people have the same hematocrit level.  However, this assumption is incorrect.

The hematocrit level, or packed cell volume, is a measure of the proportion of blood volume that is composed by solids.   Whole blood is composed of solid particles in liquid.  the solid portion of whole blood contains: (1) white blood cells; (2) red blood cells; and (3) platelets.  The liquid portion of the blood is known as plasma. 

In this manner, if a man has a hematocrit level of  .51, then his whole blood consists of 51% solids and 49% liquids (plasma).  This solid to liquid ratio will effect the outcome of a blood alcohol concentration test.  The reason is the liquid portion of the whole blood, the plasma, contains water.   Alcohol is more susceptible of being dissolved in water than is the solid portion.  Consequently, the liquid portion of the whole blood will have a higher concentration of alcohol than the solid portion.

Stated another way, the higher the hematocrit level (thus the less liquid) in the blood, the greater the concentration of alcohol in the liquid portion of the blood.  Ultimately this means several people with the same amount of alcohol in their body, but different hematocrit levels, will have different test results. 

Men and women have different average hematocrit levels.  A normal hematocrit for a man is 45 (plus or minus 7%). Women have a normal hematocrit level of 42 (plus or minus 5%).  There are numerous other factors that can effect a hematocrit level.  However, a person's hematocrit is not proportional to body size. 

Hematocrit ranges primarily effect breath alcohol testing. This is because in blood testing the blood is mixed with an internal standard (such as N-Propranol which has similar structure to Ethanol, but has a different number of carbons).  In sum, varying hematocrit levels expand the range of accuracy in blood alcohol concentration testing.  This expanded range of accuracy can be a valuable defense for a person accused of driving under the influence alcohol (DUI).

Scientific Defenses That May Be Available In Arizona DUI Cases

Challenging the reliability of a chemical test result is usually essential to effectively defending a DUI case.  In order to accomplish this task a thorough understanding is needed of the types of chemical testing used in Arizona and the scientific principals behind those tests. Here are the three types of chemical tests used by Arizona law enforcement:

  1. Breath Testing
  2. Blood Testing
  3. Urine Testing

Blood and breath testing are by far the most common.  Moreover, there is a trend in Arizona law enforcement moving towards blood as the primary testing method.  However, some police agencies such as the Department of Public Safety appear to be staying primarily with breath testing.  Urine testing is utilized mostly in cases where it is suspected that a person is under the influence of drugs.  Urine testing is seldom used in our state for blood alcohol testing.  Below is a summary the potential defenses for cases involving both breath and blood alcohol testing. 

BREATH TESTING DEFENSES

In order to know what defenses may be available to an attorney, an understanding of the breath alcohol testing process is required.  Breath testing is based on the scientific principal of Henry's law (also referred to as Henry's coefficient).  Henry's law was conceived by the English scientist William Henry. The principal provides in a closed container over time some of the molecules in a liquid will travel into the gas above the liquid. The amount of molecules that travel into the gas will be directly proportional to the number of molecules in the liquid.  Henry's law assumes a constant temperature and a closed system.  Consequently, all the factors that may influence Henry's law may also effect the results of a breath test.

The instrument used to apply Henry's law to breath testing is an Intoxilyzer.  In Arizona, most agencies now use the Intoxilyzer 8000.  The instrument is manufactured by a company named CMI.  The instrument is supposed to take a sample of the subject's lung air and use the principals of Henry's law to estimate a person's blood alcohol concentration.

Her are some of the challenges to evidentiary breath testing:

  • Range of Accuracy
  • Temperature Changes
  • Lack of a Deprivation Period
  • Radio Frequency Interference
  • Failure Keep Calibration Records
  • Partition Ratios
  • Lack of Warranty
  • Forced Agreement of Tests
  • Breathing Patterns
  • Mouth Alcohol
  • Hermatocrit Levels
  • Calibration Errors
  • Source Code Disclosure

 BLOOD TESTING DEFENSES

Similar to breath testing, blood testing also relies on the principals of Henry's law.  However, a different instrument is used to test the blood sample.  Most forensic laboratories use a headspace gaschromatograph.  Headspace refers to the space in a vial above the sample where the gas portion is located. Headspace analysis is the analysis of what is present in that gas. In its simplest terms, gas chromatography attempts to separate and identify what is in that head space gas. 

Some of the attacks that can be made on this blood testing process are:

  • Margin of Accuracy
  • Improper Tube Inversion
  • Chain of Custody
  • Contamination of Sample
  • Proper Site Cleaning
  • Lab Testing Errors
  • Expired Materials
  • Serum Samples
  • Failure to Follow Manufacturer's Instructions

While blood testing can be an accurate and precise measure of a person's blood alcohol concentration, it is not perfect.  Even under ideal conditions, there will still be a range of accuracy regarding test results.  However, conditions are not always ideal.  When basic scientific protocols are not followed the reliability of the test comes into question.