For The Agency | For The Technology Developer | For The Magazine Publisher | For The Individual

Chapter 3

Accuracy and Precision

Accuracy and precision may be the most misused words in the test and measurement lexicon

An awful lot of engineers and scientists seem to have a very muddy idea of what the words accuracy and precision mean. The same is true of the folks who write specification sheets, catalog descriptions and other product literature describing data acquisition offerings. Both of these words have very precise meanings, which you should keep in mind in order to use them accurately. (Sorry about that last sentence. I just couldn’t resist it.)

A measuring instrument can be extremely precise, yet give you totally bogus readings. An instrument can be extremely accurate, yet give readings so imprecise that they are useless. Between you and me, I’d much rather have accurate though imprecise readings. At least then you know there’s a problem!

So, before I go into accuracy and precision specifications for data acquisition equipment, I want to clear the air about what those two words really mean.

Folks who know a thing or two about data acquisition boards will often tell you that a 12-bit ADC doesn’t necessarily give you 12-bit accuracy.

Yup.

Of course, that’s like saying that a red car won’t necessarily go 120 mph (or 192 kph, either). Being red and being fast are two different things, just as being precise and being accurate are two different things.

Precision

Let’s, for the moment, forget accuracy and concentrate on precision. Precision is a property of the measuring instrument and nothing else.

Right now, I’m looking at an amazingly ancient draftsman’s triangular scale. At least, that’s what I think these things were called back when draftsmen had any use for them. This particular scale is so old that it’s made of hardwood. It’s visibly warped. It was made back when the metric system was "somethin’ them furriners talk about." The only two uses I have for the thing now are to keep papers from blowing off my desk when I have the windows open, and as an object to help explain the concept of precision.

If you pick this thing up, the odds are two to one that you won’t see anything that immediately makes sense. Turning it over once or twice, however, will get you to a reasonably familiar scale: 12 inches divided into sixteenths of an inch. The same scale we saw on the "foot rulers" we learned to use in grade school. (My kids learned both metric and American Standard measurements. I don’t know what they’re teaching these days.) If you measure the length of something with this "foot ruler" scale, you’ll be able to report the results to the nearest sixteenth of an inch. The basic precision of this instrument is ± 1/32 inch.

The other scales on this triangular scale—the ones that don’t look like anything that makes immediate sense—divide distance in other ways that made it easy for the green-eye-shade draftsmen of yore to convert measurements in scale drawings. These have all sorts of different precisions. For example, there is one little bit there that has 3/16 of an inch divided up into 12 subdivisions. Thus, each small subdivision is 3/192 (or one sixty-fourth) of an inch long. That gives a precision of ± 1/128 inch!

By now you should have the idea that an instrument’s precision tells you the smallest difference that you can reliably read. For a digital instrument that uses a binary representation, it’s the least significant bit. For a decimal display, it’s the least significant digit. You can either define the precision by giving the size of that smallest difference or (what is actually more meaningful) plus or minus half of the smallest difference.

Accuracy

Accuracy, on the other hand, tells you how well measurements made using your instrument match up with some standard instrument. It’s a calibration issue.

I’m a private pilot and as a private pilot I use standard air navigational charts. For short trips, I use what are called Sectional Charts, which have a standard scale of 8 statute miles per inch. To measure how far it is from point A to point B (and therefore how long it will take and how much fuel I’ll use), I use the standard sectional-chart plotter. I also have a little pocket-sized plotter that I got as a promotional freebie.

The scales printed on the small plotter, however, are 2.5% shorter than they should be. I was a little unhappy about that until I realized that on a one-hour flight at 120 miles per hour, I’d be off by 3 miles, which is smaller than the Denver, Colorado airport. If you can’t see three miles when flying, you are figuratively, legally and actually "in the soup." That 2.5% error is not too important.

Anyway, assuming that the standard plotter is, as we used to say around the accelerator lab, "dead nuts" (meaning perfectly accurate), the little freebie plotter gives measurement results that are too high by one part in four hundred. It’s accuracy is +2.5%.

Accuracy, obviously, means the same thing whether the instrument is digital, analog or—what else is there?

Repeatability

Another term that goes along with accuracy and precision is repeatability. Whereas precision arises from the measuring instrument’s construction details, and accuracy arises from comparing the instrument’s results to those of a standard, repeatability expresses how well it’s measurements compare with each other.

If I measure something now and then measure it again later, how close can I expect the results to be? How well do the measurement results now compare to measuring the same thing tomorrow, a week from now, a month or a year from tomorrow, or just 30 seconds from now?

I ran across a perfect example while buying plants for the new cactus garden we’re putting in the front yard. The fellow running the nursery heard that I was into measurements and just had to tell me this story.

Many moons ago, he’d been a foreman in a sawmill. Being smarter than the average bear, he’d figured out that he could improve operations if he started processing each log by cutting it off to a standard length. The number floating around in my memory associated with this length is 27 feet.

So, he set up the system and explained what to do to everyone involved, then stepped back to watch it work.

It didn’t.

They cut a bunch of logs to standard length on the first day. The next morning, he went out to measure their lengths as a quality control check. They were all too long.

So, he chewed people out and sent them back through the saw to make them right. After lunch, he checked the logs recut that morning. Now, they were too short!

After banging his head on a tree for a while, he figured out what was wrong with his system: he was using a steel tape measure. The thermal expansion coefficient of the steel tape was much larger than that of the logs, and the logs retained heat better than the tape as well. Thus, the tape was expanding and contracting as the ambient temperature changed, but the logs weren’t.

He’d explained his system and gotten the work organized during the morning of the first day, so all the logs had been measured and cut during the heat of that afternoon. The next morning, when it was cool, he’d checked the work with that same steel tape measure. The tape, being relatively cooler, was relatively shorter. Thus, it erroneously told him that the logs were too long.

They recut the logs during the morning to match the measurements using the cool tape. When he went back to recheck in the afternoon with a warm tape, the measurements said the logs were too short.

The difference had never bothered anyone because they weren’t trying to precisely measure the log lengths. What they previously most cared about was the cross-sectional dimensions of the boards, which were just a few inches. The thermal variation had not been measureable with the equipment they used, so they didn’t know or care.

It only showed up when they started measuring long distances reasonably accurately, and then rechecking their measurements later. Then, they started seeing variations on the order of an inch from measurement to measurement.

Repeatability is only a problem if you want to repeat your measurements—and have them come out the same each time.

In the example, the measurement’s repeatability was limited by thermally induced variations. There are lots of other possible physical phenomena that can limit measurement repeatability as well. Mechanical flexure of the measuring apparatus is the second most common cause of distance-measurement repeatability problems.

For data acquisition systems, the repeatability is limited by a large number of mechanisms, from thermal effects to piezoelectric effects. A detailed discussion of these problems and what to do about them is well beyond the scope of this book. If you really want to pursue the subject, get a copy of Low Level Measurements Handbook, from Keithley Instruments by visiting their website at www. keithley.com.

Ideally, you would like an instrument’s repeatability to be better than its accuracy, and its accuracy to be better than its precision. That way, you can feel confident that the number you get truly represents the property of the object you’re measuring to within the obvious limit of the measurement. Whenever we make a measurement, we tacitly assume this situation exists by reporting all the digits measured.

Figure 3.1 shows how accuracy, precision and repeatability relate to a set of measurements.

Click image to see full size

Figure 3.1: Precision, accuracy and repeatability relate to the probability of getting various measurement outcomes.

You might look at the diagram and say: "Oh, the diagram shows that the instrument reports values at 0.1 unit increments. That must mean it’s a digital instrument."

You might say that, but you’d be wrong if you did. Remember that the instrument’s precision means that you can report values only to "the nearest such-and-such value." This measuring instrument can be read to the nearest 0.1 unit, just as an analog watch can (usually) only be read to the nearest minute. The instrument shown in the figure could be analog or digital with no affect on the diagram.

The next thing to realize is that this is a really stinko instrument. Its repeatability isn’t much better than its accuracy, which is much worse than its precision. This thing needs a digit knocked off its readings and a quick trip to the nearest cal lab. Of course, if it weren’t such a piece of junk, we wouldn’t have much to talk about, would we?

In laying out Figure 1, I chose to denote the repeatability by reporting the half-width of the probability distribution. That is, I’ve indicated the repeatability by showing the distance between the points where the probability curve crosses 50%. I could have used the RMS deviation, the points between which 95% of the measurements fall, or any of several other ways of reporting repeatability. Any of those definitions is valid and all of them are used by various people. The important thing is that when someone quotes a repeatability for an instrument, you have to ask how they’re defining it.

Accuracy is the result of convolving a whole lot of different effects. As such, it doesn’t really scale with anything reliably. It’s just a mish-mash! This instrument is clearly disgustingly inaccurate. The average of measurements is 18.6% higher than the true value.

Accuracy specifications, however, can be stated in almost any way the manufacturer feels confident in stating them. An accuracy specification is a promise by the manufacturer that under specific circumstances, which include being within a range of environmental conditions, proper calibration and nobody mucking about in the unit’s guts, what the unit reads when compared to a NIST (National Institute of Standards and Technology)-traceable transfer standard will differ from the standard value by no more than the specified amount.

A promise is a promise, but you have to look carefully at what really is being promised. What one manufacturer promises may be quite different from what another manufacturer promises for a similar product. Read the fine print.

Accuracy and Precision in Data Acquisition

Accuracy, precision and repeatability figure into data-acquisition-equipment specifications in a particular way because of the digital-sampling nature of data acquisition boards. Figure 3.2 illustrates the processing of a voltage-input signal that increases linearly with time.

Click image to see full size

Figure 3.2: An ideal analog-to-digital convertor produces a stair-step readout when presented with a steadily rising voltage input.

You can tell by the scales that the data acquisition card was set up so that input voltages ranging from zero to 100 V read out in engineering units from 0 to 100 units. Being analog, the input voltage takes on a continuum of values. Being digital, the output takes on only specific discrete values. In this case, I’ve arranged ten discrete values from 0 to 90. That is, this setup has a resolution of one decimal digit. For a data acquisition card, the well-defined term resolution serves as proxy for the generally poorly understood term precision. In either case, we are talking about the smallest difference the data acquisition card can reliably report. In this case, that’s one tenth of full scale or 10 units.

Figure 3.3 makes this point a little more clearly by comparing low resolution and "high" resolution digitization. (By the way, the correct term is "digitization." The word "digitalization," which I sometimes see used is a medical term that refers to treatment with the drug digitalis, which is something even the most mercenary of health-care professionals will not do to analog measurements.)

Click image to see full size

Figure 3.3: The analog-to-digital convertor’s resolution determines the measurement precision.

The low-resolution version has only five levels, which makes it, by anybody’s lights, truly coarse, crude and low resolution. The "high" resolution version is high only by comparison with the low-resolution version. It has 20 digital levels. In an era when 12-bit DAQ boards (which give 4,096 levels) are readily available, a 20-level digitization (comparable to something absurd like 4.325 bits) is really poor. It is, however, about the best I can illustrate and still have the graph make its point.

So, measurement precision translates pretty clearly to ADC resolution. What about accuracy?

Accuracy collects the effects of a whole bunch of error sources. Figure 3.4 shows three obvious ones and how they relate to DAQ-card operation. These errors have their counterparts in all measurement situations.

Click image to see full size

Figure 3.4: Accuracy specifications encompass a number of error sources.

For example, the freebie sectional plotter I described earlier suffers from a good case of gain error. That is, the measured value is equal to the true value times a constant other than one. In the case of the plotter, the constant was 1.025. In the case of the Gain Error graph, the constant is 0.80. (I know, because that’s what I put in the simulation to get the chart.)

Linearity error means that the transfer function between the input and the output is a power law with quadratic or higher terms. In other words, it ain’t straight. A step from one ADC output level to the next means something different at one end of the scale than it does at the other.

Gain error and linearity error are fairly consistent errors that can be backed out of the data during post analysis if you have some way of knowing the transfer function. In fact, the entire field of calibration boils down to determining the gain and linearity errors, then compensating for them to bring the measurement system within its specifications. When you read in some specifications that an instrument’s accuracy is plus-or-minus such-and-such, it means that each instrument of that model has been calibrated to produce readings within that range.

Your particular unit is going to have a unique transfer characteristic within the range. Transfer characteristics will vary from unit to unit, but will be stable for any given unit. In other words, when you use your unit to measure 3.257 Volts, it will be off by some amount. There are two things you can be sure of: the amount it is off will be the same every time you use it to measure 3.257 Volts and that amount will be within the stated accuracy for that model.

What I said in the last paragraph is true, provided the unit has been calibrated per specification and is operating within its environmental (temperature and humidity) specifications. Calibration intervals arise from the fact that each unit’s transfer characteristic changes slowly over time in a predictable way. The cal lab knows how fast the unit will drift and schedules a recalibration at intervals that let them catch the unit before its readings drift out of tolerance. If you use an out-of-calibration instrument of any kind, you simply don’t know just how bogus the readings are.

The third error source illustrated in Figure 3.4 is noise. Unlike gain and linearity errors, noise is not repeatable. Therefore noise is more of a repeatability issue than an accuracy issue.

I created the noisy graph by adding in plus or minus 0-10 units from a random number generator before digitizing the simulated signal. Every time I ran the simulation, I got an entirely different chart. It works the same way in real life.

This kind of noise can get into acquired data in two ways: your DAQ-board circuitry can inject it, or it can float in from outside. Noise from data acquisition boards generally arises from thermal noise in the front ends of instrumentation amplifiers. Generally, you can figure that electromagnetic junk getting into your system through the sensor or through the hookup leads will swamp what your DAQ board puts in. Obviously, the DAQ-board manufacturer has no control over what garbage you let in through the front end, so it has nothing to do with quoted catalog specifications.

What you look for is a board whose noise specifications are considerably better than what you can hope to achieve in your system’s front end. The board’s noise gives a lower limit beyond which you cannot expect to go. The manufacturer cannot give you any guarantee of ever achieving it in your real system.

Precision (in the guise of digitizer resolution), accuracy (as specified by gain error and linearity error) and repeatability (in the form of instrumental noise level) apply to all instruments, not just data acquisition products. They also appear in some similar form whether the instrument is digital or analog.

Chapter 1 | Chapter 2 | Chapter 3 | Chapter 4