The short version: when two sets of data disagree, check three things before you suspect anyone
The same sample, two laboratories, two sets of numbers that do not agree. This happens daily in our industry, and the overwhelming majority of the time nobody did anything wrong. When you are holding data that disagrees, work through three things in order: whether the measuring equipment is in a valid calibration state and whether the calibration points cover the range actually used; whether the method has been validated and executed to the same part; and whether the report states the measurement uncertainty and the decision rule. Only after those three are aligned does the question of who is right become meaningful.
Under ISO/IEC 17025, for a number to qualify as a valid measurement result it needs two properties at once: the quantity value can be traced to a recognised metrological reference, and the dispersion of the result can be described quantitatively. The first is traceability, the second is uncertainty. Without the first, the data is just a reading. Without the second, deciding pass or fail is nothing more than throwing a number at a limit.
What a traceability chain is: following the result upwards
The shape of a traceability chain is unglamorous. The test result depends on measuring equipment, the equipment depends on calibration, the calibration depends on a higher-order measurement standard, and so on upwards until the realisation of the International System of Units. Break any link in the chain and everything below it loses evidential weight.
| Route to traceability | When it applies | Form of the evidence | Common break points |
|---|---|---|---|
| Calibration by an external metrology body | General-purpose gauges and routine measuring instruments | Calibration certificate carrying a traceability statement and the calibration uncertainty | Expired certificate; calibration points not covering the range actually used |
| Certified reference materials | Chemical and materials composition work, and some biologically related measurements | Reference material certificate with the assigned value and its uncertainty | Reference material past its validity, or stored improperly after opening |
| In-house calibration | Where no suitable external service is available | In-house calibration procedure plus traceability evidence for the higher-order standard | Procedure never validated; the person performing it not authorised |
| Comparison and proficiency testing | Quantities that are hard to trace directly, and composite tests | Comparison report and bias analysis | Only one round ever run, no trend data |
The trap that shows up most often in this table is calibration point coverage. The equipment genuinely was calibrated, the certificate genuinely is in date, but the calibration points sit in one band while the values you actually measure fall outside them. In evidential terms that is extrapolated use. The data is not necessarily unusable, but the report ought to say so, and the receiving party may well ask. The check itself is simple: put the range you actually use next to the calibration points on the certificate. If your range falls outside the coverage, add calibration points, or switch at the ordering stage to equipment that does cover it.
The certified reference material row is worth a second look too. Many people watch only the expiry date and overlook the post-opening storage requirements and the remaining quantity. If storage conditions after opening are wrong, the assigned value on the certificate no longer describes what is in the bottle, and that break point is completely invisible on the report. The only way to see it is through the laboratory's internal reference material management records.
Uncertainty is not error, and it is not the laboratory's report card
This concept gets misread more than most. Uncertainty does not state "how far the laboratory got it wrong". It describes how wide an interval around the result the true value may plausibly sit in, given the method, equipment, environment and sample conditions in play. A small uncertainty does not mean the result is necessarily closer to the true value, and a large uncertainty does not mean the laboratory is weak. To a large extent it is set by the method itself and by the sample itself.
| Source of uncertainty | How it shows up | Can it be reduced |
|---|---|---|
| Measuring equipment | Indication error, resolution, repeatability | Yes, by moving to a higher-grade instrument or adding calibration point density |
| Environmental conditions | Instrument drift and sample condition changes driven by temperature and humidity swings | Partly, through environmental control and conditioning procedures |
| Sample non-homogeneity | Unit-to-unit variation within a batch, material anisotropy, assembly variation in structural parts | Hard to reduce; it can only be characterised by increasing the number of specimens |
| Operation and set-up | Clamping arrangement, positioning offset, load alignment, connection condition | Yes, by tightening procedures and locking them in through operator training |
| The method itself | The inherent repeatability and reproducibility level of the method | Essentially no; it is a property of the method |
A practical conclusion falls out of this table. If the disagreement comes mainly from sample non-homogeneity or from the inherent dispersion of the method, then moving to yet another laboratory and testing again will most likely still not agree. What should actually be done is to increase the number of specimens, pin down the conditioning conditions, or nail the method details down at the ordering stage, rather than cycling through laboratories hoping for a better answer.
Decision rules: the frequent root cause of clashing conclusions
When a result sits near a limit and two laboratories reach different conclusions, the cause is often not a difference in the data but a difference in the decision rule.
| Relationship between result and limit | Usual way of deciding | What you need to confirm |
|---|---|---|
| Result clearly far from the limit | Decide directly on the result | Nothing special required |
| Result close to the limit but on the conforming side | Simple acceptance, or a guard band is applied | Whether the report states the decision rule that was used |
| The uncertainty interval of the result straddles the limit | The rule has to be explicit, otherwise conclusions can come out opposite | Agree the rule at the ordering stage, not after the report lands |
| Repeat measurements fall on both sides of the limit | Follow the result handling specified by the method | Whether the method specifies how the value is taken, and whether the laboratory followed it |
The purpose of a guard band is to control the risk of a wrong decision. With a guard band applied, a result has to sit clear of the limit by a margin driven by the uncertainty before it is judged conforming, which makes the decision stricter. With simple acceptance, a result on the conforming side of the limit is judged conforming, which makes the decision looser. Both practices are legitimate. The difference lies in who carries the risk. That is why the rule has to be stated in the report, and why a report that does not state it gives the receiving party a fair reason to ask for clarification.
The order to work through when two sets of data disagree
This is a sequence you can follow as written. Worked in this order, the great majority of disagreements are explained within the first three steps.
Step one, compare the samples. Same batch, same condition, same conditioning. Material parts from different batches differ from each other to begin with, so using data from two batches to challenge each other is wrong from the starting line.
Step two, compare the methods. Same part of the standard, same form of test rig, same way of applying the load or the applied condition, same conditioning. The same parameter name measured on different rig configurations is not comparable, and this shows up especially clearly in mechanical and durability work.
Step three, compare equipment and calibration. Grade of the equipment on both sides, calibration status, and whether the calibration point coverage holds up across the range actually used.
Step four, compare the uncertainties. Put both results together with their respective uncertainty intervals. If the intervals overlap, the two results are consistent within the capability of the method and there is no contradiction to resolve. This step defuses a substantial share of apparent data clashes, but only if both sides reported uncertainty in the first place.
Step five, compare the decision rules. Disagreements at the level of the conclusion very often find their answer here.
Only when all five steps have been worked and the data still does not agree does it become time to suspect execution or record-keeping, and at that point the investigation moves to raw records, equipment logs and personnel qualification records.
Three things you can lock down at the ordering stage
Nearly every argument about data credibility can be avoided with three sentences said at the ordering stage.
First, state whether the report needs to give the measurement uncertainty. Some uses do not require it, some receiving parties explicitly do, and adding it afterwards means reissuing the report.
Second, state which decision rule applies. Especially where the design value sits close to the limit by nature, leaving the rule undiscussed amounts to handing the conclusion over to luck.
Third, state whether calibration traceability information for the key equipment is needed. Some clients' internal compliance processes ask for that material, and one sentence up front means the laboratory issues it together with the report. Asking afterwards turns it into a separate documentation exercise.
All three cost one sentence at the ordering stage. Filling them in after the report has been issued means reissuing the report, and in the more serious case retesting. The information to prepare for an engagement can be checked against the testing requirements, and the recurring questions are collected under frequently asked questions.
On the relationship between accreditation and data credibility
One clarification is needed here. An accreditation mark only demonstrates that the laboratory holds the corresponding technical competence within its accredited scope; it does not constitute a commitment regarding the market access outcome in the target market. It means that this laboratory's traceability management, method validation and uncertainty evaluation for that item have been assessed by a third party, which is supporting evidence for the credibility of the data. It does not guarantee on your behalf that the product meets the access requirements of any market. How to check a scope, and how to read the mark on a report, is a separate topic and can be picked up on its own in the knowledge section.
What you can check yourself once the report arrives
- Whether the report names the principal equipment used and its calibration status
- Whether the range actually measured falls inside the calibration point coverage, and whether anything outside it is explained
- Whether the measurement uncertainty is given, and whether the way it is expressed is clear
- Whether the decision rule is stated, and whether it was actually applied where the result sits close to the limit
- Whether the part of the method standard, the form of the test rig and the conditioning conditions match your technical requirements
- Where there are repeat measurements, whether the way the value was taken follows what the method specifies
None of these six needs a metrology background, they take little time, and they surface problems before the data gets challenged. Data problems behave like credential problems: the earlier they surface, the cheaper they are.
When you want help taking a data disagreement apart
If you are holding two reports that do not agree, or the receiving party has raised questions about uncertainty and the decision rule, send the reports and the technical requirements over. We will work through them in the five-step order above, first establishing whether the disagreement comes from the sample, the method, the equipment or the rule, and then deciding whether a retest is warranted and what exactly should be retested. Coverage by category is outlined under testing services. Call +86 132 4819 8029, or request a quote.