How to Validate the Repeatability and Reproducibility of a Handheld 3D Scanner

A valid repeatability and reproducibility study must test the complete handheld 3D scanning process, not only the scanner. Repeating a scan several times without moving the part may show short-term instrument precision, but it does not represent normal production work. Operators usually reposition the part, replace or reuse targets, rebuild the alignment, rescan hidden areas, process the data, and extract features. Each step can affect the reported result.

The validation has three layers: verify the equipment, study representative parts under normal conditions, and monitor a check part over time. Acceptance criteria should be linked to part tolerances and decision risk.
 

Repeatability and Reproducibility Measure Different Conditions

Repeatability describes the agreement between repeated measurements made under repeatability conditions. These normally include the same procedure, operator, instrument, location, and a short time interval. Reproducibility describes agreement when specified conditions change. The changed conditions may include operator, setup, day, location, scanner unit, or another factor relevant to the intended use.

A reproducibility statement is incomplete unless it identifies what changed. A study that changes only the operator does not prove reproducibility across different fixtures, scanner units, sites, or temperature ranges. The study scope should match the process that will be approved.

ISO 5725-1:2023 provides current definitions and general principles for evaluating trueness and precision. NIST terminology also treats repeatability and reproducibility as parts of measurement precision, with reproducibility requiring the changed conditions to be specified. These definitions prevent a common reporting error: using “accuracy,” “repeatability,” and “reproducibility” as interchangeable terms.
 

Validate the Complete Measurement Process

The scan result is produced by hardware, software, the part, the environment, and the operator. A realistic 3D inspection validation should consider the sources that can affect the final characteristic:

· scanner calibration and verification status;

· warm-up, ambient temperature, vibration, and lighting;

· part temperature and dimensional stability;

· surface color, gloss, translucency, and preparation;

· marker quantity, adhesion, and spatial distribution;

· scan distance, viewing angle, coverage, and operator path;

· part or fixture movement during acquisition;

· registration across multiple scan positions;

· CAD alignment and datum construction;

· filtering, meshing, smoothing, and hole filling;

· feature extraction and fitting settings;

· software version and report template.

The study does not need to vary every factor simultaneously. It does need to include the factors that occur in normal use. If production operators routinely remove and remount parts, a study that leaves the part untouched understates process variation. If one approved fixture and one locked software recipe are always used, those conditions can remain fixed and should be documented.
 

Begin with Equipment Verification

Complete a status check before collecting Gauge R&R data. Confirm that calibration is current, inspect the scanner and accessories, and run the manufacturer’s required verification routine. A known artifact such as a certified length, ball bar, sphere, or another application-appropriate standard can reveal a problem before time is spent on a production study.

ISO 10360-13:2021 specifies acceptance and reverification tests for optical 3D coordinate measuring systems when measuring lengths stated by the manufacturer. It applies when the scanned surface characteristics are restricted to a cooperative range. The standard gives manufacturers and users a common basis for checking system performance, but it does not validate every material, geometry, alignment, or inspection procedure used in a factory.

Record the equipment configuration during verification. At minimum, include the scanner serial number, calibration status, scanning mode, reference artifact, temperature, software version, verification result, and any photogrammetry or scale-bar configuration. A passed verification establishes that the system entered the study in a known state. It is not the final production-capability result. 
 

Select Parts and Characteristics That Represent Production

A useful study needs real part-to-part variation. Select parts that span the expected production range instead of choosing ten nearly identical samples from the middle of the process. Include examples near both specification limits when they are available and safe to use. The set should also represent the materials, finishes, sizes, and geometries that create the normal scanning workload.

Measure characteristics that matter to the acceptance decision. These may include distances, hole diameters, plane relationships, positions, profiles, and selected surface regions. A full-field color map is useful for diagnosis, but the study also needs repeatable numerical outputs with fixed definitions.

One master part can support a precision or bias check, but it cannot show whether the measurement system can distinguish differences among production parts. When only one part is available, report the study as a limited precision assessment and avoid presenting it as a full production Gauge R&R.

Reference values should come from a traceable method with adequate capability for the characteristics being studied. Depending on the feature, the reference may be a calibrated artifact, a qualified CMM, a suitable gauge, or another approved measurement method. The reference uncertainty should be included when bias or measurement uncertainty is evaluated.
 

Design a Practical Gauge Repeatability and Reproducibility Study

A common starting design for a crossed study is ten parts, three operators, and three trials. That produces 90 measurement sets. This is a practical planning model, not a universal requirement. Part availability, inspection time, risk, customer rules, and the statistical method may justify another design.

Randomize the measurement order so that warm-up, drift, fatigue, or production sequence is not confused with a part or operator effect. Operators should not use earlier results to adjust a later trial. Each trial must begin from the same defined starting condition.

For a production-process study, an independent trial usually means removing and repositioning the part, beginning a new scan, rebuilding the specified alignment, and running the controlled analysis again. Whether targets are replaced depends on the validation objective. Keeping them in place isolates other sources of variation. Replacing them tests the marker-placement procedure as part of the process. A useful program often runs both tests at different stages.

When ten production parts are unavailable, use the largest defensible sample and repeat the study after more variation becomes available. Another option is to use qualified reference parts representing low, nominal, and high conditions. The report should state the limitation rather than implying that a small homogeneous set represents the full process.
 

Separate Instrument Repeatability from Process Repeatability

Two repeatability tests answer different questions.

Instrument repeatability keeps the part, fixture, targets, alignment method, operator, and environment as stable as possible. The operator reacquires the data over a short interval. This isolates short-term variation in data capture and processing.

Process repeatability requires a complete rerun by the same operator. The part is removed or repositioned, the normal setup steps are repeated, and a new scan and analysis are created. This result is usually more relevant to production because it includes fixture, target, alignment, and workflow effects.

Reporting both values helps locate the dominant source. Good instrument repeatability combined with poor process repeatability points toward setup, fixturing, targeting, alignment, or work instructions. If both are poor, the team should first check equipment status, surface behavior, environment, and scan coverage.
 

Lock the Analysis Method Before Collecting Data

Software choices can introduce as much variation as scanning technique. Define the analysis recipe before the first trial and apply it without discretionary changes. The procedure should identify:

· the CAD and drawing revision;

· the datum reference frame and alignment sequence;

· the regions included or excluded from alignment;

· filtering and mesh settings;

· feature fitting rules;

· treatment of edges, holes, and missing data;

· the surface comparison mask;

· report units, rounding, and color scale.

Do not change the alignment to improve an individual result. Do not alter the color scale between trials. If the method requires manual region selection, document the selection rule and include that activity in the reproducibility study. A locked template reduces operator discretion and makes later investigations traceable.
 

Analyze Numerical Features and Full Field Data

For dimensions and GD&T characteristics, a Gauge R&R analysis separates the observed variation into part-to-part variation, equipment variation, operator variation, and, when the design supports it, operator-by-part interaction. The average-and-range method can provide a practical summary. Analysis of variance is more diagnostic because it can estimate interaction effects and test whether operators behave differently on particular parts.

Report the repeatability standard deviation, reproducibility component, combined Gauge R&R, part-to-part variation, and the basis for any percentage. A percentage relative to tolerance answers a different question from a percentage relative to observed study variation. State which denominator is used.

For full-field data, use identical alignment, masks, and sampling rules. Suitable outputs may include the mean surface deviation, standard deviation within a defined region, a preselected percentile of absolute deviation, and repeatability at named critical locations. Raw maximum and minimum mesh values are often sensitive to edge noise, isolated points, or incomplete coverage. They should not be the only acceptance metrics.

Bias, linearity, and stability complete the picture. Bias compares the average result with a traceable reference. Linearity checks whether bias changes across the measurement range. Stability checks whether results change over time. A process can be highly repeatable and still produce the wrong value, so precision alone does not establish accuracy.
 

Set Acceptance Criteria Before the Study

Define the decision rule before the data are reviewed. Common practice may use percentage Gauge R&R bands as screening guidance, but no percentage is universally suitable. The threshold should reflect the feature tolerance, measurement risk, customer requirement, process capability, and consequences of false acceptance or false rejection.

Review the results feature by feature. A scanner process may be capable for broad surface profile inspection but require another method for a small bore or a tight datum-related position. The validation scope should name the approved features, part families, size range, surface conditions, software recipe, and operator qualifications.

Gauge-study variation may contribute to a measurement uncertainty evaluation, provided the study represents the real working process. NIST cautions that a gauge study can miss long-term effects. If the measurement process changes over time, the original standard deviation no longer describes current performance. This is why short-term Gauge R&R and long-term stability monitoring should be treated as connected controls.
 

Diagnose Poor Repeatability and Reproducibility

The pattern in the data often indicates where to investigate.

· High repeatability variation may indicate part movement, unstable targets, inconsistent scan distance, poor optical access, surface response, or environmental change.

· High operator variation may indicate unclear work instructions, different coverage, inconsistent alignment, or manual feature-selection differences.

· A strong operator-by-part interaction may mean that some geometries are handled differently by different operators.

· Variation concentrated at edges, deep holes, thin walls, or reflective areas may require a revised acquisition plan or feature-specific method.

· A shift by day or week may indicate changing temperature, verification status, software configuration, fixture condition, or operator practice.

Correct the assigned cause and repeat the affected portion of the study. Do not average away a known process problem. Repeated scans can reduce random variation in an average, but they do not remove systematic bias or repair an uncontrolled workflow.
 

Current SCANOLOGY Systems in a Controlled Validation

SCANOLOGY is the overseas subsidiary of SCANTECH (Hangzhou) Co., Ltd. and sells the company’s industrial product line. Current product specifications help define the starting equipment configuration, while the on-site study establishes performance for the actual task.

SIMSCAN-S Gen2 for Compact and Detailed Parts

SIMSCAN-S Gen2 is a current palm-sized metrology-grade scanner for compact parts, narrow spaces, grooves, and detailed geometry. The official page lists accuracy of 0.015 mm and states that performance is evaluated under ISO 10360-13, VDI/VDE 2634 Part 3, and JJF 1951 in an ISO/IEC 17025-accredited setting. The current palm-sized family comprises SIMSCAN-S Gen2 and SIMSCAN-E Gen2.

 

A study using SIMSCAN-S Gen2 should still include the real surface, targets, alignment, operators, and critical features. The published specification defines equipment performance under stated conditions; it is not a substitute for production Gauge R&R.

KSCAN-E for Mixed Sizes and Larger Measurement Volumes

KSCAN-E is a current wireless all-in-one metrology 3D scanner with stated accuracy of 0.020 mm. Its integrated photogrammetry and multiple scanning modes can support validation programs that cover varied part sizes and measurement volumes. The study must record the mode and photogrammetry configuration because changing the configuration changes the measurement conditions.

 

When several modes are used in production, validate each approved mode or treat mode as a study factor. Combining results from unrecorded configurations can hide a systematic difference.

DefinSight for a Controlled Analysis Recipe

DefinSight supports data acquisition, processing, scan-to-CAD inspection, feature evaluation, and reporting. It can help standardize the alignment, filters, feature definitions, comparison regions, and report template used across trials.

 

Software control should be procedural as well as technical. Save the approved project template, restrict unplanned setting changes, record the software version, and require revalidation when an update changes acquisition or analysis behavior.
 

Maintain Capability After the Initial Study

A passed study describes the process under the conditions tested. It does not guarantee permanent performance. Establish an ongoing program using a stable check part or artifact. Measure it at a defined frequency and plot a critical result on a control chart. Investigate shifts and trends before they reach an acceptance limit.

Reverification and revalidation triggers may include:

· calibration, repair, or hardware replacement;

· scanner impact or suspected damage;

· new software or firmware affecting measurement;

· a new fixture, target strategy, or alignment method;

· a new material, finish, size range, or part family;

· a sustained control-chart shift;

· a change in site, temperature range, or operating procedure.

Periodic equipment reverification and process checks serve different purposes. The equipment test confirms the scanner’s stated performance under defined conditions. The check-part history confirms that the production workflow remains stable.
 

Validation Checklist

Before approving the process, confirm that the record includes:

· the measurement purpose, tolerances, and acceptance criteria;

· scanner, accessories, calibration, and verification status;

· representative parts and traceable reference values;

· operators, trials, randomization, and independent setup rules;

· environmental and part-temperature records;

· target and surface-preparation procedures;

· controlled acquisition, alignment, mesh, and feature settings;

· repeatability, reproducibility, bias, and stability results;

· approved part families, features, surfaces, and configurations;

· periodic checks and revalidation triggers.
 

Conclusion

Handheld 3D scanner validation should begin with equipment verification and continue through a representative production study. Short-term repeatability, complete process repeatability, and reproducibility across specified changing conditions should be measured separately. Representative parts, randomized trials, independent setups, controlled software settings, and traceable reference values make the results defensible.

The final approval should identify the part families, features, surfaces, configurations, and operator conditions covered by the study. Long-term check-part monitoring and planned reverification then provide evidence that the process remains suitable for its assigned work.
 

Frequently Asked Questions

How many parts and operators are needed for a handheld scanner Gauge R and R study

Ten parts, three operators, and two or three trials are common planning choices, but they are not universal requirements. Use enough parts to represent actual production variation and enough trials to estimate the effects relevant to the approval decision.

Should the part be removed and repositioned between scans

Yes for a production-process repeatability study when repositioning is part of normal work. A separate short-term instrument test may keep the part fixed. Report the two results separately because they answer different questions.

Should targets be replaced for every trial

Replace them when marker placement is part of the process being validated. Keep them in place when the objective is to isolate other sources of variation. Many programs use an initial controlled test followed by a complete process test.

Can mesh to mesh deviation replace dimensional Gauge R and R

No. Mesh comparison can reveal spatial patterns, but critical dimensions and GD&T characteristics need defined numerical results. Full-field metrics should use fixed alignment, masks, and statistics so that trials remain comparable.

Does passing ISO 10360-13 validate the production scanning process

No. ISO 10360-13 supports acceptance and reverification of an optical 3D coordinate measuring system under defined conditions. Production validation must also cover the actual part, surface, setup, operators, software, and features.

How often should repeatability be revalidated

Set the interval from risk, use frequency, process stability, and customer requirements. Revalidate after material changes to equipment, software, fixtures, environment, part family, or procedure, and when control data show a sustained shift.