On this page
In brief: Synthetic images are useful for covering variation that is rare or expensive to capture, but a score on generated images says nothing about the production line. Only an independent set of real images can show whether the synthetic data helped.
What rendered images can contribute
A rendered scene lets you vary camera position, material properties, lighting, clutter and rare defect conditions without staging each configuration on the line. The labels come with the image: the renderer knows where every object and defect is, so masks and bounding boxes do not have to be drawn by hand. Synthetic data is therefore most useful early in a project, before enough real defects have been collected, and for conditions that seldom occur.
Where the domain gap comes from
Real surfaces scatter and reflect light in ways that a material model only approximates, and they collect dust, oil and scratches. Real parts sit within mechanical tolerances, and real cameras add noise and blur. A rendering that looks convincing to a person may still lack exactly the cue that a model learns to rely on, or contain a rendering artefact that the model learns instead. Generated defects carry a second risk: if their shape or texture differs from real failures, the model learns the wrong concept and still scores well on synthetic test images.
Keep a wall between training data and real test data
Separate real factory images into training, validation and final holdout-test sets before model development. Use validation data for iteration, and keep the final holdout out of training, tuning and model selection. Cover relevant shifts, suppliers, surface conditions and camera configurations without leaking near-duplicate images between sets. Compare a fixed baseline and a fixed candidate on the held-out real cases, reporting false accepts and false rejects by defect class as well as overall results. If repeated inspection of the test results guides further tuning, that set has become development data and a new untouched final test is needed. Synthetic-data benefits require representative real-world evidence, not just a higher synthetic benchmark score.
What vendor tooling claims, as of October 2026
On 16 March 2026 NVIDIA announced its Physical AI Data Factory Blueprint, which it describes as an open reference architecture that unifies and automates how training data is generated, augmented and evaluated. According to the release, one component, Cosmos Evaluator, automatically scores, verifies and filters generated data. The release said that the blueprint was expected to be available on GitHub in April. It gives no quantitative performance results and carries a notice on forward-looking statements. Filtering generated data for plausibility is a check on the data; it is not a measurement of your model on your line. The announced April GitHub availability was a forecast and must be checked separately from the current release status.
Decide deployment by measurement
Synthetic data is an input to development, not a reason to skip production trials. Ramp up in stages, with the previous inspection method running in parallel where possible. Keep model versions so that you can roll back, confirm processing time on the real hardware and define what the station does with an object or a lighting condition it has not seen. The NIST AI Risk Management Framework makes the general point that AI systems should be tested before deployment and regularly while in operation.
Checklist before committing budget or scope
- The variables that were varied in rendering, and the renderer and asset versions used.
- A real hold-out set, collected separately and never used for training or tuning.
- False-accept and false-reject rates per defect class, before and after adding synthetic data.
- Stress tests with a supplier, batch or lighting condition absent from all training data.
- Real examples of each defect class to compare with the generated ones.
- A rollback procedure and drift monitoring with a named owner.
What this guide does not establish
This guide is a planning aid. It does not report tests, evaluate the tools of NVIDIA or any other vendor, or state how much synthetic data helps; that depends on the part, the optics and the defects. It does not assess compliance, and inspection that feeds a safety decision needs its own validation by suitably qualified parties.
Sources and further verification
- NVIDIA — NVIDIA Announces Open Physical AI Data Factory Blueprint to Accelerate Robotics, Vision AI Agents and Autonomous Vehicle Development (press release, 2026-03-16; vendor announcement containing forward-looking statements; accessed 2026-10-12).
- NIST — AI Risk Management Framework (AI RMF 1.0), Core: Govern, Map, Measure, Manage (2023 framework; NIST states that a revision is in progress; accessed 2026-10-12).
Next useful step
Bring real sample images of good and defective parts, your defect definitions, any CAD data you have and a description of the camera and lighting set-up to the Robotics & Automation page. The AYXWORKS portfolio shows the engineering approach behind these guides. Any project that follows is scoped and delivered through Eurobots/ReBots, and it starts with feasibility and responsibility boundaries, not with an assumed turnkey installation.
Image credit: ayxworks — Original / rights held.
