By: Haziqa Sajid, Trista Li, and Pedro Pachuca

‍

A single manufacturing defect can trigger costly system failures, product recalls, and significant brand damage. Ford’s recent $570 million recall stands as a stark reminder. Yet traditional AI solutions fall short where it matters most - catching rare but critical defects before they become costly problems.

Waiting months to collect real-world data on anomalies like micro-porosity or faint weld cracks is not a viable strategy; it's a guaranteed project delay. In high-stakes manufacturing, especially automotive, every day of delay means increased risk and lost value. What if you could bypass this waiting game entirely?

This post introduces a fast, accessible workflow that transforms what was previously a months-long process into one that delivers results in hours. This approach combines two powerful tools:

‍

1. Advex AI: Generates labeled synthetic data for rare defects on demand
2. NVIDIA TAO: Leverage and fine-tune pre-trained models and fine-tuning capabilities for rapid development, customization and deployment

‍

Our testing across three high-stakes scenarios revealed dramatic improvements across crack detection, weld defect identification, and robotic handling precisions. 

‍

‍

A Proven AI Deployment Workflow: from Months to Days

First, we leverage Advex AI's diffusion-based platform to generate hundreds of edge-case scenarios with pixel-level annotations. The platform can generate hundreds of images in hours, significantly speeding up the data collection time. 

‍

                                                                                         ^{Image: Advex AI’s Generated Synthetic Data}

‍

‍

Next, we selected three pre-trained vision models from NVIDIA TAO to accelerate model building: 

‍

• mit-b5 (SegFormer): An efficient and powerful segmentation model that uses text prompts.

‍

• NVDINOv2: A general-purpose vision foundation model known for its exceptional feature extraction and ability to be fine-tuned with unlabeled training data via self-supervised learning.

‍

• C-RADIOv2: A state-of-the-art vision foundation model for visual feature extraction. Performs strongly in industrial anomaly detection. As a generic foundation model, it serves a similar role to NV-DINOv2.

‍

To ensure transparency in our setup, we explicitly define here the task heads and backbone variants used for each pre-trained vision foundation model. All models were trained with a segmentation head (decode_head) and fine-tuned following NVIDIA TAO’s SegFormer training flow.

‍

• MIT-B5

      Backbone: mit_b5

       Task head: Segmentation (decode_head with feature strides [4, 8, 16, 32])

‍

• NV-DINOv2

       Backbone: vit_giant_nvdinov2

       Task head: Segmentation (decode_head with feature strides [4, 8, 16, 32])

‍

• C-RADIOv2

       Backbone: c_radio_v2_vit_huge_patch16_224

       Task head: Segmentation (decode_head with feature strides [4, 8, 16, 32])

‍

Once their spec files using the Segformer guide were developed, the models were fine-tuned using the Tao Launcher:

‍

tao model segformer train -e spec_file.yaml

‍

Finally, we fine-tuned each model with an augmented dataset from Advex. Additional augmentation was applied using NVIDIA TAO through the spec file. The chart below summarizes the winning model for each use case, providing visual proof of the significant performance gains.

‍

‍

                                                              _{Graph: Winning Model Performance Summary on Across Segmentation Use Cases}

‍

See how they achieve stronger performance on difficult-to-catch anomalies in the following real-world case studies:

1. Prevent Recalls with Enhanced Crack Detection
2. Accelerate Weld Quality Assurance in Automotive
3. Optimizing Warehouse Automation with Box Segmentation

‍