Developing Smart City Traffic Management Systems with OpenUSD and Synthetic Data

The SmartCow team has developed a smart city traffic management system using OpenUSD and synthetic data. Inaccurate data can have dangerous consequences for self-driving cars and result in inefficient transportation systems. To address this challenge, the team created digital twins that generate synthetic data sets to validate AI model performance.

To generate synthetic data, the team used NVIDIA Omniverse Replicator, which allows developers to programmatically generate annotated synthetic data for training AI models. The digital twins rely on OpenUSD features for scalable and interoperable USD simulations of large scenes that accurately represent the real world.

One of the main challenges the team faced was license plate recognition. To provide diverse training data for the model, they developed an extension in Omniverse to generate synthetic data. The data generation process involves creating a 3D environment in Omniverse that matches reality as closely as possible.

The initial model was trained on 3,000 real images, but using synthetic data helps remove bias that may be present in real image datasets. The team randomized the data across consistent parameters such as time of day, scratches, and viewing angle to provide accurate benchmarking and comparisons.

To integrate the training data from the Omniverse LP-SDG extension, the SmartCow team used NVIDIA TAO, a low-code AI model training toolkit that leverages transfer learning for fine-tuning models.

This workflow of using synthetic data and digital twins is an important step in building more effective AI models for transportation in smart cities. Synthetic datasets help overcome the limitations of limited data and can lead to efficient transportation systems and better urban planning.

Developers interested in building their own synthetic data generation solutions can download NVIDIA Omniverse for free and try the Replicator API in Omniverse Code.