Series:
The Rise of Synthetic Image Data for Deep Learning


The modern revolution in machine learning and robotics have been largely enabled by access to massive repositories of labeled image data. Yet obtaining real-world data can be expensive, time-consuming, and inconvenient. In response, simulation tools have grown in popularity, and have become fundamental in the development of algorithms, particularly in the fields of Robotics and Deep Reinforcement Learning. In this three-part series, I discuss the co-evolution of simulation tools, particularly those that generate photorealistic images, and the data-hungry machine learning algorithms they have fueled.

  • In Part 1: The Importance of Simulation in the Age of Deep Learning, I discuss the historical significance of simulation tools in the field of robotics and the promise and limitations of photorealistic simulation tools.
  • Part 2: Bridging the Realism Gap for Synthetic Image Data is a survey of different approaches to making simulated data more useful for real-world applications.
  • Finally, in Part 3: Simulation Environments for Robotics and Deep Reinforcement Learning, I list a number of simulation environments for different applications and some trends I think reveal where the community is headed.

The modern revolution in machine learning and robotics have been largely enabled by access to massive repositories of labeled image data. AI has become synonymous with big data, chiefly because machine learning approaches to tasks like object detection or automated text translation require massive amounts of labeled training data. Yet obtaining real-world data can be expensive, time-consuming, and inconvenient. In response, many researchers have turned to simulation tools — which can generate nearly limitless training data. These tools have become fundamental in the development of algorithms, particularly in the fields of Robotics and Deep Reinforcement Learning.

This is the first post in a three-part series on the role of simulated image data in the era of Deep Learning. In this post, I discuss the significance of simulation tools in the field of robotics and the promise and limitations of photorealistic simulators.