Simulation software specialist rFpro has announced a collaboration with Sony Semiconductor Solutions Corporation (Sony) to develop high-fidelity sensor models integrated into rFpro software.
The collaboration will focus on improving the fidelity of simulation solutions to significantly reduce the requirement to collect physical data during the sensor development cycle. It will enable perception systems for ADAS and autonomous vehicles to be developed, trained and tested in a virtual environment, rapidly accelerating development.
Matt Daley, rFpro Operations Director said:
“Sony Semiconductor Solutions has been a crucial collaborator in the development of our recently launched ray tracing technology and rFpro’s Multi-Exposure Camera technology, which accurately replicates what cameras ‘see’ for the first time. By working closely with Sony and integrating its sensor models into our technology we have been able to achieve a high level of correlation in the simulation. As a result, together, we are helping the industry reduce its reliance on collecting real-world data, which is expensive and time intensive.”
“The collaboration between rFpro and Sony Semiconductor Solutions will provide an automotive-grade End2End simulation pipeline to the ADAS perception system developers,” said Kenji Onishi, Deputy Senior General Manager, Automotive Business Division, Sony Semiconductor Solutions Corporation. “Sony has prepared a sensor model based on the internal architecture of the image sensors used in camera systems to achieve automotive-grade fidelity.”
Perception software, which is developed using deep neural networks, and automotive image sensors are both components of this pipeline and have become more sophisticated to meet the demand for higher ADAS functionality and robustness, putting significant pressure on development and validation. Automotive-grade simulation is a tool to generate synthetic data and has been seen as a solution.
Simulation enables vehicle systems to be subjected to a limitless array of scenarios. The weather, time of day, amount of traffic and pedestrians can all be controlled and varied independently and automatically. In rFpro everything in the scene has been physically modelled with accurate material characteristics and the road surface recreated to within a height accuracy of 1mm.
“Vehicles can drive thousands of high-value, high-activity virtual miles every day in simulation,” said Daley. “Edge cases can be identified and new iterations generated quickly to thoroughly exercise sensor systems. It removes the need to wait for exposure in the real world, where the majority of miles driven are relatively uneventful.”
Sony is the first partner to collaborate with rFpro on its recently launched ray tracing technology, which is the company’s software-in-the-loop (SIL) solution aimed at generating synthetic training data. It uses multiple light rays to accurately capture all the nuances of the real world. As a multi-path technique, it can reliably simulate the huge number of reflections that happen around a camera. This is critical for low-light scenarios or environments where there are multiple light sources to accurately portray reflections and shadows.