Tesla CEO Elon Musk stated that the company’s Autopilot systems are being upgraded to 4D, which will improve the performance and capabilities of the semi-autonomous driving function. Currently, Autopilot is operating with “~2.5D,” Musk said.
The developments came from a question that was asked by a Twitter follower of Musk’s who has issues using Tesla’s Summon feature on his driveway. Summon allows owners to retrieve their vehicles by using their Smartphones. By holding the “COME TO ME” button within the Tesla app, the car will use GPS vectoring to travel to the location of the phone.
Summon is a part of Tesla’s Full Self-Driving suite. But, the rework of Autopilot’s dimensional upgrade is apart of something much bigger. Perhaps it deals with a complete rewrite of Autopilot that will extend the company’s FSD features.
We need to finish upgrading Autopilot to 4D vs ~2.5D, then it will go up very steep slopes
— Elon Musk (@elonmusk) July 22, 2020
However, the owner stated that the grade of his driveway is slightly steeper than 10%, which inhibits the vehicle from traveling up roads that have inclines. The steepness of the slope, along with normal transitions from a street to a driveway, can present issues for Tesla’s Autopilot. This could be due to the lack of information that Tesla’s Neural Network has for navigating these environments.
With that being said, Tesla is developing a 4-dimensional system for Autopilot. The development of new elements for Autopilot to comprehend the surroundings and road environment of the vehicle could lead to more drastic improvements and an increasingly accurate comprehension of the roads a vehicle travels on.
Tesla’s currently Autopilot suite uses ~2.5D, Musk said. Now, it uses two-dimensional images along with labels, which could account for the around 2.5 dimensions that Musk spoke of in the tweet.
Adding dimensions to the Autopilot system will simply increase the accuracy of how the car reacts in certain situations. Currently, Tesla uses images from Autopilot cameras that are labeled with information. Tesla could use 3-dimensional stereoscopic scenes that are reconstructed from video, along with timestamps to improve accuracy.
A few members of the Tesla community put their two cents in on what the 4-dimensional Autopilot system could entail.
Reddit user u/__TSLA__ stated that curating a massive series of traffic scenarios and objects that a car might encounter during a drive could improve the accuracy of Autopilot and Tesla’s self-driving capabilities.
However, another Reddit user, u/Semmel_Baecker, said that 4D could mean that the Autopilot cameras could build a real-time 3D environment and then predict the movements of labeled objects in 4D based on past behaviors of other vehicles. This strategy would effectively use the Neural Network to learn the reactions of other drivers or objects.
Tesla continues to develop Autopilot behaviors to eventually release a “feature complete” Full Self-Driving suite in the future. The electric automaker continues to release patents that aim to build a more accurate Autopilot system that will accelerate the company’s journey toward Level 5 Autonomy, which Musk says is coming soon.
Most recently, Tesla submitted a patent titled, “Enhanced Object Detection for Autonomous Vehicles Based on Field View,” that would crop important objects in images and increase the resolution of those images. If pedestrians, vehicles, or other objects are available in an image, they would be available at an increased resolution to improve the accuracy of Autopilot.
Tesla’s exact plans for an Autopilot upgrade to 4-dimensional imagery is unknown. The increased accuracy is necessary for the company’s cars to drive in any environment. Tesla will soon release FSD’s “Driving on City Streets” function, which will complete the suite.
