Google has introduced a new experimental feature that allows its AI assistant to perceive the environment outside vehicles. Using its popular Gemini system, the technology connects directly to a vehicle's built-in forward-facing camera, effectively transforming it into a pair of intelligent eyes. This integration enables Gemini to answer complex queries regarding buildings, landmarks, or street layouts as you drive by.
The technology giant showcased this project using the Volvo EX60, a recent entry into the premium market. This fully electric SUV measures 189.1 inches in length, 75.1 inches in width, and stands 64.4 inches tall, featuring an extended 116.9 inches wheelbase that maximizes cabin space. In Europe, the base rear-wheel-drive model is priced at approximately $77,900.
Sameer Samat, President of Android at Google, led the live demonstration alongside Logan Kilpatrick, Product Lead for Google AI Studio. Conducted at Google’s Mountain View campus in California, the test aimed to explore what occurs when the Gemini system visually perceives the environment during real-world driving. For those familiar with the camera features on standard mobile devices, the overall concept will feel familiar.
When tech companies discuss camera-equipped vehicles, significant privacy concerns often emerge. Google addressed this by clarifying that the vehicle does not continuously monitor the road. The camera feed is blocked until the driver interacts with the system and poses a question. Only when prompted does the front-facing camera transmit visual data to the AI server. Once the system provides an answer, access to the camera is immediately terminated.
During the test drive, the software effectively identified local monuments and features of the campus. For instance, when Samat inquired about an object outside, the AI correctly identified a local public art installation known as "The Orb." The assistant also successfully recognized the solar roof of the nearby Gradient Canopy building. To further test its limits, the team asked about the Shoreline Amphitheater, to which the AI accurately recounted the architectural history of the iconic venue.
However, early adopters should remain patient as the system is not yet perfect. The live demonstration revealed a noticeable delay while the car communicated with the remote servers. Drivers should not expect a visual experience akin to a cinematic screen; unlike a standard smartphone, the significant 15.0-inch central touchscreen in the Volvo EX60 does not display a live camera feed. Instead, the driver simply hears a spoken response to their inquiry.
The Volvo EX60 serves as a robust hardware platform for advanced digital experiments. The base P6 model comes equipped with an 83 kWh battery pack and an electric motor producing 275 kW (374 hp). For those seeking enhanced performance, the P12 all-wheel-drive variant increases the battery capacity to 117 kWh, delivering an impressive 500 kW (680 hp), although its price rises to approximately $88,400.
Google engineers continue to explore various ways this software can assist individuals during their daily commutes. For example, the AI could aid drivers in navigating areas where they do not speak the local language by translating complex road signs, thereby eliminating the risky habit of diverting attention to a mobile phone screen while maneuvering through busy intersections or attempting to read unfamiliar signage.
For those interested in the demonstration, a complete presentation of the latest Gemini developments is available; skip to 34:36 to view the Volvo and AI experiment:
Currently, Google has not provided any concrete timeline for releasing this feature to the public. The company also refrained from disclosing whether other vehicles utilizing the Google Built-in ecosystem will gain access to this camera capability in the future. For now, this technology offers a fascinating glimpse into the future of automotive software.
Source
