Which ADAS is Really Driving? It’s Mobileye!

[portdirect]

Haha I was being very alert, and checking everything way more than I normally do lol... Felt like I was driving a 93 grand cherokee again But FWIW - these systems are safety conscious enough that if they don't feel confident, they just won't turn on

Be careful there - our TriMax's front camera array failed due to a loose connection MID AUTOMATIC LANE CHANGE instantly dropping control with the truck not aligned to a lane at 70mph in traffic - which was probably the reality check I kind of needed to not blindly (pun somewhat intended) trust these systems.

Sponsored

 

[mpshizzle]

Thanks @mpshizzle for empirically confirming what many of us have long suspected.

Of course! I do love getting empiracle data to back it up! I get a kick out of knowing exactly how this stuff is working!

I don’t believe it’s guaranteed—indeed, it seems quite unlikely—that Rivian will abandon Mobileye in favor of a proprietary system built on Nvidia compute, as seen in today’s Gen 2 models for ADAS/HighwayAssist. In interviews and press releases, Rivian has been very careful to describe the updated Gen 2 system solely as a data collection tool.

Two segments from the Sandy Munro interview with Vidya Rajagopalan were particularly influential in my assessment that the 2026 “eyes-off driving” capability isn’t coming to Gen 2, but will instead debut on the next generation of Rivian vehicles:

Oh interesting. I understood that conversation very differently. I could be wrong, but my take on it was that they built in lots of different high quality sensors so they could collect really good training data. But I didn't feel like she was saying that Gen 2's system is SOLELY for data collection. The presence of the dual nvidia chips indicates to me that they have at least some intention of doing more than just data collection.

All that said - "eyes off driving" does certainly seem rather far fetched for where we're at now haha. Tesla has a HUGE leg up on ML development and data training set and they are no where near there lol.

 

[mpshizzle]

Be careful there - our TriMax's front camera array failed due to a loose connection MID AUTOMATIC LANE CHANGE instantly dropping control with the truck not aligned to a lane at 70mph in traffic - which was probably the reality check I kind of needed to not blindly (pun somewhat intended) trust these systems.

YIKES!! That sounds terrifying! Glad you didn't get hurt in the process

 

[Donald Stanfield]

Of course! I do love getting empiracle data to back it up! I get a kick out of knowing exactly how this stuff is working!


Oh interesting. I understood that conversation very differently. I could be wrong, but my take on it was that they built in lots of different high quality sensors so they could collect really good training data. But I didn't feel like she was saying that Gen 2's system is SOLELY for data collection. The presence of the dual nvidia chips indicates to me that they have at least some intention of doing more than just data collection.

All that said - "eyes off driving" does certainly seem rather far fetched for where we're at now haha. Tesla has a HUGE leg up on ML development and data training set and they are no where near there lol.

It's going to be a LOOOOONG time before I think ANY system is ready for eyes off driving. Tesla isn't far ahead enough for it to matter for me, as I wouldn't trust it unsupervised any more than I'd trust Rivian's in their current state. I see videos of mistakes it makes constantly: hitting berms, going the wrong way on roundabouts, hitting stopped cars/ emergency vehicles, running kids over. That doesn't sound like something trustworthy to me.


All of these systems are ASSISTANTS, and just as I wouldn't trust my Assistant's work without checking it I don't trust these systems. It will be years of using them supervised without making dumb mistakes before I consider it. I'm not doing these stupid and simple things while driving, so me driving is the safter choice.

 

[mpshizzle]

It's going to be a LOOOOONG time before I think ANY system is ready for eyes off driving. Tesla isn't far ahead enough for it to matter for me, as I wouldn't trust it unsupervised any more than I'd trust Rivian's in their current state. I see videos of mistakes it makes constantly: hitting berms, going the wrong way on roundabouts, hitting stopped cars/ emergency vehicles, running kids over. That doesn't sound like something trustworthy to me.

FWIW - a few days ago I went for a ride with my friend in his Model Y with the latest hardware and FSD version.... WOW. It's very, very much better than it used to be. I'd say it's gotten to the point where an onlooker wouldn't know it's not a human driving. Honestly it does better than many humans do! Even smart enough to slow down for dips in the road.

ALl that said, as good as it is, still not perfect, and still needs supervision

 

[ElGuano]

They have never confirmed this for gen2 that I'm aware, and when several of us made the conjecture that @mpshizzle effectively validated multiple folks here questioned if Mobileye was even present in gen2 despite the sensor being clearly visible

One of the sources I recalled was from an earnings call with RJ and Wassim, but on looking it up, it was back in Q2 2023, a year before Gen2 was announced:
https://www.roic.ai/quote/RIVN/transcripts/2023-year/2-quarter?year=2023&quarter=2

So who even knows what/when they were talking about. I still remember seeing an interview/walkthrough more recently during the Gen2 press circus that mentioned the same thing about Mobileye + autonomy platform data collection, but alas I can't find it now...

 

[portdirect]

Oh interesting. I understood that conversation very differently. I could be wrong, but my take on it was that they built in lots of different high quality sensors so they could collect really good training data. But I didn't feel like she was saying that Gen 2's system is SOLELY for data collection. The presence of the dual nvidia chips indicates to me that they have at least some intention of doing more than just data collection.

From my experience with Rivian, it’s clear they don’t have a concrete plan—just hopes and aspirations when it comes to the technology they’re integrating (lest we forget Gen 1). I’m sure they hope they can achieve their goals, but there’s no certainty. Notably, they didn’t include LiDAR, which is part of the Hyperion 8 reference implementation—something I’m 99% sure they’re using as the foundation for their work, given their full commitment to the rest of the ecosystem. That omission introduces a degree of risk, which likely explains their increasingly cautious phrasing over time. The Xiaomi SU7 uses the same stack - and notably does include a lidar sensor and is not relying on a Mobileye or similar system as an interim.

While they are certainly leveraging Nvidia’s hardware and software where possible (eg the infotainment display, and the eye tracking in the next update), I suspect they have a long road ahead before they feel confident enough to make the switch. They should take as much time as necessary to get it right—but perhaps adopt a more measured approach to marketing, rather than emulating that other company.

Edit: added note re Xiaomi SU7

 

[mpshizzle]

From my experience with Rivian, it’s clear they don’t have a concrete plan—just hopes and aspirations when it comes to the technology they’re integrating (lest we forget Gen 1). I’m sure they hope they can achieve their goals, but there’s no certainty. Notably, they didn’t include LiDAR, which is part of the Hyperion 8 reference implementation—something I’m 99% sure they’re using as the foundation for their work, given their full commitment to the rest of the ecosystem. That omission introduces a degree of risk, which likely explains their increasingly cautious phrasing over time.

While they are certainly leveraging Nvidia’s hardware and software where possible (eg the infotainment display, and the eye tracking in the next update), I suspect they have a long road ahead before they feel confident enough to make the switch. They should take as much time as necessary to get it right—but perhaps adopt a more measured approach to marketing, rather than emulating that other company.

Haha this is true - very much startup culture where things are constantly shifting and there is no solid plan

Certainly a risk that they are taking, but for what it's worth from my rather uneducated point of view it seems like they do have a decent chance of success with the current trajectory and hardware that they've got. And they are certainly learning from their mistakes of over promising and under delivering lol.

Lol "the other company" sure is a wild card! I still don't understand how they're able to do as much as they do with cameras. But at the same time the lack of other sensors really is their downfall

 

[mpshizzle]

One of the sources I recalled was from an earnings call with RJ and Wassim, but on looking it up, it was back in Q2 2023, a year before Gen2 was announced:
https://www.roic.ai/quote/RIVN/transcripts/2023-year/2-quarter?year=2023&quarter=2

So who even knows what/when they were talking about. I still remember seeing an interview/walkthrough more recently during the Gen2 press circus that mentioned the same thing about Mobileye + autonomy platform data collection, but alas I can't find it now...

From all of the stuff that I've seen that they published where they specifically mentioned mobileye it seems to always be in reference to gen 1.

I'm sure there's plenty of content that I haven't seen or read that's out there, but at least the stuff in my recollection that seems to be the pattern

 

[Riviot]

Soooo Gen2 system is currently run on the same cameras and data that Gen1 has? ?

 

[godfodder0901]

Soooo Gen2 system is currently run on the same cameras and data that Gen1 has? ?

No. It has an upgraded version of the MobileEye system.

 

[DuoRivians]

Backstory
It’s been pretty well publicized that Gen 1 used a Mobileye autonomy system (It’s an “off the shelf” system purchased and used by MANY automakers). It failed to live up to expectations, and is a closed system that cannot be enhanced by additional sensors etc so Rivian decided to move to an in house system. This system has been very hyped with its high res cameras, upgraded sensors, and dual Nvidia Orin chips.

That all said - while Rivian hasn’t explicitly confirmed it, it’s all but certain that Gen 2 ALSO has a mobile eye system in addition to the in house system. So the question is: how much is Rivian’s in house system ACTUALLY doing?

So I did some testing!

My Theory:

• Mobileye handles: All driving assistance features and lane line visualization
• Rivian handles: Vehicle visualization

Test 1: Mobileye ONLY
Looking at the cameras on the windshield It’s very clear that there are 2 different camera sets, in different housings, made with different materials. One is a housing with plain black plastic and a single camera, the other with 2 cameras, with a felt material (presumably to prevent light reflections and glare). I made the assumption that the cheaper looking, single camera unit is the mobile eye one. So using masking tape I covered every camera on the vehicle EXCEPT what I assumed was the mobile eye camera.

The camera module directly in the center is what I assume must be the mobileye unit

The Result
There were no error messages or complaints of any kind from the vehicle. The 3D visualization on the driver’s screen was WACKO. Phantom cars appearing and disappearing all over the place. That said, it wasn’t completely random noise. The lane lines continued to function as they always do (a very vague representation of what the lanes in real life are actually doing). In an addition to imaginary cars randomly appearing and disappearing all over the place, it did show actual cars as well (as long as they were in front of me). But it had a VERY hard time classifying what kinds of cars they were, mostly showing them as either cargo vans or semi trucks. It also seemed to have a hard time figuring out the orientation of said cars.

This was taken on an empty neighborhood street with no vehicles nearby.

As far as diver assistance - everything worked perfectly. Lane centering, lane changes, everything. Even when it showed an imaginary vehicle RIGHT next to me, it proceeded with the lane changes as normal.


Test 2: Everything BUT Mobileye
For the second test, I did the inverse. I covered the Mobileye camera and left everything else open.

The Result
Almost immediately I got a warning saying the front camera was blocked, and forward collision warning is unavailable. The visualization worked pretty much as normal, except for the lane lines. There were no lane lines visible at all. Incidentally, there’s a weird arbitrary limitation where the display won’t show vehicles that aren’t in the lane directly next to you (so vehicles 2 lanes over are hidden). Since there were no lane lines shown at all, it seemed to show cars a bit farther away than it normally would have. And it did very well at that, might I add, even correctly classifying what kind of vehicle it was.

This photo was taken with only the mobile eye camera blocked. No lane lines, but a near perfect representation of traffic conditions, as well as classification of vehicles.

Driver assistance features were totally dead. No cruise control, no lane centering.


My Takeaways
I was right! (For the most part). It appears that Mobile eye is providing the actual driver assistance (and some of the safety features too), while the Rivian system is providing the visualization. The part I got wrong: it appears that there is at least SOME data for the 3d visuals that mobile eye feeds into the system, since it did still show some cars, even with all of the other cameras covered.


So what is that fancy Rivian system even doing?
Right now - collecting data. And lots of it. In the last month it has uploaded 228GB worth of data on my home wifi - all presumably going to train the Rivian AI. Very curious as to when Rivian will switch over to using their own system.

It seems like this analysis rests strongly on the assumption that the center camera is mobile eye.

 

[dleepnw]

Cool investigation. Makes sense they kept Mobileye. I'm sure building an ADAS system from the ground up takes time and they needed to be able to sell Gen2 vehicles with some level of autonomy.

I'm sure they will start replacing the Mobileye functionality with their Autonomy Platform to the point Mobileye becomes useless or used for redundancy.

 

[mpshizzle]

Soooo Gen2 system is currently run on the same cameras and data that Gen1 has? ?

Same brand, but a newer, higher end model

It seems like this analysis rests strongly on the assumption that the center camera is mobile eye.

You're absolutely right! And this testing isn't even necessarily CONCLUSIVE evidence that it IS mobileye.

That said, there are a couple reasons I feel reasonably safe making that assumption
1) It would be stupid and wasteful design for Rivian to put the windshield cameras in two separate housings, with two separate build materials and methods, if they're designing all of it
2) it's very clear that the center camera is the ONLY one used for anything that relates to driver assistance - which falls in line with mobile eye's "all in one" approach.

 

[ThirteenElectrics]

If you’re wondering why Rivian audio sucks so much, why there are no sunglasses holders under the seats, 12V outlets in under the dash or in the frunk, or why service center wait times are so long, or they sell pollution credits to other auto manufacturers: it’s because Rivian wasted your money on two ADAS systems in every car, but is using (and plans to use) only one. And people wonder how Rivian has racked up $23 billion in losses so far.

Sponsored