Why LiDAR for self-driving UHF -- RJ explains in new interview (Stratechery)

[ACDC]

RJ Scaringe in a new interview on reason Rivian is using LiDAR for UHF hands-free / self-driving:

"In the process of building the models and until cameras can become meaningfully better, there’s very low cost, very fast ways to supplement the cameras that solve their weaknesses. So seeing through fog we can solve with a radar, seeing through dense snow or rain we can solve with a radar, seeing extremely far distances well beyond that of a camera or human eye, we can solve that with a LiDAR, our LiDAR is 900 feet. And then the benefit of having that data set from the radar and the LiDAR is you can more quickly train the cameras. The cameras, when I say train, it doesn’t mean we’re in there writing code to do this.​

​

The model understands this and so you feed this in and the neural net understands because you have the benefits of these non-overlapping modalities that have different strengths and weaknesses to identify, “Is that blurry thing out there actually a car?”, “Is it a person?”, “Is it a reflection off of a building?”, and when you have the benefit of radar and the benefit of LiDAR, that blurry thing way off in the distance that the camera sees starts to become — you can ground truth that much faster."​

Here's the full interview:
https://stratechery.com/2025/an-int...ge-about-building-a-car-company-and-autonomy/

Sponsored

 

[DuoRivians]

I’m curious why Rivian only chose forward facing lidar, whereas other fleet self driving services like Waymo and Zoox have 360 degree lidar

 

[godfodder0901]

I’m curious why Rivian only chose forward facing lidar, whereas other fleet self driving services like Waymo and Zoox have 360 degree lidar

Cost and limited ROI. All of the data behind you should have, theoretically, been mapped and the things that change can be handled adequately by vision and RADAR.

 

[DuoRivians]

Cost and limited ROI. All of the data behind you should have, theoretically, been mapped and the things that change can be handled adequately by vision and RADAR.

I dunno, busy city driving is hard to map everything around the car with just vision and radar. Bicyclists, pedestrians, etc. especially at night.

Perhaps non-dense suburbs can be fine with front lidar only. But when I see a Waymo navigate busy San Francisco roads, I think: “there’s no way a Rivian can do that any time soon”

 

[mwexler2]

I’m curious why Rivian only chose forward facing lidar, whereas other fleet self driving services like Waymo and Zoox have 360 degree lidar

One thing that was stressed and was pretty impressive was how invisible the Lidar is. From the side you can’t really see it. And head on you could easily confuse it for a camera. A far cry from the giant spinning ugliness on the Waymo. Also, most of the ones with 360 degree coverage have multple lidars. They definitely mentioned how LIDAR has gone from $10k 10 years ago to a few hundred today. Having multiple LIDARs would probably put that over $1k which would probably be prohibitive on the R2 (but maybe on a high-end Gen 3 R1???).
Finally, not mentioned but the rooftop spinning LIDAR and the side warts probably terrible for aerodynamics. Range is critical on these vehicles and the current design seems very aerodynamically efficient.

 

[BrayBay]

Always love a good RJ Interview.

My personal highlights:

• The context behind Gen 1 being dropped

  ... we launched in 2021 with our Gen 1 architecture, we almost immediately after that realized we needed a complete rethink of our self-driving approach.

  Well, we launched — and we didn’t realize, and this is the thing, and even some of our Gen 1 customers are not happy with this, but when we developed the Gen 1 system, this was on 2018, 2019, we didn’t know this big technical massive shift was going to happen.

  Gen 1, it’s asymptoted, both in terms of capability and it has no value to us in terms of data

• A good comparison why "rules based" wasn't going to work, using Alexa as an example

  And look at the progress that was made on Alexa, let’s say, relative to the progress that’s happened on GPT-3, 4 now, and beyond, it’s just like they’re not even closely related. And so the same thing is happening in the physical world with cars, and if you don’t have a data flywheel approach, you’re just not in the game and there’s no way you can compete.

• The LIDAR can see 900 feet

  ...seeing extremely far distances well beyond that of a camera or human eye, we can solve that with a LiDAR, our LiDAR is 900 feet.

• Very optimistic about autonomy

  It’d be like trying to sell a house without electricity, it’s going to become so fundamental to the functioning of the vehicle.

• More maps?!?

  So we brought Google Maps in, which was a big one, there’s more mapping platforms that’ll come in over time.

 

[HapticWagon]

To me, it sounds like they can use lidar equipped r2 (ground truth fleet) to train models and gather data that will help the non lidar gen2 cars!

 

[Honey]

I’m curious why Rivian only chose forward facing lidar, whereas other fleet self driving services like Waymo and Zoox have 360 degree lidar

Probably won't happen until gen 4

 

[KootenayEV]

Always love a good RJ Interview.

My personal highlights:

• The context behind Gen 1 being dropped
• A good comparison why "rules based" wasn't going to work, using Alexa as an example
• The LIDAR can see 900 feet
• Very optimistic about autonomy
• More maps?!?

Yes I read the whole thing too - many excellent nuggets in there! I have a Gen1 and it was interesting to see him voice some of the thoughts I've had.

Re the maps, I've always thought that incorporating backcountry maps would be good (here in BC there are digital backroad atlas available).

 

[AlphaSnowbordergirl]

I’m curious why Rivian only chose forward facing lidar, whereas other fleet self driving services like Waymo and Zoox have 360 degree lidar

Honestly, I don't think it's needed. Most of the time you will be going forward. And the things you need to see will be in front of you as that . If it's to the side or back, the radars should be enough and seeing at a large distance probably aren't really needed. Of course it depends ow far to the right and left it sees as it would be helpful for turns, but directly to the sides and behind, I do not think LiDar would add significant help that radar cannot cover. Even when driving yourself, you focus more on what's ahead as that's the direction you're going.

 

[sparked]

To me, it sounds like they can use lidar equipped r2 (ground truth fleet) to train models and gather data that will help the non lidar gen2 cars!

I think this is it. Rivian still basically believes that a vision only strategy could work. They are just at a disadvantage without all the data since their fleet is so small. So they are basically using lidar and radar to move faster and catch up since they can deploy cheaply on R2.

I believe the only publicly stated benefit to the consumer for Gen 3 is for Personal Level 4 Autonomy. So even Gen 2 has a lot of potential room to improve below that.

 

[captainjp]

To me, it sounds like they can use lidar equipped r2 (ground truth fleet) to train models and gather data that will help the non lidar gen2 cars!

This was specifically stated to be the case during the live stream.

 

[mwexler2]

I think this is it. Rivian still basically believes that a vision only strategy could work. They are just at a disadvantage without all the data since their fleet is so small. So they are basically using lidar and radar to move faster and catch up since they can deploy cheaply on R2.

I believe the only publicly stated benefit to the consumer for Gen 3 is for Personal Level 4 Autonomy. So even Gen 2 has a lot of potential room to improve below that.

They were explicit about this in the preso. Every Gen 3 R2 will be a ground truth vehicle. I don’t think this is the primary reason. They think vision + radar can handle 99% of the scenarios and 99% is not enough. They said this in the press.

Sponsored