Nobody Seems To Have An Answer To Autonomy’s Biggest Problem But I Have An Idea

Illustration: Jason Torchinsky

I’m finding myself banging on one particular autonomous vehicle drum more and more recently. It’s the annoying drum that many AV enthusiasts (and, I’m gonna say, especially Tesla-stans) like to ignore the most: The sobering but accurate fact that until autonomous vehicles figure out some kind of failover system that does not require immediate driver intervention, they’re just going to be Level 2 driver-assist semiautonomous features, no matter what misleading name Elon gives them. I have yet to hear any company’s comprehensive plan for this, so I figured what the hell, let’s take a crack at it.

So we’re all on the same page, let me explain the fundamental problem: the reason why every autonomous system on the market today, like Tesla Autopilot or GM’s Supercruise, requires a driver to be ready to take over at any moment with minimal notice is that no currently available system has any means to handle failover situations on its own.

That means that if something goes wrong with the AV system’s ability to perceive the road or understand its environment — something that can be the result of a vast number of factors, including weather conditions, a technical failure of a camera or sensor, a mud splash or insect squish over a camera or sensor lens, ice, road grime, rock impacts, whatever — then all a Level 2 system can do is call for help and hope that the person behind the wheel is paying attention.

If they’re not, and they’ve zoned out or are sleeping or watching a movie or anything else, then the end result can be a nasty wreck, as we’ve seen before.

These kinds of failures would not be necessarily uncommon, even if AVs come with sensor cleaning systems and even redundancies. Think how many times windshields get chipped driving behind gravel trucks or similar vehicles; a little chip on a sensor or camera window, even if there are redundant sensors inside, would be crippling.

G/O Media may get a commission

Redundant systems will be hugely important, but also very expensive; it’s possible there may be a demand for AVs with not total redundancies, too, or even sensor-cleaning setups, no matter how smart it is to have such things. Plus, some systems, like lidar domes, may just be too expensive to be fully redundant. There are all kinds of reasons why a good failover plan is important.

This is why we can’t consider systems like Autopilot to be actual self-driving systems. If you have to remain constantly alert and ready to take over, then you can’t sleep or read or do any of the other things that you’d want to do if you’re freed from the task of driving.

Really, you may as well just drive, because a vigilance task like monitoring how an L2 system is doing is one of the things humans are most terrible at.

All this changes if the car is able to safely get off the road and out of harm’s way on its own if something goes wrong. That’s the key needed to go beyond Level 2, and that’s the point where, yes, you can sleep or watch a movie or do anything you want while being driven by your robot car.

The sticky thing is that this is a really, really difficult problem to solve because you’re asking the car to drive to safety when it’s having to do that in the first place precisely because it can no longer drive safely.

This isn’t the same as when you or I are driving and, say, we get a flat or lose a radiator hose or some other mechanical issue. In that case, our senses and brains still work and we can usually guide the car to safety. The situations that require an AV to be able to get to safety if the system is compromised are more like if you suddenly lost your vision or had a stroke while driving, situations that usually end in nasty wrecks.

In fact, I do not think this problem is even possible to solve just in the car’s systems alone; to make this actually work I think will require a larger systems-level solution and some infrastructure changes, too.

That said, I think for city driving, at lower speeds and in urban areas, the problem is easier to solve, an unusual situation for the autonomous-driving world. An AV that loses sensor functionality on a busy 35 mph road can pretty much just put on its hazard lamps and come to a controlled stop.

Sure, it’ll piss off everyone around the car, and people will honk and yell obscenities, but really it’s no different than all the times one of my shitboxes quit on me in the middle of downtown LA. Speeds are low enough that the car can safely just come to a controlled stop in most situations.

The real problem is with what is likely to be the more common use case for AVs: high-speed highway travel. If an AV’s ability to understand its environment is compromised at 75 mph or so, then it’s a completely different situation. At those speeds, with other traffic, the window for a vehicle to get to safety is measured in mere seconds, and simply coming to a controlled stop in a lane is not a viable or safe option.

This is the situation I want to address, so let’s walk through how a compromised AV can safely get out of harm’s way with the assistance of some hopefully (relatively) inexpensive infrastructure additions. Let’s go step-by-step:

1. The AV experiences a failure

Illustration: Jason Torchinsky

Okay, so here’s our weird-looking AV on the highway, in the right lane. Part of this plan would be to keep AVs to the right two lanes as often as we can, to keep their distance to the shoulder area as minimal as possible. Really, they’d be sort of like HOV lanes.

As an AV drives, as much sensor data as possible should be tucked into a buffer. That way, when the car’s ability to perceive its environment fails, it will have a buffer of a least a few seconds of the conditions immediately prior to the failure.

So what should it do with this limited data?

2. Target the shoulder-based “sensor posts”

Illustration: Jason Torchinsky

See those little posts on the shoulder there? Those are “sensor posts” and they’re really pretty simple: flexible plastic posts with a bottom spring-loaded hinge (so you can run right over them with no real damage to your car) that contain a trip sensor of some kind. It actuates when it detects a car touching/passing it on the shoulder side.

I’m thinking these posts could very likely be solar-powered, with batteries for nighttime use; all the components are fairly low-power things, all pretty much the same stuff as in your cell phone, minus the screen and speakers.

Once the AV knows it’s experiencing a failure, it will immediately begin an attempt to communicate with the sensor posts near it.

Illustration: Jason Torchinsky

These posts have a small computer (think Raspberry Pi/Arduino-grade thing) with a cell connection that reports to the main server in charge of a given stretch of highway and reports if a vehicle has entered the shoulder and the GPS location of the post that was triggered by the vehicle. This small packet of data (post sensor tripped by vehicle, post location) would be uploaded to the server in charge of that stretch of highway.

The posts also send out constant signals to the passing AVs (and other vehicles, if so equipped) that basically just state if they’ve been tripped or not; that is, is there a car by them. If no disabled car is nearby, they send an AVAILABLE signal to the passing car; if a car is there (AV or a normal, human-driven car that is having an issue and pulled off the road) then they declare themselves OCCUPIED.

If AVAILABLE, the post handshakes with the disabled AV still on the highway and coordinates with the most recent data (or, if the GPS is still on-line, the current location data) to plot the very short course off the highway lane to a safe spot on the shoulder.

If OCCUPIED, the car continues until an available post is reached, which, if each post is roughly two car lengths — roughly 30 feet — apart shouldn’t be long at all. The AV then pulls off the road to the post (again, using steering/GPS data from the post), which it either drives harmlessly over or onto the ground strip, which trips the OCCUPIED sensor.

If the car’s speed (which is sent to the post in the handshake stage) is such that the estimated stopping distance will take several post-positions, the post computes how many clear posts ahead will be needed, and only OK’s the AV to pull off when those conditions are met. Again, this shouldn’t normally be much of an issue, but you never know.

Once the server gets the packet telling that a vehicle has entered the shoulder and is stopped at a given location (via GPS data from the car to the nearest post) then we know the car is at least out of the stream of highway traffic, and we can move to the next step.

3. Get the car to the nearest safe paddock

Illustration: Jason Torchinsky

Here’s where the other significant infrastructure component enters: simple parking lots, called paddocks, that are located along a highway every, what, 10 or 20 miles. They’re off the highway enough to be safe (think rest-stop type locations) and with frontage road access for support and/or tow vehicles.

To get to these paddocks, the AV that’s waiting by its nearby sensor post is sent a signal via the post that contains specific GPS route data to follow. The route is very simple, straight down the shoulder area, at lowish speeds (think 35 mph max), hazard lights on and interrupted only if any of the posts en route issue an OCCUPIED status, in which case the AV waits until the OCCUPIED status becomes AVAILABLE again, likely by the vehicle in question moving ahead as well.

The GPS route will take the disabled AV to the nearest paddock, where it will enter and park. Its final location is sent to the local area server, which ideally will be able to send an alert to the car’s registered owner with the GPS location information.

On its trip down the shoulder to the closest paddock, no sensors will be needed to work in the disabled AV. All it has to still be able to do is drive based on GPS data, and the path will be known to be clear because the sensor posts will report any obstructions.

Also, the shoulders and paddock area will be able to be monitored, and if an AV is mechanically disabled (or a normal combustion car) and can’t drive at all, a tow vehicle will be dispatched to clear the obstruction. We can count on the AV to maybe send a signal that it’s immobile; for other cars, other monitoring systems like cameras (perhaps on the sensor posts?) will be used to keep the shoulders clear.

At this point, hopefully the people in the car will have awakened or finished their movie or have sobered up and can contact someone to get their impaired AV fixed or towed. Their little misadventure will have ended without any humans in the car having to do a damn thing.

Now, this setup would allow for, effectively, Level 4 autonomy, since it will work only in areas with the proper infrastructure.

I tried to limit the infrastructure needs as best I could imagine: The posts, while numerous, should be cheap and able to be very easily installed and/or replaced. We’ve been sticking posts on the sides of highways for decades.

The paddocks are just parking lots, with entry via the shoulders of the roads, and at least one road to get in and out.

There’s also the need for servers that handle various sections of road and manage the posts and paddocks, but those can be pretty much anywhere. There’s software that would need to be developed too, but by no means nothing all that complex or beyond what’s currently being done.

There would also need to be agreement and standards from AV makers to work with this sort of system. Cars would have to communicate with the sensor posts using a universal protocol and take the low-speed GPS driving directions with similar agreed-upon standards.

Areas without decent shoulders off the highway will be more difficult, and perhaps shoulder areas would need to be built, or lanes repurposed. The benefits of the system are that the impaired car itself is required to manage on its own for only a few seconds in most circumstances. And the distance to a safe zone where it can be controlled is extremely limited — the width of two lanes of traffic at most.

I’m sure there are issues I haven’t adequately figured out — times when the AV is confused but doesn’t realize it’s having an issue, like it not properly interpreting highway markings. If it doesn’t know it’s failing, there’s not a lot we can do until things go really wrong.

In all the situations where there are equipment/sensor/weather and other failure situations, though, I think this system could provide just enough support to get a highway, high-speed AV out of harm’s way and on a path to recovery, like an at-risk teen or something.

As far as I’ve been able to tell, no major player seems to have any better system proposed as yet, so why not give this some thought? I’ll check the comments for your ideas/angry attacks. Maybe, just maybe, it’ll get us closer to real autonomy, not just very full-featured Level 2 systems like Autopilot that still require you to be ready to take over as if you’re being driven by a narcoleptic chauffeur.

So, for all you Tesla stans who keep telling me I need to stop being so negative, here, I’m trying. Get Elon to focus on tedious shit like this instead of adding features to Autopilot, and we’ll be getting somewhere.