Elon - Musketeers

[Red_Syns]

I disagree with your assertion that because the owner brought the car, it is responsible for the car's actions. I contend that, as long as the car was behaving in accordance with all laws governing the situation, the car is responsible for minimizing loss of life in a manner that is strictly controllable: leaving a roadway in a city or city-like area is neither safe nor predictable, for instance, and should only be performed when the vehicle is certain of what reactions will occur.

The responsibility lies with the party at fault for the situation: assuming the vehicle was behaving as it ought to prior to the start of the incident, the other party is the responsible party.

[Sab1e]

As such, it must have deliberate actions programmed

Must it? I would imagine the primary program would be "attempt to avoid humans using every system I have to the full".

If it still hits one, then chances are a human driver would have done too.

[Red_Syns]

What if it is doing 75 MPH down a highway with four occupants, and a person steps out in front of it from behind an obstacle, and the only route of escape is to cross into oncoming traffic, run another vehicle off the road, or drive into a ditch? At that sort of speed, a collision with any obstacle is likely to severely injure/kill all of the occupants. Was making every effort to avoid the one person, who was in the wrong for entering traffic, the right decision?

[Sab1e]

I really do see where you are coming from, but it's still the same situation that would be faced by a human driver.
Either braking and trying to correct course works, or it doesn't, and there's some sort of accident. Who's controlling it doesn't matter.

I would imagine that as long as the car makes every effort to avoid hitting anything, it will have reacted much the same way as a person. If the situation is really so dire that someone is going to get hurt, then it's not a case of who. It's just the result of someone doing something wrong (which is unlikely to be the car, since it can't "choose" to be unpredictable in the same way).

[Red_Syns]

But we're not discussing what makes it behave like a human. We are, after all, attempting to exceed the abilities of humans, not just mimic them.

Of course braking and controlling the slide is going to happen: at the very least, it drastically increased all chances of survival without changing the situation. But if the situation can be analyzed and objectively found, using a fixed series of logical steps, to have a superior outcome, isn't that the better choice?

The point of this thread is to argue what outcomes are the most favorable, as situations and the reactions can be decided in advance of an incident. Well, it is actually an Elon Musk thread, but this problem is one they are dealing with right now as one of the first makers of automated vehicles.

[Sab1e]

I do love a good debate, and bow to you sir for playing along so nicely! Now I must retreat before I start making sleepy-posts, which is rarely wise.

Whatever happens, it's clear automation will continue and it will be fascinating to watch the rise (and possibly many falls) of the technology! I have voiced my thoughts to be pondered or ignored and now bid you all goodnight

[Topperfalkon]

If a bicyclist suddenly emerges from a blind corner onto a street illegally

The car approaches blind corners at a safe speed and at a great enough distance to give the car and oncoming cyclist adequate time to react.

Maybe a tire blows out, causing the vehicle to swerve unpredictably

There are several solutions to this. The first would be to switch to airless tyres, but they're still prototypes at the moment.

The second is to use road sensors to avoid sharp debris and use embedded sensors in the tyre wall as well as other suspension-based metrics to monitor tyre health, and pull the vehicle out of service when those metrics reach unsafe levels

Finally, the car should model blowouts and the reaction to them on different road grades and conditions so that it can orchestrate a predicted response to a blowout that couldn't otherwise be detected (like a lucky gunshot, I guess).

First principles: the operator brought the car into the situation and is responsible for its actions.

But we're talking about automated vehicles, so the car brought itself into the situation. Logic follows that an automated car will always avoid scenarios it cannot exercise adequate control over. Automated cars must, by necessity, be incredibly paranoid about both their own state and the state of their surroundings.

So if there is a situation where the car must make the decision, the engineers must program it to favor the life of others

Good luck convincing people to buy cars they know will murder them.

Extreme events are things like a bridge collapse. It's outside the limits of what a car can do anything about.

Events that include complete loss of motor control are somewhat out of scope. If a bridge collapses under any vehicle, there is precious little it can do than offer as much protection to its occupants. Obviously, an automated car could try and find a less risky route...

Any automated car that ends up killing someone in a situation it had control over is buggy.

What if it is doing 75 MPH down a highway with four occupants, and a person steps out in front of it from behind an obstacle,

The car detects the obstacle and slows down on approach. The car then moves past the obstacle at a speed it is able to cope with any further changes to the environment.

[Red_Syns]

Those are not realistic solutions to unexpected situations. http://www.autos.com/driving-and-safety/stopping-distances-at-different-speeds

But since you insist on trying to convince me that cars should safely approach every corner, how about this: it's raining, the car has six passengers, is doing five under for safety purposes, and the vehicle approaching from the oncoming direction is not automated. Or it is, this situation won't really matter. The sidewalk is sparsely populated with only a couple people per unit distance it would take the car to stop. Suddenly, the oncoming car hits a puddle and hydroplanes, drifting into the automated vehicle's lane as it rotates to occupy most of both lanes. There are two options available to the automated vehicle: brake and brace for impact, or jump the curb and potentially kill two pedestrians. The combined speed of the two vehicles will very likely leave three or more people dead from the accident, and the collision may even injure/kill pedestrians on the sidewalk anyway.

What does the car do?

Now that we're done saying things like "the car just had to avoid all possible unpredictable situations", may we resume talking about an actual reality? The simple fact is that unpredictable situations will arise, regardless of the mode of travel. Trains derail, airplanes lose power, ships hit rogue waves and snap in two. You simply cannot detect everything in your surroundings, and there will never be a world where any and all accidents are avoidable.

[Red_Syns]

But we're talking about automated vehicles, so the car brought itself into the situation. Logic follows that an automated car will always avoid scenarios it cannot exercise adequate control over. Automated cars must, by necessity, be incredibly paranoid about both their own state and the state of their surroundings

Actually, automated vehicles don't have to do any of this, you just want them to. To gain acceptance in the general populace, all an automated vehicle has to do is maintain a casualty/fatality rate equal to or lesser than manually driven vehicles. After factoring in the massive amounts of convenience these vehicles offer, an equal or better rate would be immensely popular.

[Topperfalkon]

To gain acceptance in the general populace, all an automated vehicle has to do is maintain a casualty/fatality rate equal to or lesser than manually driven vehicles

And not intentionally kill the occupants as a default failure mode. Any vehicle that is programmed to intentionally kill its occupants in response to a hazard is going to have a hard time finding sales.

Now that we're done saying things like "the car just had to avoid all possible unpredictable situations", may we resume talking about an actual reality? The simple fact is that unpredictable situations will arise, regardless of the mode of travel. Trains derail, airplanes lose power, ships hit rogue waves and snap in two. You simply cannot detect everything in your surroundings, and there will never be a world where any and all accidents are avoidable.

But we're talking about situations where there is sitll some control available. If there is literally nothing a vehicle can do to prevent injury to the occupants, all it has to do is ensure it is responsible for no further damage.

Back to the particular scenario:

But since you insist on trying to convince me that cars should safely approach every corner, how about this: it's raining, the car has six passengers, is doing five under for safety purposes, and the vehicle approaching from the oncoming direction is not automated. Or it is, this situation won't really matter. The sidewalk is sparsely populated with only a couple people per unit distance it would take the car to stop. Suddenly, the oncoming car hits a puddle and hydroplanes, drifting into the automated vehicle's lane as it rotates to occupy most of both lanes. There are two options available to the automated vehicle: brake and brace for impact, or jump the curb and potentially kill two pedestrians. The combined speed of the two vehicles will very likely leave three or more people dead from the accident, and the collision may even injure/kill pedestrians on the sidewalk anyway.

In this situation it probably would matter, as the automated car would detect the hazard and attempt to negotiate it in a way that would prevent or minimise loss of control. Regardless, assuming the other car isn't automated, the automated car could simply wait for the other vehicle to navigate the obstacle before proceeding. Given the human driver, the automated car also can't predict whether the vehicle will come back under control into either of the lanes, so it may end up killing pedestrians for no good reason, so mounting the pavement isn't going to be a good course of action (unless it's completely vacant, and offers a suitable surface to maintain control. Optimum evasion strategies would require more sophisticated modelling, of course.

TL;DR: In the case where a hazard was initiated by an external actor, the car should undertake an action with the highest probability of a successful evasion without causing further endangerment. It should also try to avoid entering a situation where it might interact with an external actor approaching a hazard, removing the need for evasion.

[Red_Syns]

But we're talking about situations where there is sitll some control available. If there is literally nothing a vehicle can do to prevent injury to the occupants, all it has to do is ensure it is responsible for no further damage.

Actually, we aren't. We're discussing events where the vehicle must choose between the injury and/or death of the non-occupants, or the lesser chance of injury and/or death of the occupants: it's a sort of trolley dilemma. Some argue the occupants are safer than the non-occupants, and therefore the vehicle should always default to the most safe alternative for them. Others argue that the car is the property of the owner, and therefore should always default to safety of the occupants. Personally, I argue in favor of the safety of the occupants when there is no way to remove risk to uninvolved third parties, and safety of the larger party when there is.

Traffic will not and should not come to a screeching halt just because it is raining and there are puddles on the ground. I don't know if you don't drive/do very little of it, but your statements so far show a great deal of naivety with regards to traffic safety. Slowing down to speeds safe enough to immediately stop when approaching a blind corner is actually incredibly dangerous for the drivers behind you. Blind hills are the same way. Until every vehicle is automated, there will always be a manual driver who won't benefit from systems that let cars talk to each other.

From an economic standpoint, congestion would get infinitely worse, which would begin to impact the economy. Driving would rapidly begin to take FAR longer than it's worth, and I shudder to imagine the number of times traffic in cities comes to an even greater halt as people realize cars will LET them jaywalk safely.

Programming a car to avoid perfectly normal situations is absurd and will stymie advancement of a technology that is already demonstrating itself to be safer than manual driving, and it's still in a prototype phase. Everything about driving has an inherent built-in risk. Automated cars should not avoid any situations a rational person would not themselves avoid: they should, however, be programmed with the superior response.

Anyway, that's the last I'm going to spend countering "it should just avoid the situation" arguments. If the situation is reasonably avoided, then there is no need to worry about how to respond to it.

[Topperfalkon]

Slowing down to speeds safe enough to immediately stop when approaching a blind corner is actually incredibly dangerous for the drivers behind you.

I'm not saying that the car should slam on the brakes the moment it reaches a corner it's unsure about. This is, after all, the way humans should be driving cars in hazardous environments.

From an economic standpoint, congestion would get infinitely worse, which would begin to impact the economy. Driving would rapidly begin to take FAR longer than it's worth, and I shudder to imagine the number of times traffic in cities comes to an even greater halt as people realize cars will LET them jaywalk safely.

This is arguably a civic planning issue. A place with large footfall should typically see less cars, and arterial roads that serve primarily to move traffic should typically have less footfall pedestrian barriers and, preferably, grade seperated crossings. Cities that don't do this well, like London, already have traffic problems with or without automated cars. Ultimately though, the utility of automation is amplified when a greater percentage of cars are fully automated.

Programming a car to avoid perfectly normal situations is absurd and will stymie advancement of a technology that is already demonstrating itself to be safer than manual driving, and it's still in a prototype phase. Everything about driving has an inherent built-in risk. Automated cars should not avoid any situations a rational person would not themselves avoid: they should, however, be programmed with the superior response.

I'm not suggesting we program cars to avoid normal situations, just that they operate within their level of capability and not exceed it. I don't think the trolley problem is particularly useful in these discussions because it assumes a binary choice and a predetermined output, when the reality of any such situation is that the vehicle will have far more data to model outcomes. The question isn't about who to kill, it's about how much degree of certainty you need to have that a death is unavoidable before you accept intentionally causing someone's death is an acceptable course of action.

As far as I can tell, Google's approach so far has been to work from the paranoid machine upwards, whereas the likes of Tesla have opted for deferred responsibility instead. I can see how it might be harder for Tesla to move past it's current position to a fully automated machine, but the theory behind Google's approach is that their platform's capability will increase as it learns.

[Skyentist]

Definitely some of my favorite gifs of all time:

[Crayfish]

The sooner we realise that automated cars are not going to be moral, the quicker we will be able to get on with saving millions of lives.

An automated car should be treated exactly the same as a personal tram car. People are completely happy with tram cars in their cities and don't blame them if there is an accident. The worst thing you can do is to attempt to make the car make moral choices. An automated car should not ride up onto the curb ever deliberately, even to avoid an accident.

Cars will always be dangerous, lets treat them that way and stop pretending that morals should come into it. Just make them consistent in their actions and make sure everybody understands those simple rules.

Mine would be:

1. Automation will never break the accepted rules of the road.
2. The vehicle will avoid all collisions it can without breaking rule 1.
3. When a collision cannot be avoided it will choose the path that results in the lowest impact speed.

The third rule is important because attempting to count bodies and minimise casualties gets far too complicated when you also need to bear in mind collision speed and vulnerability. That's just something even a neural net won't be able to do for a long time. Keep the rule simple and consistent and the net result will be a benefit to all.

[TARS]

Automation will never break the accepted rules of the road.
The vehicle will avoid all collisions it can without breaking rule 1.
When a collision cannot be avoided it will choose the path that results in the lowest impact speed.

I like the sentiment, but the rules do not completely work, or you need to clarify them more.
What about the following scenario:

• A man crosses the road, his traffic light was red (he breaks the law)
• An AI driver approaches at a speed where he either has to hit the man, or crash into a tree on the side of the road, or crash into traffic on the other side

What does the AI do now?

• Hit the man, because hes not supposed to be there
• Hit the tree and kill the passenger
• Hit traffic on the other side and kill the passengers of both cars

Ultimately you can't fully avoid situations where a decision has to be made whom to save.

I think your answer will be "the path that results in the lowest impact speed", but what if that doesn't matter for the outcome? (death in all scenarios)
And how would killing five people in a car traveling at 50mph better than killing one person traveling at 70mph...

[Crayfish]

I think your answer will be "the path that results in the lowest impact speed", but what if that doesn't matter for the outcome? (death in all scenarios)

Well it wouldn't hit the tree as the primary rule is that cars never automatically leave the road. That's very important as a contract with pedestrians.

Out of the car that broke the law and the car coming the other way it is far more likely to choose the law breaking car as that is much more likely to result in the lowest impact speed.

And how would killing five people in a car traveling at 50mph better than killing one person traveling at 70mph...

We are fifty years away at least from a computer able to make such a distinction. All we can say is that an AI driver will on average do better than human drivers by following those three simple rules. There will be isolated cases where a human driver would have saved a life that an driverless car wouldn't but the net effect will be way positive.

[Playbenni]

This whole discussion is redundant in a way. Ais will have to make decisions based on morality/law eventually.
I would argue that there are already Ais that make life and death decisions today.
For example the software that decides which organs from a donor go to which sick person. That Ai solely decides who reveives what on urgency, compatibility, time waited for an organ etc.
Based on that we already have an Ai that effectivly discriminates against ethnicity, for example in a country that is predominatly of european ethnicity an african or asian person is less likely to receive a transplant because things like the distribution of blood types differs among ethnicities. The most obvious fact that this ai already makes a moral decision is the factor of time waited for a transplant in making the decision. A person who waited longer is more likely to receive a organ than another person who might be a better fit if they are at the same urgency. The moral here is that the person shouldn"t wait much longer because that would mean continued suffering. Thats a moral decision right here. A better fit is basically ignored because of that.

As usuall german politicians/lawmakers ignored reality and instead opted to do the "politically correct" thing.

[Crayfish]

1. Automation will never break the accepted rules of the road.
2. The vehicle will avoid all collisions it can without breaking rule 1.
3. When a collision cannot be avoided it will choose the path that results in the lowest impact speed.

At the risk of opening a morality can of worms again I might amend rule 3 to "the least lethal impact speed". Although sensors and computers will not be capable of counting passengers in vehicles for a long time, they should be able to distinguish the difference between pedestrians, cyclists, cars and trucks. The calculation of least lethal impact speed could certainly be weighted by the classification of the object that would be hit.

[cybercritic]

I see you are going with Asimov's approach, which is very nice, however Asimov spent a lot of his time showing how his rules are flawed.

[Crayfish]

All rules are flawed, just like all systems of government are flawed. Our only way forward is to choose the least flawed option.