I know this is sarcasm, but I agree with the point you're making.
This is how I see it: although we can put off having vehicles take over completely, unless there's a concerted effort to "keep humans in the drivers seat", and having the car forcibly pay attention, having a self driving vehicle is just asking the occupants to text, stream, or do their nails (I saw this) while in the car.
When, in the situation arises where in our hypothetical situation a kid jumps in front of the car, and the kid or the occupants die or are injured, it's basically a lose-lose situation for the maker of the car. If the car was supposed to hand control over to the driver, even if the driver was 'paying attention', the people will demand why there wasn't a program to avoid the accident. If the car had a sub-routine that kicked in, then people will demand why the program was so bad, or why there was even one in the first place. The reason why we or they would bother with self-driving cars is the fact that overall, they are simply better than us at driving, which would be reflected with a statistically significant decrease in overall accidents.
IMO, if self driving cars take over to compensate for our own inability to pay attention, we as consumers are going to rely on these self driving cars more and more to get us from a to b. Thus, the demand for less human input is going to be present by the consumers to the car manufacturers until we can do our preferred car activity without worrying where we are going. This is probably many years off, but it's what I believe is going to happen.
But for the moment, yes, minimizing the risk is the best choice.
Here's a video of the Tesla Autopilot in action: