Is this safe?
Naturally, I should have paid attention and kept my hands on the wheel. However, everything from the rental guy to the advertising and the car interior suggests: relax, we’ll handle it for you. It is like a permanent lullaby on a long ride. But how can I be sure this car is safe when I drive away with it?
This is where the EU HEADSTART project comes in. It develops a consistent way to test and collect safety evidence for a vehicle (see previous blog by Nicolas Wagener). This means collecting situations (or ‘scenarios’) and defining what is relevant for this car. It means limiting the automated functions to a well-defined and well-tested type of road situation (the ‘operational domain’). It also means finding the right combination of various types of tests and then testing until you are very sure. And agreeing which extremely rare cases (‘edge cases’) can be ignored.
Two topics are interesting for today: what is relevant for the safety of a given automated car? And: how about communicating cars?
What is relevant?
The intuitive answer would be: anything the car could possibly meet on the street in its life. This does not fly with today’s assisted driving functions since they are simply not designed to handle all this. Also: ‘in its life’ means a new situation might occur tomorrow that was not anticipated in the design. The concept of the ‘Operational design domain’ (ODD) is very useful here (SAE J3016): it describes the operating conditions under which the car is designed to work. This can be road type, weather, etc. Essentially you put a fence around the subset of real-world driving that your vehicle can handle and define everything outside irrelevant. To paraphrase: everything within the ODD is not odd but relevant.
So now the new challenge is to handle everything the car meets in the ODD. By driving around and collecting scenarios we can smoothly collect common scenarios. But knowing this car might be bought by millions of people who each drive it at least 50 000 km, surely our users will find more scenarios than our data collection team can. Also, it is not ethical to ask our test drivers to look for critical situations on the road. So we are still left with a snag: how to systematically collect rare scenarios? And how rare should a situation be to become irrelevant? Should we design a car that can handle a bicyclist that ends up on the highway? And an airplane that lands on the highway?
Since serious accidents occur around once every 2.5 million km of driving for Germany (Wachenfeld, 2016), we need to drive a lot more to prove that safety is improved. This takes lot of test driving or a fresh approach. Using the sensors of high end production vehicles is the most powerful and consistent approach but can only be done with the previous generation of CAD functionality that is already type approved… Current approaches focus on combining and synthesizing risk triggers into edge cases – on the edge of what is possible and relevant.
Once you have all the scenarios collected, you are ready, right? Well, you need to keep collecting as their parameters change over time – with the introduction of more CAD vehicles the balance between conventional and automated driving changes. The numbers of scenarios are so high that you will need to test many of them in simulation for safety and cost effectiveness. And for complete safety assessment or type approval you need more. Sensor tests on component level are crucial and can be done in the lab. Emergency braking is most convincingly and safely tested on a test track. Human factors can be tested in a simulator, but you really want to validate all this by test driving on the public road. So, we end up with a well-balanced mix, including public road testing. Real-world traffic does not nicely stick to one isolated scenario at a time: a critical situation is usually a soup of many ingredients spiced with all the noisy distractions of the road.
Cars start to become more talkative: transmitting traffic jam information to navigation systems and receiving warnings from infrastructure on advised speed and roadblocks. Real-time communication between vehicles allows for fast cooperation and possibly even higher levels of safety. This raises the question how to test the integrity of the data and robustness of the wireless link. After all, you now rely on external information from a variety of infrastructure sources and multi-brand vehicles. Communication standards, interoperability and trust relations are crucial here.
HEADSTART looks at what can go wrong (failure modes) and includes communication events in scenarios to test this on the level of interaction between vehicles. Scenarios are important as a missed message (for example) may have a much higher impact in an urban congested scenario than in a nightly highway scenario.
If you want to test vehicle-to-vehicle (V2V) communication including two or more vehicles (like platooning), you need to release or approve each vehicle brand and model in the configuration, respecting its role (leading, following, etc.) and the compliance with the applicable V2V protocols.
This is the moment to give praise to a mostly invisible expertise in automotive: the test engineers, be it on the test track or with a simulator. If we want to find out the safety of a vehicle, safety engineers first analyse hazards and risks before the test engineer starts. This includes vehicle failures and also risks during driving when no technical failures occur (operational safety). The appropriate response to such hazards is then specified, modelled and designed. Though based on a massive body of expertise, this is still paperwork. If we want proof, we need a test engineer and a test, be it virtual of physical.
Figure 2 Chipping away at the parameter distributions
So: what to test? From the scenarios collected before, we can learn the statistical variety of all kind of scenario aspects and the corresponding parameter distributions. Combined with edge cases, we can sample a set of relevant tests that give sufficient insight into the behaviour of the tested vehicle to make a release decision. By patiently going from test to test or by running a batch of many simulations, the test engineer chips away parts of the parameter distributions until nothing is left uncovered. This hard and noble work is what gives us evidence of safety.
This blog post was produced by Sytze Kalisvaart.
Sytze Kalisvaart is a senior project manager at TNO Integrated Vehicle Safety. He is responsible for scenario database projects, the StreetWise methodology and pipeline. He has a background in system design, usability and safety in medical and automotive.