Research / Transport and mobility / WP6b
WP6b · Transport and mobility
Automated vehicle safety
In the project plan: Smart & safe mobility - the safety envelope
Quiet electric vehicles are harder for pedestrians, cyclists and elderly road users to notice. This work package developed the concept of an autonomous safety shell around the vehicle that anticipates what other road users will do, using deep learning trained mostly on safe situations, and acts or warns before an accident happens.
✎ draftPrincipal investigators
PhD researchers
Caspar HanselaarPhd candidate | autonomous vehicles · Eindhoven University of Technology
Kurt StollePhd candidate | ai for the improvement of autonomous vehicles · Eindhoven University of Technology
Saba SiadatiPhd candidate | smart logistics hubs and mobility as a service · Eindhoven University of Technology
Other researchers
- Gijs Dubbelman, Associate Professor
Partners
Also involved: NXP, SWOV, De Verkeersonderneming
About this work package
Objectives
- The EV comes with quite some benefits w.r.t environment, lower to zero carbon emissions and zero
urban air pollution. The objective is to eliminate negative impact on traffic safety, due to the low
noise emissions, causing less perception of approaching vehicles by other vulnerable road users,
like elderly, pedestrians and/or cyclists. - To address the challenges from 1) develop and introduce a safety shell around the vehicle.
- Obtain a good understanding of an autonomous safety shell around a car, which autonomously
maximally prevents accidents or minimizes accident-impact by active anticipating actuation within
the limits of the vehicles capabilities, and then continues-gives back control to the driver. - Give insight in the ways the safety shell may activate automated, advisory or warning signals to
the EV-driver or vulnerable other road users, by auditive, tactile, visible, or vehicle control
measures.
Activities
- Sensor data interpretation and fusion techniques for anticipatory safety systems.
a. Safe driving requires anticipating on possible future events. Deep learning has, so far, solved
object detection and classification. The next step is to automatically hypothesize what these
objects, e.g. vehicles and vulnerable road users, will be doing in the future. In complex safety
scenarios, this requires a holistic scene understanding approach that is able to detect and
interpret (inter-) object behavioral cues embedded deep in the spatio-temporal domain.
b. This research line focusses on automatically learning these behavioral cues from big data using
semi-supervised techniques. The innovation is in the training strategy that takes as input many
positive examples, i.e. safe situations, and only few to none negative examples, i.e. (near)
collision situations.
c. The solution that will be explored is casting this problem into a regression task, where a deep
Generative Adversarial Network (GAN) is trained to i) regress on positive examples, and ii)
learned to generate and extrapolate negative samples.
d. The trained GAN is embedded in the safety shell system, to feed world modeling and motion
planning. - World Modeling and Motion planning for EV Safety Shell
- To define a safety-shell for an autonomous vehicle, understanding and modeling the
context/situation the car is in is crucial.
a. Looking into classes of scenarios and complex situations rather than single use case (e.g. define
the safety shell with respect to. various Road Users)
b. The search of the free space for maneuvering and for fail-safe situations - Determine regions of interest or regions of care (i.e. safety shell), for all possible maneuvers and
incorporate these into motion planning.
a. Investigate with respect to what are these regions parametrized or if they change with context.
b. Which path can we plan using the information of the safety shell, such that no or nearly 0
collisions exit.
Expected output
- At least 6 journal papers describing the results of the safe mobility research (M12x1, M24x1,
M36x2, M48x6). - At least 4 conference papers. Most co-authored with other WPs (M24x2,M48x2).
- A tested concept system in 1 digital twin in 1 of the 5 representative neighborhoods in WP10
- At least 2 workshops in which the safety shell concept is explored with NEON stakeholders
interactively. - At least 1 PhDs will have spent more than 3 months abroad with knowledge institutions or from
abroad hosted by NEON. - At least 5 mentions of Safety shell concept in national newspapers or on television.
Text from the former NEON website (original page).
What the plan set out to do
Promises from the 2019 project plan, not results.
A safety shell concept that prevents accidents or limits their impact by anticipating actuation, then hands control back to the driver.
Sensor interpretation and fusion for anticipatory safety, using generative adversarial networks trained mainly on safe situations.
World modelling and motion planning with regions of care around the vehicle.
Insight into how the shell can warn drivers and vulnerable road users by sound, touch, light or vehicle control.
At least 6 journal papers and 4 conference papers.
A tested concept system in 1 digital twin of 1 of the 5 representative neighbourhoods of WP10.
At least 2 stakeholder workshops on the safety shell and at least 5 mentions in national media.
At least 1 PhD candidate spending more than 3 months abroad, or hosted from abroad.
Publications and talks
- Efficient Multi-Scale Deformable Attention on GPUs (journal-article, 2026)
- Transferable Tracker Configurations for Image-Trained Video Panoptic Segmentation (conference-paper, 2026)
- The Safety Shell: A Multi-Channel Architecture for Enhanced Automated Driving (phd-thesis, 2025)
- Balancing Shared and Task-Specific Representations: A Hybrid Approach to Depth-Aware Video Panoptic Segmentation (conference-paper, 2025)
- Identification of Hazardous Driving Scenarios Using Cross-Channel Safety Performance Indicators (conference-paper, 2025)
- Maintaining Stability During Channel Switching in Multi-Channel Automated Driving Systems (conference-paper, 2025)
- The Safety Shell: An Architecture to Handle Functional Insufficiencies in Automated Driving (journal-article, 2024)
- Evaluation of the Safety Shell Architecture for Automated Driving in a Realistic Simulator (conference-paper, 2024)
- Characterization and Mitigation of Insufficiencies In Automated Driving Systems (conference-paper, 2023)
- Unified Perception: Efficient Depth-Aware Video Panoptic Segmentation with Minimal Annotation Costs (report, 2023)
- Block-Level Surrogate Models for Inference Time Estimation in Hardware Aware Neural Architecture Search (conference-paper, 2022)
- Detection and Mitigation of Performance Limitations and Faults in Autonomous Vehicles The Safety Shell (conference-paper, 2022)
- Safety Shell: Reducing performance-limits in autonomous vehicles (conference-paper, 2022)
- PhD defence: The Safety Shell: A Multi-Channel Architecture for Enhanced Automated Driving (phd-defence, 2025-11-25)

