Helios De Rosario Martínez1,2, José S. Solaz Sanahuja1, David Moro Pérez1, Paulo Gameiro3, Wilder Eduardo Castellano Hernández1, Alfredo Ballester Fernández1, Andrés Soler Valero1, Carlos V. García Molina1
1 INSTITUTO DE BIOMECÁNICA DE VALENCIA
2 HEALTHCARE TECHNOLOGY GROUP AT IBV, BIOMEDICAL RESEARCH NETWORKING CENTER IN BIOENGINEERING, BIOMATERIALS AND NANOMEDICINE (CIBER-BBN)
3 DEVELOPMENT MANAGER AT BORGSTENA GROUP PORTUGAL
HARKEN. Heart and respiration unobtrusive sensors integrated in the vehicle
The HARKEN project gathers a consortium of European research centers and enterprises that produce vehicle components, smart materials, and sensors for biomonitoring, to create a physiological monitor integrated in the car. This system is in constant contact with the driver’s body through the car seat cover and the safety belt, and it monitors the physiological, mechanical activity related to respiration and the cardiac cycle. Redundant measures of vibrations and artifacts that may distort these signals are used to improve their quality by means of adaptive filters, programmed in a signal processing unit.
Introduction
Traffic accidents caused by fatigue are a very significant social and financial problem. In 2010, there were around 31,000 deaths in Europe caused by this type of accident (OECD 2012) and it is estimated that driver fatigue is the cause of between 20% and 35% of all accidents leading to serious injuries (Hell et al., 1997, Horne and Reyner, 2000, Sagberg, 1999).
To tackle this serious problem, governments have devoted a great deal of resources to advertising campaigns, improving infrastructure and changing the legal and regulatory frameworks, while the automotive industry strives to develop technological solutions that detect and prevent driver drowsiness. There are already some car models with systems that monitor driver behavior (patterns of the steering and acceleration of the vehicle) and assess when the driving style could indicate fatigue or distraction.
However, behavior on its own may underestimate the extent to which the driver loses the ability to react (Lal and Craig, 2001), so the industry is searching for more reliable solutions based on directly monitoring psycho-physiological variables. Some of the most important of these relate to the activity of the nervous system, such as heart rate and breathing, which can be measured these days using portable devices and even systems built into clothing.
The main challenge is measuring these signals in a way which is completely “invisible” to the driver, without them having to wear special devices or clothing. This is the reason for forming the HARKEN project consortium, with the aim of creating a non-invasive monitor using intelligent materials, which forms part of the vehicle’s components.
The structure of the companies and research centers that make up the HARKEN consortium is shown in figure 1. It is a group of small and medium-sized companies (SMEs) that will manufacture the product and a few technological research and development centers (CIDT) that will carry out the research needed, along with the company that will market the product.
Figure 1. Structure of the HARKEN development chain.
Product Concept
The solution developed at HARKEN is a non-invasive system of sensors that measures cardiac activity and breathing, which is integrated into the seat cover and seat belt of the vehicle. Heart rate and the measurements of its variability are good indicators of people’s concentration and mental activity, whilst the reduction and intensity of the breathing rate is correlated with increased fatigue (Lal and Craig, 2001, Milosevic, 2010, Hadjileontiadis, 2006).
The HARKEN system detects the mechanical effect of these physiological actions, and filters out noise and artifacts that are added while driving (Figure 2). This system adds various innovations to those that exist currently, resolving their major limitations. Firstly, it replaces the electrodes that are conventionally used to monitor physiological signals with “intelligent” materials, composed of combinations of fibers and fabrics with electrical properties, integrated into the standard textiles used for the seat and belt, although this imposes certain conditions on the type of signals that may be measured.
Figure 2. Outline of the HARKEN concept.
El concepto para el cinturón de seguridad es tecnológicamente semejante al de los monitores de pletismografía que se usan normalmente en el entorno médico para medir la respiración, a través de la acción biomecánica transmitida a un par de bandas torácicas y abdominales. Por otro lado, la ubicación de las bandas del cinturón sobre el conductor y la presión que éste ejerce sobre su cuerpo son normalmente distintos de las condiciones usadas para la sensorización clínica. Por eso, además de la integración del material sensible en la estructura del cinturón, ha de abordarse el reto de adaptar el cinturón y sus anclajes para mejorar dichas condiciones. Esto se ha logrado a través de un estudio antropométrico, que determina la ubicación óptima de los sensores, y el diseño de unos tensores que ayudarán a mejorar el control de la ubicación de los sensores.
The concept for the seat belt is technologically similar to that of the plethysmography monitors normally used in the medical environment to measure breathing, through the biomechanical action transmitted to a pair of thoracic and abdominal bands. Furthermore, the location of the belt bands on the driver and the pressure the belt exerts on the person’s body are usually different from the conditions used by clinical sensor systems. Therefore, in addition to the integration of the sensitive material into the belt structure, there is a need to overcome the challenge of adapting the belt and its anchorages to improve these conditions. This has been achieved through an anthropometric study, which determines the optimal location of the sensors, and the design of some tighteners that will help improve control over the location of the sensors.
With respect to the solution for the seat, this differs from the majority of physiological sensors because the areas of the body in direct contact with the system are not those closest to the organs whose activity is being monitored (heart, lungs and abdomen). However, the materials are optimized to achieve their greatest sensitivity with the pressure exerted by the driver’s body on these areas.
In terms of the physiological variables themselves, and in particular that of cardiac activity, although the most common test for recording this in the medical field is the electrocardiogram (ECG), HARKEN focuses on the mechanical measurement of the blood flow (ballistocardiogram, BCG), which is observable in everyday environments without direct contact with the skin (Baek et al., 2012). Although BCG signals do not present the specific patterns of the ECG waves used for clinical diagnosis, they do show the peaks associated with the heartbeat, allowing them to easily measure parameters such as heart rate and its variability.
Finally, the use of redundant measures of physiological and dynamic signals from the body and environment makes it possible to use data merging strategies to improve the reliability of the output signal. This is achieved by employing adaptive filters, which take into account the correlation between these auxiliary signals and sources of noise and artifacts.
Configuration and results of the HARKEN System
The HARKEN system’s sensors were manufactured using a fabric that is sensitive to the local pressure exerted by the driver’s body, making it possible to detect small pressure differences created through the cardio-respiratory action, which may be less than 1 g/cm2 in the case of the heartbeat.
In order to detect these signals reliably, the areas of interest in a seatbelt and a seat (Figure 3) were defined on the basis of the levels and uniformity of pressure sustained during driving tests.
Figure 3. Areas and points of interest in the sensorized seat.
The processing unit was designed by configuring the input channels and filters to capture physiological signals from the HARKEN sensors. A prototype of this system was installed in the cabin of a driving simulator to be validated through testing in use (Figure 4).
Figure 4. HARKEN system installed in the vehicle.
Figure 5 shows examples of the signals recorded during these tests in use. Graph 5a shows a fragment of the signal in one of the sensors (thoracic region of the seatbelt) during a controlled test without noise, where a respiratory waveform pattern is clearly visible, with cycles of between 3 and 4 seconds. Figure 5b corresponds to a high-pass filter of the previous signal, in which the higher frequency peaks of the heart rate can be observed (cycles slightly longer than 1 second, corresponding to around 55 beats per minute).
Furthermore, figure 5c shows the signal from this sensor in a fragment during actual use (driving in the simulator), including distortions due to the user’s movement (approximately the first 150 seconds). Graph 5d shows the background noise, measured with an accelerometer located on the seatbelt, after integrating the signal to estimate the user’s torso movements. As can be seen, the disturbances of the noisy phase in 5c correspond to the greatest movements in 5d, which will be used to “clean” the signal.
Figure 5. (a) Signal from the HARKEN sensor without noise (respiratory wave);
(b) signal without noise with low frequencies filtered (heart rate peaks); (c) signal with noise
due to movement of the user; (d) background signal of the user’s torso movement.
This operation is performed with the adaptive filter, whose operation is shown in the interface in figure 6. Three graphs are shown in the left section: from top to bottom, the contaminated signal (in red), the background noise (in green) and the filtered signal (in blue). On the right we see the spectrograms for the signal before (above) and after (below) the filtering process. It can be seen that, although the signal spectrum to be obtained (the regular wave of the heartbeat) is superimposed by the noise, the adaptive filter is capable of cancelling out this noise and leaving the original signal clean.
Figure 6. Adaptive filter interface programmed in Matlab.
Conclusion
The HARKEN system captures heart rate and breathing in a completely non-invasive way. The test results show its potential as a tool that can be built into vehicles in the near future. Once the functional requirements have been achieved, work will focus on improving the industrialization of the prototypes developed and complying with the standards of the automotive industry, making it the technological foundation for future driver drowsiness detectors.
Acknowledgements
This work has been funded by the Seventh Framework Program of the European Union (FP7/2007-2013) through funding agreement no. 286265.
Bibliography
Baek, H.J., Chung, G. S., Kim, K.K., and Park, K.S., 2012. A Smart Health Monitoring Chair for Nonintrusive Measurement of Biological Signals, IEEE Transactions on Information Technology in Biomedicine, 16(1): 150 -158.
Hadjileontiadis, L.J., 2006. Biosignals and compression standards, in: M-Health. Emerging Mobile Health Systems, edited by R.S.H. Istepanian et al., Berlin: Springer: 277-292.
Hell, W., Langwieder, K., Sporner, A., and Zulley, J., 1997. Driver inattention and other causative factors in fatal highway crashes. Proceedings of the 41st Annual Conference of the Association for the Advancement of Automotive Medicine, Orlando, USA, Nov. 10-11, 1997.
Horne, J., and Reyner, L, 2000. Sleep Related Vehicle Accidents. Sleep Research Laboratory, Loughborough University.
Lal, S.K.L., and Craig, A., 2001. A critical review of the psychophysiology of driver fatigue. Biological Psychology, 55(3): 173–194.
Milosevic, C., 2010. Driver’s fatigue studies. Ergonomics, 40(10): 381-399.
OECD, 2012. Trends in the Transport Sector 1970-2010. Paris: OECD Publishing / International Transport Forum.
Sagberg, F., 1999. Road Accidents Caused by Drivers Falling Asleep. Accident Analysis and Prevention, 31(6): 639–649.
