嵌入式与移动计算

实时分析驾驶行为

Vehicle Steering Detector

有调查显示,交通事故往往是由驾驶员危险驾驶行为所引起的,及时检测这些行为并进行预警对于避免交通事故至关重要。车载诊断系统OBD(On-Board Diagnostic)和高级驾驶辅助系统ADAS(Advanced Driver Assistant System)造价昂贵,因此通常不适用于大众车型。随着智能设备的发展,智能手机的快速发展和强大的内置传感器是解决此问题的新途径。

成果展示

Achievement Display

采集平台的实现

通过无线网络
Android智能移动端采集系统
通过众包技术的采集策略对数据

行驶数据采集

不同车型的车辆
不同类型的智能终端
不同水平的驾驶员

行驶模式识别

Hadoop云平台
MultiWave波形识别算法

学术转化

会议论文: IEEE ICC2016
会议论文: IEEE ISPA2017
期刊论文:《华科学报自然版》
专利3项

产品介绍

Product Description

  • Android APP
  • 介绍Web
  • 智能终端

实验结果

Experimental Result

大数据

采集坐标系

网络

结构

准确率

UI

Monitoring the Tooth Brushing
Quality for Teenagers with Smart Watch

With the increasing job stress and fast tempo of modern life, people from all walks of life have less time for their hygiene routines—the daily tooth brushing is one of them. In fact, brushing teeth every day may be the most cost-efficient way to maintain oral health, when considering the frequent occurrence of oral health problems and the high cost of consequent dental cares.

Previous Solution

In 1988, the authors presented a smart system by modifying the holder of a toothbrush with chip that has build-in accelerometer, magnetic sensor and communication module. In 2013, Authors of Rudiger ZillmerRudiger also present a 3-axial accelerometer-based toothbrushing activity detector. In 2016, due to the non-reusable of toothbrush, Hua HuangHua and Lin, used the sensors in smart watch to capture the hand poses during tooth brushing. In 2018, Boyu proposed a smart Water Flosse that was also an IMU-based detection system, in which the researchers modified the holder of the water flosser to capture the usage of during washing the gingiva and gaps between teeth. They divided the whole mouth into 16 different areas, and compared different machine learning models for detecting the motions.

By adding a simple and cheap 3D colored target at the end of the toothbrush, the researchers developed a camera-based application to capture the brushing pose through a tablet or a smartphone. The added target is a colored ball with different color lumps. And then Marco updated their system by adding IMU sensors with the colored ball. By placing a small camera in the center of the Halo, some researchers designed a vision based brushing evaluator called LumiO. They used Quantitative Light-induced Fluorescence (QLF) to analyze the pictures captured inside the mouth, and detection the plaque on the tooth surface with three machine learning techniques.

Korpela et al., 2015 first evaluated the daily tooth brushing performance with sound data. Their system utilized the normal smartphone microphone to capture the brushing signals and build hidden Markov models (HMMs) to recognize the brushing actions. Different combination of features are used to evaluate brushing performance scores. A novel asymmetrical acoustic field detection system Zhenchaoetal.,2017 is designed by combining a single-channel throat microphone and a single-channel bluetooth earphone. Different time domain and frequent domain features are calculated from the raw signals for the training of five different machine learning models. And the researcher found the random forest model can achieve the best performance.

In this paper, we present a novel tooth brushing monitoring and evaluating system by using smart devices. Our system captures the motion behavior of the user’s hand and unique acoustic signals during tooth brushing through the inertia measurement unit (IMU) sensor (accelerator and gyroscope) and a normal microphone, respectively. We also analyze the relationship between the characteristics of the signal and the corresponding brushing surface, and build several different machine learning models in supervised learning method.

References

  • H. Huang and S. Lin, “Toothbrushing monitoring using wrist watch,” in SenSys’16: Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems. ACM, 2016, pp. 202–215.
  • Y.-J. Lee, P.-J. Lee, K.-S. Kim, W. Park, K.-D. Kim, D. Hwang, and J.-W. Lee, “Toothbrushing region detection using three-axis accelerometer and magnetic sensor,” in IEEE Transactions on Biomedical Engineering, vol. 59(3). IEEE, 2012, pp. 872–881.
  • Y.-J. Lee and et al., “Quantitative assessment of toothbrushing education efficacy using smart toothbrush,” in 2011 4th International Conference on Biomedical Engineering and Informatics (BMEI), vol. 2. IEEE, 2011, pp. 1160–1164.
  • J. Korpela, R. Miyaji, T. Maekawa, K. Nozaki, and H. Tamagawa, “Evaluating tooth brushing performance with smartphone sound data,” in UbiComp’15: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 2015, pp. 109–120.
  • Y. Bi, M. Lv, C. Song, W. Xu, N. Guan, and W. Yi, “Autodietary: A wearable acoustic sensor system for food intake recognition in daily life,” IEEE Sensors Journal, pp. 806–816, 2016.

Data Collection

During data collection, each time the volunteer wears the smart watch and brushes their teeth for three minutes. An assistant will label the brushing event and use the smart watch App to collect IMU sensor data and microphone acoustic signal which will be transmitted to the smartphone later. The smartphone divides the sensor data into fixed windows and calculates the time and frequent domain features. Combined with the labeled brushing events, we organized a brushing event dataset for machine learning modeling.

System Design

We first divide the oral area into 6 different teeth surfaces, two front teeth and four back teeth. Each front teeth contains a inner surface and a outer surface, and each back teeth contains a inner surface, a chewing surface and a outer surface. Therefore, the oral area is total divided into 16 different areas.

The smart watch based monitoring system that can capture user’s hand motion and brushing acoustic signal thought build-in IMU sensor and microphone. The IMU are commonly used for measuring the gestures, while the microphone can capture the sound when taking phone calls.

The smart watch is used for collecting tooth brushing behaviors of the user.

Model training: The smart watch App will collect IMU sensor data and microphone acoustic signal, and transmit to the smartphone. A sliding window with fixed step is used for cutting the data into fixed frames to calculate the time and frequent domain features. Combined with the labeled brushing events, we organized a brushing event dataset for machine learning modeling. Different models are trained to select the best candidate model.

Real-time monitoring: After the selected model is integrated into smartphone data processing flow, the system can directly use the data of the total brushing time on each surface to evaluate the whole brushing quality.

If the Bass brushing method is used, the total brushing time is required to be no less than 3 minutes. The model will alerts the user when it detects that a tooth surface is cleaned for less than 10 seconds.