AIoT breakthrough: Enhancing home security with MSF-Net

Researchers at Incheon National University have developed the Multiple Spectrogram Fusion Network (MSF-Net), an Artificial Intelligence of Things (AIoT) framework aimed at enhancing smart home security through WiFi-based human activity recognition.

This AIoT solution utilises deep learning and multimodal frequency fusion to address the challenge of environmental interference, substantially improving recognition accuracy. Professor Gwanggil Jeon of the College of Information Technology at the university led the team behind this study.

AIoT combines Artificial Intelligence and the Internet of Things, enabling devices to process data and make decisions locally in real-time. This approach diverges from conventional IoT systems where data collection and processing occur separately. This real-time capability makes AIoT suitable for applications such as intelligent manufacturing, smart home security, and healthcare monitoring.

In the realm of smart home technology, human activity recognition using AIoT is vital. It allows devices to discern different activities like cooking and exercising. Consequently, the system can adjust environmental settings such as lighting and music based on the detected activity, improving user experience and energy efficiency. The integration of WiFi in this context is promising due to its ubiquity, privacy assurance, and cost-effectiveness.

Professor Jeon detailed the motivation behind this research, stating, "As a typical AIoT application, WiFi-based human activity recognition is becoming increasingly popular in smart homes. However, WiFi-based recognition often has unstable performance due to environmental interference. Our goal was to overcome this problem."

To address these concerns, the researchers created the MSF-Net framework. This robust deep learning framework enables both coarse and fine activity recognition via Channel State Information (CSI). It comprises a dual-stream structure that utilises short-time Fourier transform and discrete wavelet transform, a transformer component, and an attention-based fusion branch. The dual-stream structure identifies irregularities in CSI, while the transformer efficiently extracts high-level features. The attention-based fusion branch enhances cross-model fusion.

Experimental validation of MSF-Net demonstrated impressive performance, achieving Cohen's Kappa scores of 91.82%, 69.76%, 85.91%, and 75.66% on the SignFi, Widar3.0, UT-HAR, and NTU-HAR datasets, respectively. These results indicate that MSF-Net outperforms existing technologies in WiFi data-based activity recognition.

Professor Jeon noted, "The multimodal frequency fusion technique has significantly improved accuracy and efficiency compared to existing technologies, increasing the possibility of practical applications. This research can be used in various fields such as smart homes, rehabilitation medicine, and care for the elderly. For instance, it can prevent falls by analysing the user's movements and contribute to improving the quality of life by establishing a non-face-to-face health monitoring system."

Activity recognition via WiFi, as proposed in this study, has substantial potential to enhance daily life by ensuring convenience and safety. The full findings of this research are detailed in the IEEE Internet of Things Journal, Volume 11, Issue 24, released on 15 December 2024.