About 

The number of everyday smart devices (e.g., Nest, Notion,  Samsung SmartThings) is projected to grow to the billions in the coming decade. The Cyber-Physical Systems or Internet of Things systems that consist of these devices obtain human information for various smart applications. Due to the complexity of the physical world, data acquisition quality and sensing data distributions can change significantly under different sensing conditions.

        From the system perspective, my research focuses on self-assessing and self-adaptive heterogeneous cyber-physical sensing systems for accurate non-intrusive human information inference.  From the data perspective, my research addresses the challenges of limited labeled data by combining physical and data-driven knowledge to model the change.

        UC Merced is located between Silicon Valley and beautiful Yosemite National Park, allowing us to work closely with our industrial collaborators. 

I'm looking for highly motivated PhD students.
Please contact me via email.

 

News

Important Links:

  • Workshop DATA: Acquisition to Analysis 2018 2019

  • Workshop Combine Physical and Data-Driven Knowledge in Ubiquitous Computing (CPD) 2018 2019 2020

  • Workshop Device-free Human Sensing 2019

  • Workshop Continual and Multimodal Learning for Internet of Things (CML-IoT) 2019 2020

  • AutoCheckout Competition @ CPS-IoT Week 2020

  • GitHub repo for Structural Vibration-based Human Sensing 

Media Exposure:

  • UC Merced news: link

  • UC Merced EECS Graduate Studies: link

  • Bloomberg, AutoCheck Competition: link

 

Projects

Self-assessing CPS/IoT: Task-based Sensing Signal and Data Quality Assessment
Data acquisition quality directly affects the information representative as well as the model accuracy in applications for real-world deployed cyber-physical systems. We combine physical and data-driven knowledge to design metrics and methods to assess the signal and dataset quality for particular sensing tasks. The assessments are used for:
  • Fair dataset quality comparison for system performance evaluation and dataset sharing
  • Collaboratively sensing system adaptation to optimize data quality.
  • System self-configuration to enhance CPS scalability.
Publication
Yue Zhang, Susu Xu, Laixi Shi, Shijia Pan. Poster Abstract: Using Mobile Sensing to Enable the Signal Quality Assessment for Infrastructure Sensing Systems. In the 21st Annual International Workshop on Mobile Computing Systems and Applications (ACM HotMobile 2020).
 
Yue Zhang, Lin Zhang, Hae Young Noh, Pei Zhang, and Shijia Pan. A Signal Quality Assessment Metric for Vibration-based Human Sensing Data Acquisition. In the 2nd Workshop on Data Acquisition to Analysis. November 10, 2019, New York, NY, USA.
Elderly Monitoring with Heterogeneous IoT Systems
Collaborator: HAA

​Monitoring ​older adults' walking patterns and analyzing their fall risk is essential for fall prevention. Prior technologies such as computer vision or audio sensing raise privacy issues for long term home monitoring scenarios. We look into an alternative solution through structural (e.g., floor) vibration (infrastructural sensing) and wearables (mobile sensing). By utilizing their complementary sensing properties and shared context, we can obtain high-fidelity data for older adults' information learning and modeling.

Publication:

Laixi Shi, Yue Zhang, Shijia Pan, Yuejie Chi. Poster Abstract: Data Quality-Informed Multiple Occupant Localization using Floor Vibration Sensing. In the 21st Annual International Workshop on Mobile Computing Systems and Applications (ACM HotMobile 2020).

Laixi Shi, Mostafa Mirshekari, Jonathon Fagert, Yuejie Chi, Hae Young Noh, Pei Zhang, and Shijia Pan. Device-free Multiple People Localization through Floor Vibration. In the 1st ACM International Workshop on Device-Free Human Sensing, November 10, 2019, New York, NY, USA.

Heterogeneous Cyber-Physical Sensing Systems
Application: Autonomous Retail 
Collaborators: AiFi Inc., CMU, Stanford

The heterogeneity of cyber-physical systems brings challenges and opportunities for various autonomous system applications. The application we target is autonomous retail and store inventory management. We utilize the complementary characteristics of multiple sensing modalities including computer vision, weight, and location information of the items to achieve accurate item pick up detection and recognition.

Publication

Carlos Ruiz, Joao Falcao, Shijia Pan, Hae Young Noh, and Pei Zhang.   AIM3S: Autonomous Inventory Monitoring through Multi-Modal Sensing for Cashier-Less Convenience Stores. In the Proceeding of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys' 19), November 2019.

Structures as Sensors: floor-vibration-based pig monitoring
Collaborator: CMU, CMKL, Betagro

Pigs' behavior and health condition directly affect farms' profit as well as meat product quality. However, wearables are often easily destroyed by pigs and cameras often suffer from occlusion situations. We look into the structural vibration induced by pigs and placed vibration sensors underneath the floor slab of pig pens for pig activity monitoring.

Publication:

Ariyadech, Sripong, Amelie Bonde, Orathai Sangpetch, Woranun Woramontri, Wachirawich Siripaktanakon, Shijia Pan, Akkarit Sangpetch, Hae Young Noh, and Pei Zhang. "Dependable Sensing System for Pig Farming." In 2019 IEEE Global Conference on Internet of Things (GCIoT), pp. 1-7. IEEE, 2019.

Amelie Bonde, Shijia Pan, Orathai Sangpetch, Akkarit Sangpetch, Woranun Woramontri, and Pei Zhang. "Structural vibration sensing to evaluate animal activity on a pig farm." In the 1st Workshop on Data Acquisition to Analysis. pp. 25-26. 2018.

Sleep stage monitoring through contact-less sensing
Collaborator: NCH, CMU

Sleep disorder impairs people's health. To accurately analyze patients' sleep quality, it is important to monitor their sleep stages in their natural status. Prior methods, such as polysomnography (PSG) and Fitbit, are often intrusive and having the potential to change the user's daily sleep routine. We non-intrusively identify sleep stages through bed-frame vibrations. Our system detects patients' movements during their sleep and estimates their sleep stages via their movements induced vibration on the bed frame.

Publication

Zhizhang Hu, Emre Sezgin, Simon Lin, Pei Zhang, Hae Young Noh, and Shijia Pan. Device-free Sleep Stage Recognition through Bed Frame Vibration Sensing. In the 1st ACM International Workshop on Device-Free Human Sensing, November 10, 2019, New York, NY, USA.

 

People

PI: Shijia Pan (cv pdf, Google scholar)

Dr. Shijia Pan received her Bachelor's degree in Computer Science and Technology from the University of Science and Technology of China (USTC) and her Ph.D. degree in Electrical and Computer Engineering at Carnegie Mellon University (CMU). Her research interests include cyber-physical systems, Internet-of-Things (IoT), and ubiquitous computing. She worked in multiple disciplines and focused on self-assessing and self-adaptive heterogeneous cyber-physical systems for accurate human information inference. 

PhD Students

Yue Zhang 

Yue received his Master's degree and Bachelor's degree in the Department of Electronic Engineering at Tsinghua University in 2019 and 2016. His research interests include sensor networks, embedded systems, indoor human information acquisition. He is an exchange scholar since fall 2019 and will join the lab as a PhD student in 2020 fall. 

Zhizhang Hu

Zhizhang has an interdisciplinary background: mechanical engineering, building science, and data science (He will receive his Master's degree from School of Architecture at Carnegie Mellon University in May 2020). His research interests include data mining for time-series sensor data streams, machine learning, deep learning, and non-intrusive behavior monitoring. Zhizhang will join the lab as a PhD student in the 2020 fall semester.

Supervised Students

  • Anthony Sainez (2020 summer - present)

  • Rahul Sidramappa Hoskeri (2020 summer - present)

  • Lixing He, UESTC (2020 spring - present)

  • Laixi ShiCMU (2019 spring - 2020 spring)

  • Jothi Prasanna Shanmuga Sundaram, UC Merced (2019 fall)

  • Di An, UC Merced (2019 fall)

 

Courses

2020 Spring: EECS 283 Advanced Topics in Intelligent Systems

(syllabus)

2020 Fall: EECS 283 Advanced Topics in Intelligent Systems

 

Publications (since 2019 fall)

Fully Peer-Reviewed Conference Publication

Amelie Bonde, Shijia Pan, Mostafa Mirshekari, Carlos Ruiz, Hae Young Noh, and Pei Zhang. OAC: Overlapping Office Activity Classification through IoT-Sensed Structural Vibration. 

Will appear in the Proceeding of IoTDI 2020, April 2020.

Acceptance rate = 35%

Carlos Ruiz, Shijia Pan, Adeola Bannis, Ming-Po Chang, Hae Young Noh, and Pei Zhang. IDIoT: Towards Ubiquitous Identification of IoT Devices through Visual and Inertial Orientation Matching During Human Activity. Will appear in the Proceeding of IoTDI 2020, April 2020. (received the Best Paper Award).

Acceptance rate = 35%

Shijia Pan, Mario Berges, Juleen Rodakowski, Pei Zhang, and Hae Young Noh. Fine-Grained Activities of Daily Living Recognition through Structural Vibration and Electrical Sensing. In the Proceeding of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys' 19), November 2019.
Acceptance rate = 30%

 

Carlos Ruiz, Joao Falcao, Shijia Pan, Hae Young Noh, and Pei Zhang.   AIM3S: Autonomous Inventory Monitoring through Multi-Modal Sensing for Cashier-Less Convenience Stores. In the Proceeding of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys' 19), November 2019. (the corresponding demo has received the Best Demo Award) 
Acceptance rate = 30%

Other Conference/Workshop Publication

Yue Zhang, Susu Xu, Laixi Shi, Shijia Pan. Poster: Using Mobile Sensing to Enable the Signal Quality Assessment for Infrastructure Sensing Systems. In the 21st Annual International Workshop on Mobile Computing Systems and Applications (ACM HotMobile 2020) link.

Laixi Shi, Yue Zhang, Shijia Pan, Yuejie Chi. Poster: Data Quality-Informed Multiple Occupant Localization using Floor Vibration Sensing. In the 21st Annual International Workshop on Mobile Computing Systems and Applications (ACM HotMobile 2020) link.

Yue Zhang, Lin Zhang, Hae Young Noh, Pei Zhang, and Shijia Pan. A Signal Quality Assessment Metrics for Vibration-based Human Sensing Data Acquisition. In the 2nd Workshop on Data Acquisition to Analysis. November 10, 2019, New York, NY, USA. link

Laixi Shi, Mostafa Mirshekari, Jonathon Fagert, Yuejie Chi, Hae Young Noh, Pei Zhang, and Shijia Pan. Device-free Multiple People Localization through Floor Vibration. In the 1st ACM International Workshop on Device-Free Human Sensing, November 10, 2019, New York, NY, USA.

Zhizhang Hu, Emre Sezgin, Simon Lin, Pei Zhang, Hae Young Noh, and Shijia Pan. Device-free Sleep Stage Recognition through Bed Frame Vibration Sensing. In the 1st ACM International Workshop on Device-Free Human Sensing, November 10, 2019, New York, NY, USA.

Earlier publications can be found HERE

© 2023 By Emilia Kent. Proudly created with Wix.com