Sleep is an important behavioral biomarker for patients with serious mental illness (SMI). The ability to accurately quantify sleep in a real-world setting could thus provide insight into patient well-being. In this study, patients in a sleep lab wore a patch that is part of a digital medicine system (aripiprazole with sensor (AS)) designed to provide objective records of medication ingestion. The patch provided accelerometer and electrocardiogram (ECG) data; polysomnography (PSG) data was collected to be used as the gold standard for sleep stage classification. The accelerometer and ECG data were used to build machine learning classification models to distinguish periods of wake from periods of sleep. To optimize these models for a real-world environment, different data sampling paradigms and methodologies were explored, and resultant model performances were analyzed.

Data was collected for a total of 220 nights, across 73 unique subjects—42 subjects had a diagnosed SMI (schizophrenia, bipolar disorder I, or major depressive disorder) and 31 subjects were healthy volunteers. PSG data, which provides a sleep stage designation at 30-second intervals, was combined into 5-minute windows, labeled as either “Sleep” or “Wake” based on which class comprised the majority of the 30-second intervals within the window. Accelerometer and ECG features were derived for each 5-minute window. Models were trained with three learning methodologies: a light gradient boosting machine (LGBM), a conditional random field (CRF), and a long short-term memory (LSTM) network. Model performance was tested with the full complement of accelerometer and ECG data, as well as down-sampled subsets of data. Additionally, ECG data from the PSG system was incorporated to test the effect of other ECG sampling strategies.

CRF models produced the best classification performance (AUC = 0.91) with the full patch dataset. Down-sampling to include less than half of the accelerometer data did begin to degrade the specificity of the model. Down-sampling to include less frequent ECG collection did not have a significant effect on model performance; however, changing the sampling paradigm to continuous ECG collection from a block sampling paradigm did lead to more robust classification of when a patient was awake.

Accurately recording sleep in a logistically simple way can provide insights into the well-being of SMI patients. Combining these insights with the objective medication ingestion records provided by AS would be of great value to SMI patients, as well as their caregivers and physicians. This research explores what amount of sensor data is required to accurately quantify sleep and some of the machine learning strategies that can ameliorate data limitations, providing guidance for the optimization of digital device design.

Otsuka Pharmaceutical Development & Commercialization, Inc.