Journal publication

Nov 2023
Validating generalizability of ophthalmic Artificial Intelligence models on real-world clinical data. Rashidisabet, H., Sethi, A., Jindarak, B., Edmonds, D., Chan, P., Vajaranant, T., Yi, D. Translational Vision Science & Technology

Abstract:
Purpose This study aims to investigate generalizability of deep learning (DL) models trained on commonly used public fundus images to an instance of Real-World Data (RWD) for glaucoma diagnosis.
Methods We used Illinois Eye and Ear Infirmary fundus dataset as an instance of RWD in addition to six publicly available fundus datasets. We compared the performance of DL-trained models on public data and RWD for glaucoma classification and Optic Disc (OD) segmentation tasks. For each task, we created models trained on each dataset, respectively, and each model was tested on both datasets. We further examined each model's decision-making process and learned embeddings for the glaucoma classification task.
Results Using public data for test set, public-trained models outperformed RWD-trained models in OD segmentation and glaucoma classification with mean Intersection over Union (IoU) of 96.3% and mean Area Under the Receiver Operating Characteristic Curve (AUROC) of 95.0%, respectively. Using the RWD test set, the performance of public models decreased by 8.0% and 18.4% to 85.6% and 76.6% for OD segmentation and glaucoma classification tasks, respectively. RWD models outperformed public models on RWD test sets by 2.0% and 9.5%, respectively, in OD segmentation and glaucoma classification tasks.
Conclusions DL models trained on commonly used public data have limited ability to generalize to RWD for classifying glaucoma. They perform similarly to RWD models for OD segmentation.
Translational Relevance RWD is a potential solution for improving generalizability of DL models and enabling clinical translations in care of the prevalent blinding ophthalmic conditions, such as glaucoma.

Show more
Oct 2023
Personalized relapse prediction in patients with major depressive disorder using digital biomarkers. Vairavan, S., Rashidisabet, H., Li, Q., Ness, S., Morrison, R., Soares, C., Uher, R., Frey, B., Lam, R., Kennedy, S., Trivedi, M., Drevets, W., Narayan, V. Scientific Reports - Nature

Abstract: Major depressive disorder (MDD) is a chronic illness wherein relapses contribute to significant patient morbidity and mortality. Near-term prediction of relapses in MDD patients has the potential to improve outcomes by helping implement a ‘predict and preempt’ paradigm in clinical care. In this study, we developed a novel personalized (N-of-1) encoder-decoder anomaly detection-based framework of combining anomalies in multivariate actigraphy features (passive) as triggers to utilize an active concurrent self-reported symptomatology questionnaire (core symptoms of depression and anxiety) to predict near-term relapse in MDD. The framework was evaluated on two independent longitudinal observational trials, characterized by regular bimonthly (every other month) in-person clinical assessments, weekly self-reported symptom assessments, and continuous activity monitoring data with two different wearable sensors for ≥ 1 year or until the first relapse episode. This combined passive-active relapse prediction framework achieved a balanced accuracy of ≥ 71%, false alarm rate of ≤ 2.3 alarm/patient/year with a median relapse detection time of 2–3 weeks in advance of clinical onset in both studies. The study results suggest that the proposed personalized N-of-1 prediction framework is generalizable and can help predict a majority of MDD relapses in an actionable time frame with relatively low patient and provider burden.

Show more
May 2023
Which OCT parameters can best predict visual field progression in glaucoma? Sethi, A., Rashidisabet, H., Hallak, J., Vajaranant, T. Eye - Nature

Abstract: Which OCT-based parameters are the best predictors for current and future functional damage in glaucoma patients? The answer is that it depends. Overall, RNFLT changes can predict VF progression; however, limitations such as the “floor effect”, myopia, sensitivities to decentration error, thickness changes outside the range of the VF test may prove challenging for accurate monitoring, especially in eyes with advanced glaucoma. Therefore, we propose three clinical implications: (1) Leverage all imaging: utilize RNFL, macular OCT and optic disc evaluation in tandem with VF for glaucoma diagnosis and monitoring. Be mindful that key diagnostic and prognostic parameters, such as disc hemorrhages and pallor, cannot be detected using OCT. (2) OCT macula can be helpful in early detection in eyes with anomalous nerves or myopia and in advanced stage-glaucoma when the RNFL reaches the floor. (3) Future studies can consider OCT angiography to evaluate optic disc perfusion, which could enhance the prognostic prediction, in combination with our current OCT parameters.

Show more
April 2022
Revisiting Power-Law Estimation with Applications to Real-World Human Typing Dynamics. Rashidisabet, H., Ajilore, O., Leow, A., Demos, A. Physica A: Statistical Mechanics and its Applications.

Abstract: Ubiquitous use of smartphones has significantly shaped interpersonal communications in modern life, in particular interactions via text messaging. To study the underlying dynamics of these smartphone-based human communications, we used a unique smartphone typing dataset that was passively and unobtrusively collected in-the-wild from 296 users via a custom-made iPhone keyboard. To reliably and accurately characterize the underlying distribution of the inter-event time between two consecutive keypresses, we (i) developed a statistical approach that integrates existing methods for estimating power-law distribution, and (ii) showed that power-law is a plausible candidate to represent human typing dynamics. We designed synthetic-data simulations in multiple scenarios where the synthetic data may or may not imitate human typing characteristics. Using numerical simulations, we showed that our approach, in all scenarios, improves the accuracy and stability of power-law estimates upon the common methods. We further demonstrated that when the synthetic data follow human typing characteristics, common methods lead to significant misestimations of power-law exponent as they fail to take into account the key characteristics of the observed data. More broadly, our approach applies beyond the power-law estimation for human typing dynamics data.

Show more
April 2020
A Systems Biology Approach to the Digital Behaviorome. Rashidisabet, H., Thomas, P., Ajilore, O., Zulueta, J., Moore, R., Leow, A. Elsevier Journal of Current Opinion in Systems Biology.

Abstract: This review article summarizes how emerging connected technologies (e.g., smartphones and wearables) may provide novel avenues to understand an individual's behavior through the lens of systems biology. First, we surveyed recent research efforts that leveraged the multimodal high temporal resolution data derived from connected devices to build digital phenotypes and/or discover digital biomarkers of the behaviorome. We write this review with a particular emphasis on the detection, diagnosis, and symptom monitoring of neuropsychiatric disorders, as these pathologies may manifest primarily as disruptions to the behaviorome. We then discussed new opportunities and challenges these state-of-art research efforts bring as they intersect with other areas of natural and social sciences. Ultimately, we suggest how incorporating systems biology and connected technologies data can lead to a better understanding of complex neuropsychiatric disorders.

Show more
July 2020
Effects of Mood and Aging on Keystroke Dynamics Metadata and Their Diurnal Patterns in A Large Open-Science Sample: A BiAffect iOS Study. Vesel, C., Rashidisabet, H., Zulueta, J., Stange, J., Duffecy, J., Hussain, F., Piscitello, A., Bark, J., Langenecker, S., Young, S., Mounts, E., Omberg, L., Nelson, P., Moore, R., Koziol, D., Bourne, K., Bennett, C., Ajilore, O., Demos, A., Leow, A. Journal of the American Medical Informatics Association (JAMIA).

Abstract:
Objective Ubiquitous technologies can be leveraged to construct ecologically relevant metrics that complement traditional psychological assessments. This study aims to determine the feasibility of smartphone-derived real-world keyboard metadata to serve as digital biomarkers of mood.
Materials and Methods BiAffect, a real-world observation study based on a freely available iPhone app, allowed the unobtrusive collection of typing metadata through a custom virtual keyboard that replaces the default keyboard. User demographics and self-reports for depression severity (Patient Health Questionnaire-8) were also collected. Using > 14 million keypresses from 250 users who reported demographic information and a subset of 147 users who additionally completed at least 1 Patient Health Questionnaire, we employed hierarchical growth curve mixed-effects models to capture the effects of mood, demographics, and time of day on keyboard metadata.
Results We analyzed 86 541 typing sessions associated with a total of 543 Patient Health Questionnaires. Results showed that more severe depression relates to more variable typing speed (P < 0.001), shorter session duration (P < .001), and lower accuracy (P < .05). Additionally, typing speed and variability exhibit a diurnal pattern, being fastest and least variable at midday. Older users exhibit slower and more variable typing, as well as more pronounced slowing in the evening. The effects of aging and time of day did not impact the relationship of mood to typing variables and were recapitulated in the 250-user group.
Conclusions Keystroke dynamics, unobtrusively collected in the real world, are significantly associated with mood despite diurnal patterns and effects of age, and thus could serve as a foundation for constructing digital biomarkers.

Show more

Book chapter

July 2023
Passive Sensing of Affective and Cognitive Functioning in Mood Disorders by Analyzing Keystroke Kinematics and Speech Dynamics. Hussain, F., Stange, J., Langenecker, S., McInnis, M., Zulueta, J., Piscitello, A., Ross, M., Demos, A., Vesel, C., Rashidisabet, H., Cao, B., Huang, H., Yu, P., Nelson, P., Ajilore, O., Leow, A. Digital phenotyping and mobile sensing: New developments in psychoinformatics, pp.161-183

Abstract: Mood disorders can be difficult to diagnose, evaluate, and treat. They involve affective and cognitive components, both of which need to be closely monitored over the course of the illness. Current methods like interviews and rating scales can be cumbersome, sometimes ineffective, and oftentimes infrequently administered. Even ecological momentary assessments, when used alone, are susceptible to many of the same limitations and still require active participation from the subject. Passive, continuous, frictionless, and ubiquitous means of recording and analyzing mood and cognition obviate the need for more frequent and lengthier doctor’s visits, can help identify misdiagnoses, and would potentially serve as an early warning system to better manage medication adherence and prevent hospitalizations. Activity trackers and smartwatches have long provided exactly such a tool for evaluating physical fitness. What if smartphones, voice assistants, and eventually Internet of Things devices and ambient computing systems could similarly serve as fitness trackers for the brain, without imposing any additional burden on the user? In this chapter, we explore two such early approaches—an in-depth analytical technique based on examining meta-features of virtual keyboard usage and corresponding typing kinematics, and another method which analyzes the acoustic features of recorded speech—to passively and unobtrusively understand mood and cognition in people with bipolar disorder. We review innovative studies that have used these methods to build mathematical models and machine learning frameworks that can provide deep insights into users’ mood and cognitive states. We then outline future research considerations and conclude with discussing the opportunities and challenges afforded by these modes of researching mood disorders and passive sensing approaches in general.

Show more