Publications

My work spans representation learning for sparse and structured signals, interactive textile sensing, computational behavior analysis, and physiological signal modeling. Below is a set of publications organized by research area.

Interactive and Knitted Sensor Systems

Research on minimally wired textile sensing systems that recover human intent through touch and gesture using representation learning, temporal modeling, and user-grounded experimentation.

Recognizing Complex Gestures on Minimalistic Knitted Sensors: Toward Real-World Interactive Systems

Denisa Qori McDonald, Richard James Vallet, Lev Saunders, Genevieve Dion, and Ali Shokoufandeh.
arXiv, 2023.

Key Contribution: Implements a CNN-LSTM architecture on a 4-electrode knitted sensor to recognize complex continuous gestures, moving from static touch localization toward deployable, real-time-capable interactive systems.

PDF DOI

Developments in touch-sensitive textiles have enabled many novel interactive techniques and applications. Our digitally-knitted capacitive active sensors can be manufactured at scale with little human intervention. Their sensitive areas are created from a single conductive yarn, and they require only few connections to external hardware. This technique increases their robustness and usability, while shifting the complexity of enabling interactivity from the hardware to computational models. This work advances the capabilities of such sensors by creating the foundation for an interactive gesture recognition system. It uses a novel sensor design, and a neural network-based recognition model to classify 12 relatively complex, single touch point gesture classes with 89.8% accuracy, unfolding many possibilities for future applications. We also demonstrate the system’s applicability and robustness to real-world conditions through its performance while being worn and the impact of washing and drying on the sensor’s resistance.

Interaction with Touch-Sensitive Knitted Fabrics: User Perceptions and Everyday Use Experiments

Denisa Qori McDonald, Shruti Mahajan, Richard James Vallet, Erin Solovey, Genevieve Dion, and Ali Shokoufandeh.
ACM CHI Conference on Human Factors in Computing Systems (CHI), 2022. Acceptance rate: 24.6%.

Key Contribution: Formative study and real-world experimentation linking user expectations, accidental touch, and durability to concrete technical research questions for interactive knitted sensing.

PDF DOI Slides Sensor Demo

Recent work has investigated the construction of touch-sensitive knitted fabrics, capable of being manufactured at scale, and having only two connections to external hardware. Additionally, several sensor design patterns and application prototypes have been introduced. Our aim is to start shaping the future of this technology according to user expectations. Through a formative focus group study, we explore users’ views of using these fabrics in different contexts and discuss potential concerns and application areas. Subsequently, we take steps toward addressing relevant questions, by first providing design guidelines for application designers. Furthermore, in one user study, we demonstrate that it is possible to distinguish different swipe gestures and identify accidental contact with the sensor, a common occurrence in everyday life. We then present experiments investigating the effect of stretching and laundering of the sensors on their resistance, providing insights about considerations necessary to include in computational models.

Knitted Sensors: Designs and Novel Approaches for Real-Time, Real-World Sensing

Denisa Qori McDonald, Richard James Vallet, Erin Solovey, Genevieve Dion, and Ali Shokoufandeh.
Proc. of ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT). Presented at UbiComp 2021.

Key Contribution: Subject-independent touch localization on a 36-button knitted sensor using multi-frequency feature construction and LSTM-based temporal modeling.

PDF DOI

Recent work has shown the feasibility of producing knitted capacitive touch sensors through digital fabrication with little human intervention in the textile production process. Such sensors can be designed and manufactured at scale and require only two connection points, regardless of the touch sensor form factor and size of the fabric, opening many possibilities for new designs and applications in textile sensors. To bring this technology closer to real-world use, we go beyond previous work on coarse touch discrimination to enable fine, accurate touch localization on a knitted sensor, using a recognition model able to capture the temporal behavior of the sensor. Moreover, signal acquisition and processing are performed in real-time, using swept frequency Bode analysis to quantify distortion from induced capacitance. After training our network model, we conducted a study with new users, and achieved a subject-independent accuracy of 66% in identifying the touch location on the 36-button sensor, while chance accuracy is approximately 3%. Additionally, we conducted a study demonstrating the viability of taking this solution closer to real-world scenarios by testing the sensor’s resistance to potential deformation from everyday conditions. We also introduce several other knitted designs and related application prototypes to explore potential uses of the technology.

Toward Accurate Sensing with Knitted Fabric: Applications and Technical Considerations

Richard James Vallet, Denisa Qori McDonald, Genevieve Dion, Youngmoo Kim, and Ali Shokoufandeh.
Proc. of ACM on Human–Computer Interaction. Presented at EICS 2020/2021. Best Paper Award, awarded to the top 1% of submissions.

Key Contribution: Introduces Mixed-Source Description (MSD) and Euclidean Levenshtein Distance (ELD) to compute invariant, sparse representations of variable-length sequential signals, reducing processing time by 50 times.

PDF DOI

Fabric sensors have been introduced to enable flexible touch-based interaction. We advance the technical capabilities of a scalable and low-profile knitted capacitive touch sensing system by introducing methods to improve its touch localization accuracy. The sensor hardware design tends toward minimalism by using a single conductive yarn and two external connections located at each endpoint. Fewer connectors simplify the textile system integration, but this comes at the expense of reduced signal information output from the system. The electrical continuity of the sensing element, essential to the process of knitting, also increases the uncertainty of localizing touch. We propose using Bode analysis to measure changes in signal due to capacitive touch, as well as design a new algorithm, Mixed-Source Description (MSD), which retains the most significant aspects of the signal in terms of touch location identification. We do not classify location of touch, but focus on an invariant signal representation. To evaluate our methods, we introduce Euclidean Levenshtein Distance (ELD), a distance metric to compute the similarity of pairs of key-presses, generalizable to computing distances of tensors of varying lengths. Our experiments show that the proposed sensing method results in high-fidelity signals. Furthermore, the sparse representation of key-presses produced by MSD significantly increases separability between different touch locations. Possible applications based on these sensors are also illustrated through prototypes and use case descriptions.

Advanced Manufacturing of Touch–Sensitive Textile

Richard Vallett, Denisa Qori McDonald, Dario Salvucci, Geneviève Dion, and Ali Shokoufandeh.
Artificial Intelligence in Manufacturing, 2024. Invited book chapter, peer–reviewed.

Key Contribution: Synthesizes the end-to-end architecture of functional textile interfaces, from fabric creation, circuit design and signal acquisition to the unsupervised and supervised modeling strategies that map sparse signals to high-level interaction.

DOI

The ubiquity of textiles, as well as their soft and flexible nature, makes them an appealing medium for tangible embedded interactions. This interactivity is achieved through advanced textile manufacturing processes, novel sensing circuitry, computational strategies, and trained stochastic models to create sensitive and intelligent functional fabric devices. Recent work has demonstrated functional textile capacitive touch sensors that infer contact location through a simple but adaptable fabric-based circuit that can form a variety of physical interface layouts. The textile circuit connects to external sensing electronics using up to four macro-scale electrodes, thus improving systems integration between the fabric substrate and durability when stretched or folded. Supplementary sensing electronics perform real-time signal generation, acquisition, and processing to capture numerical data representative of touch interactions. This data is passed to both unsupervised and supervised models which process and estimate touch location, contact pressure, and temporal gestures which constitute the high-level input to end-user applications.

Computational Behavior Analysis and Social Interaction

Interpretable modeling of conversational behavior, with an emphasis on relational features, developmental variation, and clinically relevant analysis in naturalistic social settings.

Predicting Autism from Head Movement Patterns during Naturalistic Social Interactions

Denisa Qori McDonald, Evangelos Sariyanidi, Casey J. Zampella, Ellis DeJardin, John D. Herrington, Robert T. Schultz, and Birkan Tunç.
International Conference on Medical and Health Informatics (ICMHI), 2023. Best Presentation Award.

Key Contribution: Evaluates monadic and dyadic representations of conversational head movement for autism-related behavioral analysis, demonstrating that explicitly modeling interpersonal coordination improves accuracy while maintaining non-diagnostic, interpretable framing.

PDF DOI Slides

Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized in part by difficulties in verbal and nonverbal social communication. Evidence indicates that autistic people, compared to neurotypical peers, exhibit differences in head movements, a key form of nonverbal communication. Despite the crucial role of head movements in social communication, research on this nonverbal cue is relatively scarce compared to other forms of nonverbal communication, such as facial expressions and gestures. There is a need for scalable, reliable, and accurate instruments for measuring head movements directly within the context of social interactions. In this study, we used computer vision and machine learning to examine the head movement patterns of neurotypical and autistic individuals during naturalistic, face–to–face conversations, at both the individual (monadic) and interpersonal (dyadic) levels. Our model predicts diagnostic status using dyadic head movement data with an accuracy of 80%, highlighting the value of head movement as a marker of social communication. The monadic data pipeline had lower accuracy (69.2%) compared to the dyadic approach, emphasizing the importance of studying back-and-forth social communication cues within a true social context. The proposed classifier is not intended for diagnostic purposes, and future research should replicate our findings in larger, more representative samples.

Head Movement Patterns during Face-to-Face Conversations Vary with Age

Denisa Qori McDonald, Casey J. Zampella, Evangelos Sariyanidi, Aashvi Manakiwala, Ellis DeJardin, John D. Herrington, Robert T. Schultz, and Birkan Tunç.
International Conference on Multimodal Interaction (ICMI) Companion, 2022. ICMI Workshop on Bridging Social Sciences and AI for Understanding Child Behavior (WoCBU).

Key Contribution: Interpretable modeling of conversational head movement showing that dyadic representations better capture developmental signal than monadic ones.

PDF DOI Slides

Advances in computational behavior analysis have the potential to increase our understanding of behavioral patterns and developmental trajectories in neurotypical individuals, as well as in individuals with mental health conditions marked by motor, social, and emotional difficulties. This study focuses on investigating how head movement patterns during face–to–face conversations vary with age from childhood through adulthood. We rely on computer vision techniques due to their suitability for analysis of social behaviors in naturalistic settings, since video data capture can be unobtrusively embedded within conversations between two social partners. The methods in this work include unsupervised learning for movement pattern clustering, and supervised classification and regression as a function of age. The results demonstrate that 3–minute video recordings of head movements during conversations show patterns that distinguish between participants that are younger vs. older than 12 years with accuracy. Additionally, we extract relevant patterns of head movement upon which the age distinction was determined by our models.

Automatic Measurement of Social Gaze During Naturalistic Conversations in Autism

Yankowitz, L., Pargi, M.K., Dejardin, E., Zampella, C.J., Guthrie, W., Pandey, J., Bartley Jr, G.K., Chen, D., McDonald, D.Q., Manakiwala, A. Khanna, M., Keen, K., Buboltz, G., Yang, A., Herrington, J., Sariyanidi, E., Schultz, R.T., and Tunc, B.
Pre-print, 2024.

Key Contribution: Develops an AI-derived social gaze measurement framework from naturalistic dyadic video, achieving 73% cross-validated diagnostic classification accuracy.

PDF DOI

Standardized, granular measurement of autistic behaviors, such as social gaze during interactions, is needed for a range of clinical applications including diagnosis and detecting clinical change. Computational approaches show promise in automatically measuring social behaviors within natural settings. This study aims to automatically measure social gaze features from videos of dyadic conversations, characterize autism-related differences, and perform individual-level diagnostic classification. 46 autistic Participants and 36 neurotypical Participants, aged 8-29 years, engaged in a brief video-recorded conversation with a research staff member (Partner). A deep learning AI model trained to detect whether each partner was looking at the other achieved 89% cross-validated accuracy. Comparing these automatic gaze measurements, autistic Participants spent less time looking at Partners and engaging in mutual gaze than neurotypical Participants did. They also initiated mutual gaze less frequently and had shorter mutual gaze episodes, but did not differ in mutual gaze counts. An AI-derived social gaze summary score correlated specifically with ADOS-2 Social Affect scores and not Restricted and Repetitive Behavior scores. Cross-validated machine learning using gaze features predicted diagnostic group with 73% accuracy. This study provides a framework for automatically quantifying social gaze behaviors, with potential for enhancing diagnostic precision and tracking therapeutic progress in autism.

Comparing Eye Contact in Autistic and Neurotypical People during Naturalistic Social Conversation

Ellis DeJardin, Denisa Qori McDonald, Casey J. Zampella, Aashvi Manakiwala, Gabirella Buboltz, Kelsey Keen, Maya Khanna, Mohan Kashyap-Pargi, Evangelos Sariyanidi, John D. Herrington, Robert T. Schultz, and Birkan Tunç.
International Society for Autism Research (INSAR), 2023. Conference Abstract.

Key Contribution: Analyzes eye-gaze behavior during naturalistic three-minute conversations to establish foundational social interaction baselines that inform automated behavioral measurement approaches.

PDF

Physiological Signal Modeling and Neuroadaptive Systems

Work on recovering stable individual structure from noisy physiological signals through careful feature construction, temporal resolution, and evaluation design, as a step toward building human-centered adaptive systems based on brain data.

User Identification from fNIRS Brain Data Using Deep Learning

Denisa Qori McDonald and Erin T. Solovey.
Proc. of Neuroadaptive Technology Conference, 2017

Key Contribution: Demonstrates the recovery of stable individual structure from resting-state physiological data, achieving 63% subject identification accuracy while highlighting the importance of temporal resolution in feature construction.

PDF Conference Link

This paper discusses the potential of functional near-infrared spectroscopy (fNIRS) brain-computer interfaces (BCIs) to identify an individual using only her brain data. fNIRS is a lightweight, portable, non-invasive functional neuroimaging tool that uses light to capture hemodynamic responses in the brain. We show that among 30 subjects, it is possible to determine the subject from whom a segment of the fNIRS data originated with 63% accuracy. Random chance is 3.3% for 30 subjects. Additionally, we explore the effect of the fNIRS brain data window size used during feature construction, on the classification accuracy.

Semantically Far Inspirations Considered Harmful: Accounting for Cognitive States in Collaborative Ideation

Joel Chan, Kanya “Pao” Siangliulue, Denisa Qori, Ruixue Liu, Reza Moradinezhad, Safa Aman, Erin Solovey, Krzysztof Gajos, and Steven Dow.
Proc. of ACM Creativity and Cognition, 2017. Acceptance Rate: 25%.

Key Contribution: Correlates behavioral and neuroimaging data to demonstrate that introducing “far” inspirational ideas during productive ideation can negatively impact ideation speed without improving novelty.

PDF DOI

Collaborative ideation systems can help people generate more creative ideas by exposing them to ideas different from their own. However, there are competing theoretical views on whether and when such exposure is helpful. Associationist theory suggests that exposing ideators to ideas that are semantically far from their own maximizes novel combinations of ideas. In contrast, SIAM theory cautions that systems should offer far ideas only when ideators reach an impasse (a cognitive state in which they have exhausted ideas within a particular category), and offer near ideas during productive ideation (a cognitive state in which they are actively exploring ideas within a category), which maximizes exploration within categories. Our research compares these theoretical recommendations. In an online experiment, 245 participants generated ideas for a themed wedding; we detected and validated participants’ cognitive states using a combination of behavioral and neuroimaging data. Receiving far ideas during productive ideation resulted in slower ideation and less within-category exploration, without significant benefits for novelty, compared to receiving no inspirations. Participants were also more likely to hit an impasse when receiving far ideas during productive ideation. These findings suggest that far inspirational ideas can harm creativity if received during productive ideation.

Workshop Papers: Computing and Society

Shorter papers (peer-reviewed) exploring emerging applications, design questions, and broader implications of AI, sensing, and human-centered systems.

Exploring Use of AR and Soft Knitted Sensor Technology for Co-located Parent-Child Quality Time.


Shruti Mahajan, Denisa Qori McDonald, Richard James Vallett, Fannie Liu, and Erin Solovey.
ACM International Conference on Supporting Group Work (GROUP), 2022. Workshop Paper.

Key Contribution: Explores the integration of soft knitted capacitive sensors with augmented reality to facilitate interactive, co-located engagement between parents and children.

PDF

Communicating emotions and spending quality time with parents is an essential component of childrens’ wellbeing and development. Existing technology like smartphones often replaces quality time or creates distractions that take away from communication and connection, partially due to the fact that they are not integrated into the home environment. We explore the use of touch-sensitive fabrics and augmented reality (AR) visuals to create an interactive environment for parent-child communication to promote co-location, play and comfort in an unintrusive way at home. Through participatory design with children and parents, we hope to develop designs that will encourage children to express themselves and talk about their feelings with their parents.

Mental Health Markers in Language and Brain Data: Potential Diagnostic Use and Privacy Concerns

Denisa Qori McDonald, Rachel Greenstadt, Girija Kaimal, and Erin T. Solovey.
At 2nd Symposia on Computing and Mental Health, 2017. Workshop Paper and Poster Presentation.

Key Contribution: Analyzes how machine learning applied to language and brain data can reveal sensitive mental-health markers, contrasting the diagnostic potential of these models with the severe privacy risks of unintended inference.

PDF Poster

This paper discusses technology’s potential role in the inadvertent leaking of information related to mental health conditions, a particularly sensitive and legally protected part of one’s identity. Social media as well as emerging technologies such as brain-computer interfaces (BCIs) are changing the way that we interact with each other and the world. They also offer new windows into deeply personal and previously private aspects of our identity, and may leak this information. For example, with the vast amounts of public data on sites such as Twitter, we can now identify individuals even when they are using anonymous user accounts [35]. The internal state of an individual’s neural activity is, in many senses, the most private of personal data. Until recently it was impossible or highly inconvenient to gain access to this type of information. With these barriers lowering rapidly, it is critical to look carefully at the privacy implications.

AgileFood: Facilitating Adaptive Food Donation to Address Hunger & Reduce Waste

Denisa Qori McDonald, and Erin Solovey.
Proc. of Designing Sustainable Food Systems Workshop, 2017. Workshop Paper and Presentation.

Key Contribution: Proposes an adaptive scheduling system that matches grocery-store surplus with dynamic charity requirements to reduce food waste while improving local food donation logistics.

PDF Slides

In this paper, we describe AgileFood, a scheduling app designed to facilitate the distribution of food that would otherwise go to waste from grocery stores, to people that need it. There is a considerable amount of food wasted in developed countries, from which many hungry and malnourished people would benefit. Moreover, food waste creates problems for the environment and
economy. Therefore, we need to find effective ways for grocery stores to donate food to people who need it, rather than throw it away when the food approaches its best before date. We have created a system that makes it easier for grocery stores and local charities to schedule food exchange, depending on the changing charities’ needs, the grocery stores’ availability, and their location.

Dissertation

Long-form scholarly work that develops the broader conceptual and technical foundations of my research areas.

On the Real–World Interactivity Potential of Minimalistic Knitted Sensors at the Intersection of Artificial Intelligence and User Experience

Denisa Qori McDonald.
ProQuest One Academic (2628161494), 2022. Dissertation.

Available from Dissertations & Theses @ Drexel University (Order No. 28963281).
Key Contribution: Develops the conceptual and technical architecture for expanding the real-world interaction potential of minimalistic knitted sensors through representation learning and temporal modeling.

PDF DOI

Recent work has shown the feasibility of producing knitted capacitive touch sensors through digital fabrication, relying on a single conductive yarn and two external connections located at each endpoint. This technique increases their robustness and usability, while shifting the complexity of enabling interactivity from the hardware to computational models. The application of algorithmic and artificial intelligence models to these novel pervasive technologies is key to unfolding their potential, particularly when real-world and user experience considerations are also included. To bring this technology closer to real-world use, this dissertation goes beyond previous work on coarse touch discrimination to enable fine, accurate touch localization and complex gesture recognition on such low-profile knitted sensors. Deep learning, algorithmic, and computational models are presented to analyze noisy time–series signal data, which are able to capture the temporal behavior of the sensor and relevant local features. Furthermore, several user studies are conducted to train these models, demonstrate their generalizability with new users, and investigate their robustness when exposed to everyday use events. To start shaping the future of touch-sensitive fabric technology according to user expectations and everyday use scenarios, through a formative focus group study, users’ views of these fabrics are explored in different contexts. The contributions of this work set the foundations for creating pervasive interactive systems powered by artificial intelligence models that use minimalistically-designed knitted sensors as an input medium.