Talk Title: “Neural and Behavioral Mechanisms for Human-to-Human Interaction: A multimodal framework for social disorders and treatment”
Abstract: Humans are profoundly social and interactive social behaviors are essential features for well- being. Nonetheless, little is known about the neural basis for live social interactions. We investigate live face-to-face interaction as an exemplar of the basic social interaction elements. Emerging interest in human-inspired models of natural communication and potential applications for understanding clinical disorders of social function including the neurobiology that underlies the loneliness epidemic as well as issues related to social media as a replacement for live interaction motivate this investigation. Hemodynamic signals that represent neural activity acquired by two-person functional near-infrared spectroscopy, fNIRS, were acquired simultaneously with EEG, eye-tracking, pupillometry, and facial movements from dyads engaged in mutual gaze. This multi-modal dyadic paradigm has provided evidence for unique right hemisphere neural systems specialized for facial interaction (1,2) that is consistent with on- going research using TMS to provide causal evidence for live face specificity in typical participants, variations associated with autism spectrum disorder (3), and schizophrenia (4).
Similarly, live face perceptions in-person vs on-line (Zoom) activate separable neural systems (5) suggesting a neural basis for further investigation of on-line variations of social interactions. Based on this background, recent and on-going investigations are focused on the neural components for live interactions and acquire neural signals during live face-to-face gaze, eye- tracking, pupillometry, and facial movements under natural (in-person) and sparse (on-line) face- gaze (6). Findings confirm that neural-behavioral correlates are higher for the in-person than the on-line condition (p FDR corrected < 0.05) within the right parietal and ventrolateral frontal areas.
Additionally, neural correlates for eye-tracking and pupil dynamics are conjoined within the right supramarginal gyrus and pars opercularis/superior temporal regions for the [in-person > on-line] contrast (p FDR corrected 0.0025) suggesting within-region multi-variate decoding for interaction functions. Findings are consistent with a posterior-to-anterior anatomical loop, such as the right superior longitudinal fasciculus-III, with integrated mechanisms that bind multiple, rapid, and bidirectional interactive functions in the human brain.
Wang, et al., A Neural Mechanism for Natural Face-to-Face Interactions: submitted, 2026.
Noah, J. A., Zhang, X., Dravida, S., Ono, Y., Naples, J. A., McPartland, J. C., & Hirsch, J. (2020). Real-time eye-to-eye contact is associated with cross-brain neural coupling in angular gyrus. Frontiers in Human Neuroscience 14(19), 1-10. doi: 10.3389/fnhum.2020.00019
Kelley, M., Noah, J.A., Zhang, X., Scassellati, B., & Hirsch, J. (2021). Comparison of human social brain activity during eye-contact with another human and a humanoid robot. Frontiers in Robotics and AI, 7, 209. doi: 10.3389/frobt.2020.599581
Hirsch J, Zhang X, Noah JA, Dravida S, Naples A, Tiede M, Wolf, J, McPartland, J. (2022) Neural correlates of eye contact and social function in autism spectrum disorder. PLoS ONE 17(11): e0265798. https://doi.org/10.1371/journal.pone.0265798
Singh R, Zhang Y, Bhaskar D, Srihari V, Tek C, Zhang X, Noah JA, Krishnaswamy S and Hirsch
J (2025) Deep multimodal representations and classification of first-episode psychosis via live face processing. Front. Psychiatry 16:1518762. doi: 10.3389/fpsyt.2025.1518762. Purushothaman, d. et al.,A multimodal approach to identify atypical face processing systems in schizophrenia, submitted, 2026.
Zhao, N., Zhang, X., Noah, J.A., Tiede, M., and Hirsch, J. (2023) Separable processes for live “in-person” and live “zoom-like” faces. Imaging Neuroscience. 2023; 1 1–17. doi: https://doi.org/10.1162/imag_a_00027.
Bio: Emerging interest in human-inspired models of natural communication and increasing needs for
treatments for social disorders motivate investigations of the neurobiological mechanisms that
underlie live human-to-human interactions. Professor Hirsch and her laboratory aim to
understand the fundamental neural mechanisms for this new “neuroscience of two” Her
laboratory has pioneered the development of novel two-person live paradigms to investigate
neural responses during live face-to-face interactions that elicit natural communicative bi-
directional behaviors such as micro-expressions and spontaneous language using functional
neuroimaging (near infrared spectroscopy) with concurrently acquired measures of eye-tracking,
pupillometry, EEG, facial classifications, subjective perceptual and sensory reports. as well as
spoken language, and acoustic measures of prosody. Groundbreaking findings include previously
undescribed neuroanatomical substrates for integration of diverse and complex signals for on-
going computations during natural real-time face-to-face interactions; and also insight and
models for treatments related to autism spectrum disorder, schizophrenia, loneliness, depression,
and others.