Open-Source NLP Systems for Identifying Social Support and Isolation in Psychiatric Notes | HackerNoon

Table of Links

Abstract and 1. Introduction

2 Data

2.1 Data Sources

2.2 SS and SI Categories

3 Methods

3.1 Lexicon Creation and Expansion

3.2 Annotations

3.3 System Description

4 Results

4.1 Demographics and 4.2 System Performance

5 Discussion

5.1 Limitations

6 Conclusion, Reproducibility, Funding, Acknowledgments, Author Contributions, and References

SUPPLEMENTARY

Guidelines for Annotating Social Support and Social Isolation in Clinical Notes

Other Supervised Models

6 CONCLUSION

We offer two open-source NLP systems with different approaches, as well as a manual annotation guideline for identifying SS and SI. The rule-based approach and the LLM approach each have strengths and limitations in performing this challenging task of creating a portable system to identify fine and coarse-grained categories of SS and SI in psychiatric patients.

REPRODUCIBILITY

All code for rule- and LLM-based systems are available on GitHub.

Link: https://github.com/CornellMHILab/Social Support Social Isolation Extraction.

The annotation guidelines and lexicons are available in the supplementary files.

FUNDING

This study was funded in part by grants from the National Institutes of Health (R01 MH119177, R01 MH121907, and R01 MH121922).

ACKNOWLEDGEMENTS

This work was supported in part through the computational and data resources and staff expertise provided by Scientific Computing and Data at the Icahn School of Medicine at Mount Sinai Health System and supported by the Clinical and Translational Science Awards (CTSA) grant UL1TR004419 from the National Center for Advancing Translational Sciences. Research reported in this publication was also supported by the Office of Research Infrastructure of the National Institutes of Health under award number S10OD026880 and S10OD030463. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

AUTHOR CONTRIBUTIONS

BGP and LAL worked on planning, data collection, data annotation, data extraction, software development, writing, and editing. PR worked on LLM implementation and editing. VV helped in data annotation, writing, and editing. MMS, BF, and GH helped in data annotation. JP and PA helped in planning, advising, and editing. MW, JJM, and MO helped in initial category identification, lexicon development, advising, and editing. JASR, ER, JMB, IL, GNN, AT, and AWC helped in advising and editing. JP and AWC provided resources and funding.

REFERENCES

[1] World Health Organization. Social determinants of health. https://www.who.int/ health-topics/social-determinants-of-health#tab=tab_1. Online; accessed 29 October 2023.

[2] Braja G Patra, Mohit M Sharma, Veer Vekaria, Prakash Adekkanattu, Olga V Patterson, Benjamin Glicksberg, Lauren A Lepow, Euijung Ryu, Joanna M Biernacka, Al’ona Furmanchuk, et al. Extracting social determinants of health from electronic health records using natural language processing: a systematic review. Journal of the American Medical Informatics Association, 28(12):2716–2727, 2021.

[3] Maria Polikandrioti. Perceived social isolation in heart failure. The Journal of Innovations in Cardiac Rhythm Management, 13(6):5041, 2022.

[4] Wengen Zhu, Yuzhong Wu, Yuanyuan Zhou, Weihao Liang, Ruicong Xue, Zexuan Wu, Dexi Wu, Jiangui He, Yugang Dong, and Chen Liu. Living alone and clinical outcomes in patients with heart failure with preserved ejection fraction. Psychosomatic Medicine, 83(5):470–476, 2021.

[5] Tin-Kwang Lin, Bo-Cheng Hsu, Yi-Da Li, Chi-Hsien Chen, Jiunn-Wen Lin, Chen-Yu Chien, and Chia-Ying Weng. The impact of sources of perceived social support on readmissions in patients with heart failure. Journal of Psychosomatic Research, 154:110723, 2022.

[6] Maddi Olano-Lizarraga, Sara Wallstr¨om, Jes´us Mart´ın-Mart´ın, and Axel Wolf. Causes, experiences and consequences of the impact of chronic heart failure on the person´ s social dimension: a scoping review. Health & Social Care in the Community, 30(4):e842–e858, 2022.

[7] Crystal W Cen´e, Theresa M Beckie, Mario Sims, Shakira F Suglia, Brooke Aggarwal, Nathalie Moise, Monik C Jim´enez, Bamba Gaye, and Louise D McCullough. Effects of objective and perceived social isolation on cardiovascular and brain health: A scientific statement from the american heart association. Journal of the American Heart Association, 11(16):e026493, 2022.

[8] Ryo Naito, Martin McKee, Darryl Leong, Shrikant Bangdiwala, Sumathy Rangarajan, Shofiqul Islam, and Salim Yusuf. Social isolation as a risk factor for all-cause mortality: Systematic review and meta-analysis of cohort studies. PloS one, 18(1):e0280308, 2023.

[9] US Surgeon General. Our epidemic of loneliness and isolation. The US Surgeon General’s Advisory on the Healing Effects of Social Connection and Community 2023, 2023.

[10] John T Cacioppo and Stephanie Cacioppo. Social relationships and health: The toxic effects of perceived social isolation. Social and personality psychology compass, 8(2):58–72, 2014.

[11] Myung Hyun Kim, Ji Hyun An, Hye Rin Lee, Seo Hyun Jeong, Soo Jin Hwang, and Jin Pyo Hong. Social isolation, loneliness and their relationships with mental health status in south korea. Psychiatry Investigation, 18(7):652, 2021.

[12] Priya J Wickramaratne, Tenzin Yangchen, Lauren Lepow, Braja G Patra, Benjamin S Glicksberg, Ardesheer Talati, Prakash Adekkanattu, Euijung Ryu, Joanna M Biernacka, Alexander Charney, et al. Social connectedness as a determinant of mental health: A scoping review. PLOS ONE, 17:1–30, 2022. 2

[13] Sara J Czaja, Jerad H Moxley, and Wendy A Rogers. Social support, isolation, loneliness, and health among older adults in the prism randomized controlled trial. Frontiers in Psychology, 12:728658, 2021.

[14] M Lamblin, C Murawski, S Whittle, and A Fornito. Social connectedness, mental health and the adolescent brain. Neuroscience & Biobehavioral Reviews, 80:57–68, 2017.

[15] Carrie Purcell, Susie Smillie, Michele Hilton Boon, Sharon A Simpson, and Rod S Taylor. Effectiveness of social network interventions to support cardiac rehabilitation and secondary prevention in the management of people with heart disease. Cochrane Database of Systematic Reviews, 2021(1), 2021.

[16] Rosanne Freak-Poli, Joanne Ryan, Johannes T Neumann, Andrew Tonkin, Christopher M Reid, Robyn L Woods, Mark Nelson, Nigel Stocks, Michael Berk, John J McNeil, et al. Social isolation, social support and loneliness as predictors of cardiovascular disease incidence and mortality. BMC geriatrics, 21(1):1–14, 2021.

[17] M Robin DiMatteo. Social support and patient adherence to medical treatment: a metaanalysis. Health psychology, 23(2):207, 2004.

[18] M Robin DiMatteo. Variations in patients’ adherence to medical recommendations: a quantitative review of 50 years of research. Medical care, pages 200–209, 2004.

[19] Theodore F Robles and Janice K Kiecolt-Glaser. The physiology of marriage: Pathways to health. Physiology & behavior, 79(3):409–416, 2003.

[20] Bert N Uchino. Social support and health: a review of physiological processes potentially underlying links to disease outcomes. Journal of behavioral medicine, 29:377–387, 2006.

[21] Camelia E Hostinar, Regina M Sullivan, and Megan R Gunnar. Psychobiological mechanisms underlying the social buffering of the hypothalamic–pituitary–adrenocortical axis: A review of animal models and human studies across development. Psychological bulletin, 140(1):256, 2014

[22] Julianne Holt-Lunstad. The potential public health relevance of social isolation and loneliness: Prevalence, epidemiology, and risk factors. Public Policy & Aging Report, 27(4):127–130, 2017.

[23] Elisa M Trucco, Nicole M Fava, Michelle G Villar, Mari Kumar, and Matthew T Sutherland. Social isolation during the covid-19 pandemic impacts the link between child abuse and adolescent internalizing problems. Journal of youth and adolescence, pages 1–12, 2023.

[24] Hena Thakur, Morgan Stutts, Jae Wan Choi, Jeff R Temple, and Joseph R Cohen. Adolescent loneliness during the covid-19 pandemic: the role of pre-pandemic risk factors. Child Indicators Research, 16(2):617–639, 2023.

[25] Hannah P Truong, Alina A Luke, Gmerice Hammond, Rishi K Wadhera, Mat Reidhead, and Karen E Joynt Maddox. Utilization of social determinants of health icd-10 z-codes among hospitalized patients in the united states, 2016-2017. Medical care, 58(12):1037, 2020.

[26] Vivienne J Zhu, Leslie A Lenert, Brian E Bunnell, Jihad S Obeid, Melanie Jefferson, and Chanita Hughes Halbert. Automatically identifying social isolation from clinical narratives for patients with prostate cancer. BMC medical informatics and decision making, 19(1):1–9, 2019.

[27] Hadi Kharrazi, Laura J Anzaldi, Leilani Hernandez, Ashwini Davison, Cynthia M Boyd, Bruce Leff, Joe Kimura, and Jonathan P Weiner. The value of unstructured electronic health record data in geriatric syndrome case identification. Journal of the American Geriatrics Society, 66(8):1499–1507, 2018.

[28] Jeffrey L Greenwald, Patrick R Cronin, Victoria Carballo, Goodarz Danaei, and Garry Choy. A novel model for predicting rehospitalization risk incorporating physical function, cognitive status, and psychosocial support using natural language processing. Medical Care, 55(3):261– 266, 2017.

[29] Sunil Bhatt, P Connor Johnson, Netana H Markovitz, Tamryn Gray, Ryan D Nipp, Nneka Ufere, Julia Rice, Matthew J Reynolds, Mitchell W Lavoie, Madison A Clay, et al. The use of natural language processing to assess social support in patients with advanced cancer. The Oncologist, 28(2):165–171, 2023.

[30] Amol S Navathe, Feiran Zhong, Victor J Lei, Frank Y Chang, Margarita Sordo, Maxim Topaz, Shamkant B Navathe, Roberto A Rocha, and Li Zhou. Hospital readmission and social risk factors identified from physician notes. Health services research, 53(2):1110–1136, 2018.

[31] Liqin Wang, Joshua Lakin, Clay Riley, Zfania Korach, Laura N Frain, and Li Zhou. Disease trajectories and end-of-life care for dementias: latent topic modeling and trend analysis using clinical notes. In AMIA Annual Symposium Proceedings, volume 2018, page 1056. American Medical Informatics Association, 2018.

[32] Marco Guevara, Shan Chen, Spencer Thomas, Tafadzwa L Chaunzwa, Idalid Franco, Benjamin H Kann, Shalini Moningi, Jack M Qian, Madeleine Goldstein, Susan Harper, et al. Large language models to identify social determinants of health in electronic health records. NPJ digital medicine, 7(1):6, 2024.

[33] Kevin Lybarger, Oliver J Bear Don’t Walk IV, Meliha Yetisgen, and Ozlem Uzuner. Advance- ¨ ments in extracting social determinants of health information from narrative text, 2023.

[34] Manuel Barrera Jr. Distinctions between social support concepts, measures, and models. American journal of community psychology, 14(4):413–445, 1986.

[35] Lixia Ge, Chun Wei Yap, Reuben Ong, and Bee Hoon Heng. Social isolation, loneliness and their relationships with depressive symptoms: a population-based study. PloS one, 12(8):e0182145, 2017.

[36] Christopher M Masi, Hsi-Yuan Chen, Louise C Hawkley, and John T Cacioppo. A metaanalysis of interventions to reduce loneliness. Personality and social psychology review, 15(3):219–266, 2011.

[37] Catherine A Heaney and Barbara A Israel. Social networks and social support. Health behavior and health education: Theory, research, and practice, 4:189–210, 2008. [38] James S House. Social support and social structure. Sociological forum, 2(1):135–146, 1987.

[39] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 26, 2013.

[40] NLPLAB. Brat rapid annotation tool. https://brat.nlplab.org.

[41] Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spacy: Industrialstrength natural language processing in python. Zenodo, 2020.

[42] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551, 2020.

[43] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022.

[44] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.

[45] Wenxuan Zhou, Sheng Zhang, Hoifung Poon, and Muhao Chen. Context-faithful prompting for large language models. arXiv preprint arXiv:2303.11315, 2023.

[46] Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than incontext learning. Advances in Neural Information Processing Systems, 35:1950–1965, 2022.