Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial

Nature Medicine volume 31, pages 1895–1902 (2025)Cite this article

Subjects

Abstract

The COmmunicating Narrative Concerns Entered by RNs (CONCERN) early warning system (EWS) uses real-time nursing surveillance documentation patterns in its machine learning algorithm to identify deterioration risk. We conducted a 1-year, multisite, pragmatic trial with cluster-randomization of 74 clinical units (37 intervention; 37 usual care) across 2 health systems. Eligible adult hospital encounters were included. We tested if outcomes differed between patients whose care teams were and patients whose care teams were not informed by the CONCERN EWS. Coprimary outcomes were in-hospital mortality (examined as instantaneous risk) and length of stay. Secondary outcomes were cardiopulmonary arrest, sepsis, unanticipated intensive care unit transfers and 30-day hospital readmission. Among 60,893 hospital encounters (33,024 intervention; 27,869 usual care), intervention group encounters had 35.6% decreased instantaneous risk of death (adjusted hazard ratio (HR), 0.64; 95% confidence interval (CI), 0.53–0.78; P < 0.0001), 11.2% decreased length of stay (adjusted incidence rate ratio, 0.91; 95% CI, 0.90–0.93; P < 0.0001), 7.5% decreased instantaneous risk of sepsis (adjusted HR, 0.93; 95% CI, 0.86–0.99; P = 0.0317) and 24.9% increased instantaneous risk of unanticipated intensive care unit transfer (adjusted HR, 1.25; 95% CI, 1.09–1.43; P = 0.0011) compared with usual-care group encounters. No adverse events were reported. A machine learning-based EWS, modeled on nursing surveillance patterns, decreased inpatient deterioration risk with statistical significance. ClinicalTrials.gov registration: NCT03911687.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: CONCERN EWS display and detailed prediction screen integrated into the EHR.
Fig. 2: Flow diagram of hospital encounters assessed for eligibility and allocation to intervention or usual care for in-hospital event outcomes.

Similar content being viewed by others

Data availability

We will make our de-identified outcomes data publicly available within 6 months of publication with no anticipated end date. This will include data dictionaries, study protocol, statistical analysis plan and de-identified individual shift-level and encounter-level data for our primary and secondary outcomes. We will make the data available under the Creative Commons Attribution 4.0 International License on a public repository, such as PhysioNet, that provides functionality for users to register, agree to terms of use and provide evidence of human subjects research training before downloading. Users will be able to access the data upon agreeing to the Creative Commons Attribution 4.0 International License and the public repository terms of use. Access will be granted according to the time frame for processing requests provided by the public repository. Our study website (www.concernearlywarningscore.org) will provide study team contact information and a link to the public repository to support findability. Prior to our data being uploaded on a public repository please contact the corresponding author for data requests. The CONCERN study team can also be contacted directly at [email protected]. The conditions and purposes for sharing include that requesting individuals provide proof of a training program in human research subject protections and HIPAA regulations, not being from a for-profit organization and agreeing to criteria consistent with the PhysioNet Credentialed Health Data Use Agreement 1.5.0. The CONCERN study team will process data requests within 1 month of receipt of a data sharing request that meets the above stated criteria. The CONCERN EWS algorithm is considered intellectual property that will not be shared but is described in our online supplement.

Code availability

We will share our SAS statistical analysis code, along with our data as mentioned above, within 6 months of publication on a public repository such as PhysioNet that provides functionality for users to register, agree to terms of use and provide evidence of human subjects research training before downloading. Prior to our code being uploaded on a public repository please contact the corresponding author for data requests. The conditions and purposes for sharing include that requesting individuals provide proof of a training program in human research subject protections and HIPAA regulations, not being from a for-profit organization and agreeing to criteria consistent with the PhysioNet Credentialed Health Data Use Agreement 1.5.0. The CONCERN study team will process code requests within 1 month of receipt of a code request that meets the above stated criteria.

References

  1. American Hospital Association. Report: Rising patient acuity driving up hospital costs as payments fall. AHA News https://www.aha.org/news/headline/2022-08-15-report-rising-patient-acuity-driving-hospital-costs-payments-fall (2022).

  2. Hogan, H. et al. Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study. BMJ Qual. Saf. 21, 737–745 (2012).

    PubMed  PubMed Central  Google Scholar 

  3. Patient safety. The Joint Commission www.jointcommission.org/facts_about_patient_safety/ (2015).

  4. Wan, Y.-K. J. et al. Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review. J. Am. Med. Inform. Assoc. 31, 256–273 (2023).

    PubMed  PubMed Central  Google Scholar 

  5. Schmidt, P. E. et al. Impact of introducing an electronic physiological surveillance system on hospital mortality. BMJ Qual.Saf. 24, 176–177 (2015).

    PubMed  Google Scholar 

  6. Lee, T. C., Shah, N. U., Haack, A. & Baxter, S. L. Clinical implementation of predictive models embedded within electronic health record systems: a systematic review. Informatics (MDPI) 7, 25 (2020).

    PubMed  Google Scholar 

  7. Bates, D. W., Saria, S., Ohno-Machado, L., Shah, A. & Escobar, G. Big data in health care: using analytics to identify and manage high-risk and high-cost patients. Health Aff. (Millwood) 33, 1123–1131 (2014).

    PubMed  Google Scholar 

  8. Fu, L. et al. Development and validation of early warning score system: a systematic literature review. J. Biomed. Inform. 105, 103410 (2020).

    PubMed  PubMed Central  Google Scholar 

  9. Drew, B. J. et al. Insights into the problem of alarm fatigue with physiologic monitor devices: a comprehensive observational study of consecutive intensive care unit patients. PLoS ONE 9, e110274 (2014).

    PubMed  PubMed Central  Google Scholar 

  10. Rossetti, S. et al. Healthcare process modeling to phenotype clinician behaviors for exploiting the signal gain of clinical expertise (HPM–ExpertSignals): development and evaluation of a conceptual framework. J. Am. Med. Inform. Assoc. 28, 1242–1251 (2021).

    PubMed  PubMed Central  Google Scholar 

  11. Halverson, C. C. & Scott Tilley, D. Nursing surveillance: a concept analysis. Nurs. Forum 57, 454–460 (2022).

    PubMed  Google Scholar 

  12. Collins, S. A., Fred, M. R., Wilcox, L. & Vawdrey, D. K. Workarounds used by nurses to overcome design constraints of electronic health records. NI 2012 2012, 93–97 (2012).

    PubMed  Google Scholar 

  13. Collins, S. A. et al. Relationship between nursing documentation and mortality. Am. J. Crit. Care. 22, 306–313 (2013).

    PubMed  PubMed Central  Google Scholar 

  14. Endsley, M. R. Toward a theory of situation awareness in Dynamic Systems. Hum. Factors J. Hum. Factors Ergon. Soc. 37, 32–64 (1995).

    Google Scholar 

  15. Tokareva, I. & Romano, P. Failure to rescue. PSNet https://psnet.ahrq.gov/primer/failure-rescue (2025).

  16. Odell, M., Victor, C. & Oliver, D. Nurses’ role in detecting deterioration in ward patients: systematic literature review. J. Adv. Nurs. 65, 1992–2006 (2009).

    PubMed  Google Scholar 

  17. Burns, P. B., Rohrich, R. J. & Chung, K. C. The levels of evidence and their role in evidence-based medicine. Plast. Reconstr. Surg. 128, 305–310 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  18. Agency for Healthcare Research and Quality. Rapid Response Systems https://psnet.ahrq.gov/primer/rapid-response-systems (2019).

  19. Cioffi, J. Recognition of patients who require emergency assistance: a descriptive study. Heart Lung 29, 262–268 (2000).

    CAS  PubMed  Google Scholar 

  20. Jones, L., King, L. & Wilson, C. A literature review: factors that impact on nurses’ effective use of the Medical Emergency Team (MET). J. Clin. Nurs. 18, 3379–3390 (2009).

    PubMed  Google Scholar 

  21. Collins, S. A. & Vawdrey, D. K. “Reading between the lines” of flowsheet data: nurses’ optional documentation associated with cardiac arrest outcomes. Appl. Nurs. Res. 25, 251–257 (2012).

    PubMed  Google Scholar 

  22. Opitz, D. & Maclin, R. Popular ensemble methods: an empirical study. J. Artif. Intell. Res. 11, 169–198 (1999).

    Google Scholar 

  23. Hobensack, M. et al. Identifying barriers to the implementation of Communicating Narrative Concerns Entered by Registered Nurses (CONCERN), an early warning system smartapp. Appl Clin Inform 15, 295–305 (2024).

    PubMed  PubMed Central  Google Scholar 

  24. Singh, K. et al. Evaluating a widely implemented proprietary deterioration index model among hospitalized patients with COVID-19. Ann. Am. Thorac. Soc. 18, 1129–1137 (2021).

    PubMed  PubMed Central  Google Scholar 

  25. Sashegyi, A. & Ferry, D. On the interpretation of the hazard ratio and communication of survival benefit. Oncologist 22, 484–486 (2017).

    PubMed  PubMed Central  Google Scholar 

  26. Adams, R. et al. Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis. Nat. Med. 28, 1455–1460 (2022).

    CAS  PubMed  Google Scholar 

  27. Escobar, G. J. et al. Automated identification of adults at risk for in-hospital clinical deterioration. N. Engl. J. Med. 383, 1951–1960 (2020).

    PubMed  PubMed Central  Google Scholar 

  28. Levin, M. A. et al. Real-time machine learning alerts to prevent escalation of care: a nonrandomized clustered pragmatic clinical trial. Crit. Care Med. 52, 1007–1020 (2024).

    PubMed  Google Scholar 

  29. Churpek, M. M. et al. Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Crit. Care Med. 44, 368–374 (2016).

    PubMed  PubMed Central  Google Scholar 

  30. Plana, D. et al. Randomized clinical trials of machine learning interventions in health care: a systematic review. JAMA Netw. Open 5, e2233946 (2022).

    PubMed  PubMed Central  Google Scholar 

  31. Moher, D. et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Brit. Med. J. 340, c869 (2010).

    PubMed  PubMed Central  Google Scholar 

  32. Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J. & Denniston, A. K. on behalf of SPIRIT-AI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Lancet Digit. Health 2, e537–e548 (2020).

    PubMed  PubMed Central  Google Scholar 

  33. Campbell, M. K., Piaggio, G., Elbourne, D. R. & Altman, D. G. CONSORT Group. Consort 2010 statement: extension to cluster randomised trials. Brit. Med. J. 345, e5661 (2012).

    PubMed  Google Scholar 

  34. Zwarenstein, M. et al. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. Brit. Med. J. 337, a2390 (2008).

    PubMed  PubMed Central  Google Scholar 

  35. Kiekkas, P. et al. Delayed admission to the intensive care unit and mortality of critically ill adults: systematic review and meta-analysis. Biomed. Res. Int. 2022, 4083494 (2022).

  36. Churpek, M. M. et al. Association between intensive care unit transfer delay and hospital mortality: A multicenter investigation. J. Hosp. Med. 11, 757–762 (2016).

    PubMed  Google Scholar 

  37. Solomon, R. S., Corwin, G. S., Barclay, D. C., Quddusi, S. F. & Dannenberg, M. D. Effectiveness of rapid response teams on rates of in-hospital cardiopulmonary arrest and mortality: A systematic review and meta-analysis. J. Hosp. Med. 11, 438–445 (2016).

    PubMed  Google Scholar 

  38. Hu, W., Chan, C. W., Zubizarreta, J. R. & Escobar, G. J. An examination of early transfers to the ICU based on a physiologic risk score. Manuf. Serv. Oper. Manag. 20, 531–549 (2018).

    Google Scholar 

  39. Grieve, R. et al. Analysis of benefit of intensive care unit transfer for deteriorating ward patients: a patient-centered approach to clinical evaluation. JAMA Netw. Open 2, e187704 (2019).

    PubMed  PubMed Central  Google Scholar 

  40. Hripcsak, G. & Albers, D. J. Next-generation phenotyping of electronic health records. J. Am. Med. Inform. Assoc. 20, 117–121 (2013).

    PubMed  Google Scholar 

  41. Albers, D. J. & Hripcsak, G. Using time-delayed mutual information to discover and interpret temporal correlation structure in complex populations. Chaos 22, 13111 (2012).

    CAS  Google Scholar 

  42. Albers, D. J. & Hripcsak, G. A statistical dynamics approach to the study of human health data: resolving population scale diurnal variation in laboratory data. Phys. Lett. A 374, 1159–1164 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  43. Pivovarov, R., Albers, D. J., Hripcsak, G., Sepulveda, J. L. & Elhadad, N. Temporal trends of hemoglobin A1c testing. J. Am. Med. Inform. Assoc. 21, 1038–1044 (2014).

    PubMed  PubMed Central  Google Scholar 

  44. Pivovarov, R., Albers, D. J., Sepulveda, J. L. & Elhadad, N. Identifying and mitigating biases in EHR laboratory tests. J. Biomed. Inform. 51, 24–34 (2014).

    PubMed  Google Scholar 

  45. Schwartz, J. M. et al. Factors influencing clinician trust in predictive clinical decision support systems for in-hospital deterioration: qualitative descriptive study. JMIR Hum Factors 9, e33960 (2022).

    PubMed  PubMed Central  Google Scholar 

  46. Wong, A. et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern. Med. 181, 1065–1070 (2021).

    PubMed  Google Scholar 

  47. Sankey, C. B., McAvay, G., Siner, J. M., Barsky, C. L. & Chaudhry, S. I. “Deterioration to door time”: an exploratory analysis of delays in escalation of care for hospitalized patients. J. Gen. Intern. Med. 31, 895–900 (2016).

    PubMed  PubMed Central  Google Scholar 

  48. Office of Surgeon General. Addressing Health Worker Burnout: The U.S. Surgeon General’s Advisory on Building a Thriving Health Workforce (US Department of Health and Human Services, 2022).

  49. Lewis, A. E. et al. Electronic health record data quality assessment and tools: a systematic review. J. Am. Med. Inform. Assoc. 30, 1730–1740 (2023).

    PubMed  PubMed Central  Google Scholar 

  50. Weiskopf, N. G., Bakken, S., Hripcsak, G. & Weng, C. A data quality assessment guideline for electronic health record data reuse. EGEMS (Wash DC) 5, 14 (2017).

    PubMed  Google Scholar 

  51. Singer, M. et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 315, 801–810 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  52. Yale New Haven Health Services Corporation – Center for Outcomes Research and Evaluation (YNHHSC/CORE). 2023 Hospital-Wide Readmission Measure Updates and Specifications Report – Version 12.0. (2023); https://qpp.cms.gov/resources/document/fab5e228-1667-4e8f-9b49-ea04fb3cfd81

  53. Shcherbatykh, I., Holbrook, A., Thabane, L., Dolovich, L. & COMPETE III investigators. Methodologic issues in health informatics trials: the complexities of complex interventions. J. Am. Med. Inform. Assoc. 15, 575–580 (2008).

    PubMed  PubMed Central  Google Scholar 

  54. Rossetti, S. C. et al. The Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) clinical decision support early warning system: protocol for a cluster randomized pragmatic clinical trial. JMIR Res. Protoc. 10, e30238 (2021).

    PubMed  PubMed Central  Google Scholar 

  55. Rossetti S. C. et al. Leveraging clinical expertise as a feature - not an outcome - of predictive models: evaluation of an early warning system use case. In AMIA Annual Symposium Proc. 323–332 (AMIA, 2019).

  56. Collett D. Modelling Survival Data in Medical Research 3rd edn (Chapman and Hall/CRC, 2014).

  57. Jolliffe I. T. & Stephenson D. B. Forecast Verification: A Practitioner’s Guide in Atmospheric Science (Wiley, 2012).

  58. Fu, L.-H. et al. Utilizing timestamps of longitudinal electronic health record data to classify clinical deterioration events. J. Am. Med. Inform. Assoc. 28, 1955–1963 (2021).

    PubMed  PubMed Central  Google Scholar 

  59. Abbey, E. J., Mammen, J. S. R., Soghoian, S. E., Cadorette, M. A. F. & Ariyo, P. In-hospital mortality and the predictive ability of the modified early warning score in Ghana: single-center, retrospective study. JMIRx Med. 2, e24645 (2021).

    PubMed  PubMed Central  Google Scholar 

  60. Delgado-Hurtado, J. J., Berger, A. & Bansal, A. B. Emergency department modified early warning score association with admission, admission disposition, mortality, and length of stay. J Community Hosp. Intern. Med. Perspect. 6, 31456 (2016).

    PubMed  Google Scholar 

  61. Kao, C.-C. et al. Prognostic significance of emergency department modified early warning score trend in critical ill elderly patients. Am. J. Emerg. Med. 44, 14–19 (2021).

    PubMed  Google Scholar 

  62. Kim W. Y. et al. Modified early warning score changes prior to cardiac arrest in general wards. PLoS ONE https://doi.org/10.1371/JOURNAL.PONE.0130523 (2015).

  63. Knoery, C. et al. Modified early warning score and risk of mortality after acute stroke. Clin. Neurol. Neurosurg. 202, 106547 (2021).

    PubMed  Google Scholar 

  64. Lee, J.-R. et al. Derivation and validation of modified early warning score plus SpO2/FiO2 score for predicting acute deterioration of patients with hematological malignancies. Korean J. Intern. Med. 35, 1477–1488 (2020).

    PubMed  PubMed Central  Google Scholar 

  65. Martín-Rodríguez, F. et al. Early warning scores in patients with suspected COVID-19 infection in emergency departments. J. Pers. Med. https://doi.org/10.3390/jpm11030170 (2021).

  66. Monzon, L. D. R. & Boniatti, M. M. Use of the modified early warning score in intrahospital transfer of patients. Rev. Bras. Ter. Intensiva 32, 439–443 (2020).

    PubMed  PubMed Central  Google Scholar 

  67. Takada, K. et al. Association between Intraoperative early warning score and mortality and in-hospital stay in lower gastrointestinal spontaneous perforation. Anesthesiol. Res. Pract. 2023, 8910198 (2023).

    PubMed  PubMed Central  Google Scholar 

  68. Quan, H. et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med. Care. 43, 1130–1139 (2005).

    PubMed  Google Scholar 

  69. Figueiredo, S. in Stroke Engine (eds Zeltzer, L. et al.) (Canadian Partnership for Stroke Recovery Heart & Stroke Foundation, 2009); https://strokengine.ca/en/assessments/charlson-comorbidity-index-cci

  70. Hosmer, D. W., Lemeshow, S. & May, S. Applied Survival Analysis: Regression Modeling of Time-To-Event Data (Wiley, 2008).

  71. Austin, P., Rothwell, D. & Tu, J. A comparison of statistical modeling strategies for analyzing length of stay after CABG surgery. Health Serv. Outcomes Res. Methodol. 3, 107–133 (2002).

    Google Scholar 

Download references

Acknowledgements

This study was funded by the National Institute of Nursing Research (NINR R01NR016941, COmmunicating Narrative Concerns Entered by RNs (CONCERN): Clinical Decision Support Communication for Risky Patient States) (S.C.R., P.C.D., C.K., S.C., J.W., G.L., D.A., R.Y.L., H.J., S.B., M.J.K., F.Y.C., L.Z., T.D., F.L., M.T., S.M.A.B., J.T., K.D.C.) and Reducing Health Disparities Through Informatics (T32NR007969) (J.W., R.Y.L., S.B., J.S.D.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. We thank all the nurses, prescribing providers and patients who participated in this study. We also acknowledge B. Westra for serving on our Advisory Board and the American Nurses Foundation Reimagining Nursing Initiative funding for sharing and evaluating the implementation of CONCERN EWS to additional study sites. We thank C. Stillwell for her contribution to the design of tables and figures and L. Schweig for editorial review.

Author information

Authors and Affiliations

  1. Columbia University Irving Medical Center, Department of Biomedical Informatics, New York, NY, USA

    Sarah C. Rossetti, Chris Knaplund, Jennifer Withall, David Albers, Rachel Y. Lee, Suzanne Bakken, Temiloluwa Daramola, Mai Tran & Syed Mohtashim Abbas Bokhari

  2. Columbia University Irving Medical Center, School of Nursing, New York, NY, USA

    Sarah C. Rossetti, Haomiao Jia, Suzanne Bakken & Jessica Schwartz-Dillard

  3. Brigham and Women’s Hospital, Boston, MA, USA

    Patricia C. Dykes, Graham Lowenthal, Min-Jeoung Kang, Frank Y. Chang, Li Zhou & David W. Bates

  4. Harvard Medical School, Boston, MA, USA

    Patricia C. Dykes, Min-Jeoung Kang, Li Zhou & David W. Bates

  5. Newton-Wellesley Hospital, Newton, MA, USA

    Sandy Cho

  6. University of Colorado, Anschutz Medical Campus, Department of Biomedical Informatics, Aurora, CO, USA

    David Albers

  7. Columbia University Irving Medical Center, Mailman School of Public Health, New York, NY, USA

    Haomiao Jia

  8. University of Pennsylvania, Philadelphia, PA, USA

    Fang Liu & Kenrick D. Cato

  9. Hospital for Special Surgery, New York, NY, USA

    Jessica Schwartz-Dillard

  10. Siena College, Loudonville, NY, USA

    Jennifer Thate

  11. Children’s Hospital of Philadelphia, Philadelphia, PA, USA

    Kenrick D. Cato

Authors
  1. Sarah C. Rossetti
    View author publications

    Search author on:PubMed Google Scholar

  2. Patricia C. Dykes
    View author publications

    Search author on:PubMed Google Scholar

  3. Chris Knaplund
    View author publications

    Search author on:PubMed Google Scholar

  4. Sandy Cho
    View author publications

    Search author on:PubMed Google Scholar

  5. Jennifer Withall
    View author publications

    Search author on:PubMed Google Scholar

  6. Graham Lowenthal
    View author publications

    Search author on:PubMed Google Scholar

  7. David Albers
    View author publications

    Search author on:PubMed Google Scholar

  8. Rachel Y. Lee
    View author publications

    Search author on:PubMed Google Scholar

  9. Haomiao Jia
    View author publications

    Search author on:PubMed Google Scholar

  10. Suzanne Bakken
    View author publications

    Search author on:PubMed Google Scholar

  11. Min-Jeoung Kang
    View author publications

    Search author on:PubMed Google Scholar

  12. Frank Y. Chang
    View author publications

    Search author on:PubMed Google Scholar

  13. Li Zhou
    View author publications

    Search author on:PubMed Google Scholar

  14. David W. Bates
    View author publications

    Search author on:PubMed Google Scholar

  15. Temiloluwa Daramola
    View author publications

    Search author on:PubMed Google Scholar

  16. Fang Liu
    View author publications

    Search author on:PubMed Google Scholar

  17. Jessica Schwartz-Dillard
    View author publications

    Search author on:PubMed Google Scholar

  18. Mai Tran
    View author publications

    Search author on:PubMed Google Scholar

  19. Syed Mohtashim Abbas Bokhari
    View author publications

    Search author on:PubMed Google Scholar

  20. Jennifer Thate
    View author publications

    Search author on:PubMed Google Scholar

  21. Kenrick D. Cato
    View author publications

    Search author on:PubMed Google Scholar

Contributions

S.C.R. and K.D.C. had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. More than one author (K.D.C., S.C.R., G.L., H.J., M.T., J.W. and R.Y.L.) have directly accessed and verified the underlying data reported in the paper. The research concept was designed by S.C.R. and K.D.C. System development and study design was provided by S.C.R., K.D.C., P.C.D., C.K., S.C., D.A., G.L., S.B., M.J.K., F.Y.C., L.Z., J.W., H.J., F.L. and J.S.D. Acquisition, analysis or interpretation of data was performed by S.C.R., K.D.C., P.C.D., C.K., S.C., D.A., G.L., S.B., J.W., H.J., F.L., R.Y.L., T.D., M.T., S.M.A.B. and J.T. The paper was drafted by S.C.R., R.Y.L., K.D.C. and P.C.D. All authors contributed to the critical review of the paper for important intellectual content. Statistical analysis was carried out by H.J., K.D.C. and D.A. Funding was obtained by S.C.R. and K.D.C. Administrative, technical or material support was provided by S.C.R., K.D.C., P.C.D., R.Y.L., T.D. and F.L. Supervision was carried out by S.C.R., K.D.C., P.C.D. and S.B.

Corresponding author

Correspondence to Sarah C. Rossetti.

Ethics declarations

Competing interests

Columbia University has filed a US nonprovisional patent application (US Patent Application No. 18/814,823) related to the technology that is the subject of this article. S.C.R., C.K. and K.D.C. are named inventors on the patent application and are entitled to revenue sharing with the university per the terms of the university’s patent policy. The university and the named inventors are committed to making the technology freely available upon request for academic noncommercial research purposes. Any entity interested in obtaining a license to practice the technology for commercial purposes may contact Columbia Technology Ventures at [email protected]. D.W.B. reports grants and personal fees from EarlySense, personal fees from CDI Negev, equity from ValeraHealth, equity from Clew, equity from MDClone, personal fees and equity from AESOP, personal fees and equity from FeelBetter, personal fees and equity from Guided Clinical Solutions and grants from IBM Watson Health, outside the submitted work. The other authors declare no competing interests.

Peer review

Peer review information

Nature Medicine thanks Alessio Crippa, Rupert Pearse, Jean-Louis Vincent and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Lorenzo Righetto, in collaboration with the Nature Medicine team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 CONCERN Predictive Model Conceptual Modeling Approach.

Figure adapted from ref. 10 under a Creative Commons license CC BY 40.

Extended Data Fig. 2 Percentage of Time Intervention and Usual Care Hospital Encounters Spent in Each Area 42 Hours Before Deterioration Event or Discharged Alive (N = 60,893).

This scatter plot visualizes the distribution of hours spent in the usual care (control) area (x-axis) and intervention area (y-axis) 42 h prior to an in-hospital deterioration event or being discharged alive for patients in the Control Group (blue) and Intervention Group (red).

Extended Data Table 1 Characteristics of Hospital Encounters During the Trial (N = 60,893) by Study Site
Full size table
Extended Data Table 2 Top 10 Admission and Principal Diagnosesa Across Study Sites
Full size table
Extended Data Table 3 Sensitivity Analysis for Imbalance in COVID-19 Diagnoses for Hospital Encounters Across Study Arms
Full size table
Extended Data Table 4 Count of Study Units by Type and Intervention versus Usual Care Group Across Study Sites
Full size table
Extended Data Table 5 Performance Metrics for CONCERN Predictive Modela
Full size table
Extended Data Table 6 CONCERN Model Factors and Features
Full size table
Extended Data Table 7 Statistical Power Analysis
Full size table

Supplementary information

Supplementary Information

Supplementary Table 1, Methods 1–2, Fig. 1 and References.

Reporting Summary

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rossetti, S.C., Dykes, P.C., Knaplund, C. et al. Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial. Nat Med 31, 1895–1902 (2025). https://doi.org/10.1038/s41591-025-03609-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41591-025-03609-7

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing