A wearable patch based remote early warning score (REWS) in major abdominal cancer surgery patients

Introduction: The shift toward remote patient monitoring methods to detect clinical deterioration requires testing of wearable devices in real-life clinical settings. This study aimed to develop a remote early warning scoring (REWS) system based on continuous measurements using a wearable device, and compare its diagnostic performance for the detection of deterioration to the diagnostic performance of the conventional modi ﬁ ed early warning score (MEWS). Materials and methods: The study population of this prospective, single center trial consisted of patients who underwent major abdominal cancer surgery and were monitored using routine in-hospital spotcheck measurements of the vital parameters. Heart and respiratory rates were measured continuously using a wireless accelerometer patch (HealthDot). The prediction by MEWS of deterioration toward a complication graded Clavien-Dindo of 2 or higher was compared to the REWS derived from continuous measurements by the wearable patch. Main results: A total of 103 patients and 1909 spot-check measurements were included in the analysis. Postoperative deterioration was observed in 29 patients. For both EWS systems, the sensitivity (MEWS: 0.20 95% CI: [0.13 e 0.29], REWS: 0.20 95% CI: [0.13 e 0.29]) and speci ﬁ city (MEWS: 0.96 95% CI: [0.95 e 0.97], REWS: 0.96 95% CI: [0.95 e 0.97]) were assessed. Conclusions: The diagnostic value of the REWS method, based on continuous measurements of the heart and respiratory rates, is comparable to that of the MEWS in patients following major abdominal cancer surgery. The wearable patch could detect the same amount of deteriorations, without requiring manual spot check measurements. © 2022 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).


Introduction
Despite continuous efforts and improvements in surgical techniques, postoperative patient deterioration is frequently encountered in the general surgical ward [1,2].Many studies have shown that deterioration is often preceded by abnormalities in vital signs [3e6].To anticipate on these abnormalities, regular measurements of vital parameters combined with early warning scoring systems have been adopted in hospitals.However, the best strategy is a much-debated subject, and limitations such as lack of validation, generalizability, or improvement of clinical outcomes are often discussed [7e9].A drawback of early warning scoring systems is that they are manually measured and calculated, and therefore, labor intensive.Recent advances in wearable sensors that measure vital parameters provide an opportunity to replace the current manual procedures with remote monitoring systems and potentially improve the recognition of deterioration [10].Observational studies have shown that wearables can measure abnormalities in vital signs preceding deterioration [11].Despite the clear theoretical advantages of automated early warning systems, few wearables have been tested in prospective clinical settings [11,12], even fewer have already been implemented in clinical practice [13].
The current early warning scores are based on intermittent spotcheck measurements and a response protocol based on manually calculated scores.A commonly used tool is the modified early warning score (MEWS), which assigns points to impaired vital parameters and triggers follow-up care when the score exceeds a threshold [14].The implementation of bedside devices that automatically calculate early warning scores based on manually recorded measurements results in an increase in the number of registered MEWSs [15].Developing automated early warning systems by applying machine learning techniques to large-scale clinical datasets provides alternative early warning scores [16,17].Patient data can be derived from electronic medical records (EMRs), laboratory data, and wearable sensors.
One recently developed wearable sensor is Healthdot (Philips Electronic Nederland B.V.), a small patch that is attached to the patient's left lower rib using an adhesive patch.The device has the ability to continuously, remotely and unobtrusively measure vital parameters for a period of up to two weeks, which makes it suitable for remote monitoring.In addition, this wearable device has previously been shown to be clinically acceptable in terms of accuracy [18,19].However, the clinical performance of wearable data compared to current practice in the surgical ward is unknown.
The aim of this study was to develop a remote early warning score (REWS) using vital parameters collected by a wearable sensor, and compare its diagnostic performance for the detection of deterioration to the diagnostic performance of the conventional modified early warning score (MEWS).

Study design
The TRICA study, NCT03923127, is a prospective single-center study of wearable sensors in postoperative patients in a tertiary hospital (Catharina Hospital, Eindhoven, the Netherlands).The trial was approved by the Medical Ethics Committee (Maxima Medical Center, Veldhoven, the Netherlands [W19.001]).

Patient population
All adult patients scheduled to undergo major abdominal cancer surgery between April 2019 and August 2020 were eligible to participate in the study.The exclusion criteria were: pregnancy or breastfeeding, allergy to tissue adhesives, antibiotic-resistant skin infection, active implantable devices, and any skin condition in the area of device application.With an expected complication rate of up to 25% for the different types of major abdominal cancer surgery a sample size of 143 inclusions was selected in order to collect a reasonable amount relevant complications.Patients that met the inclusion criteria were identified by their surgeon during a preoperative routine visit, patients interested in participation met with a member of the clinical trial team to be further informed about the study.Informed consent was obtained prior to the start of the study procedures.

Data collection
After the surgical procedure the investigational device Healthdot (Philips Electronic Nederland B.V.), a wearable patch of 5 Â 3 Â 1 cm that weighs 13.6 g, was applied to the patient's lower left rib on the midclavicular line.Heart rate (HR) and respiratory rate (RR) measurements were stored in the internal memory of the patch at time intervals of 8 s and 1 s, respectively.The accuracy of the device in postoperative patients has previously been established [18,19].The HR data were resampled to a 1 s interval using linear interpolation.Every 5 min, the averages (mean) of the data collected over the past 5 min were transferred to a cloud server using built-in wireless communication technology (LoRa).Immediately after surgery, the wearable patch was applied in the postanesthesia or intensive care unit, whichever was applicable to the patient.The wearable patch sensor then collected vital parameters (HR and RR) over a two-week period.Although the device can also collect data at home, only the vital parameters collected during the patients' hospital stay were considered in the present analysis.During the data collection both the regular care team and research team did not have access to the data collected by the wearable patch.
Once a patient was admitted to the general ward, MEWS scores were collected and registered in the electronic medical records (EMR) by the nursing staff as part of routine care.In general, MEWSs were measured every shift of 8 h.The local early warning score table is included in Appendix A, and the score can have a value between 0 and 19.The MEWS scores were retrospectively extracted.
All postoperative complications were retrospectively documented and graded by physicians according to the Clavien-Dindo classification [20].Complications of grade 2 or higher were considered clinically relevant deteriorations, as those patients needed at least pharmacological, surgical, endoscopic, or radiological intervention.

Pre-processing of data
HR and RR measurements extracted from the internal memory of the wearable patch were used in the present analysis.A quality index (range 0e100) was reported together with the HR and RR measurements, low-quality measurements (quality index ¼ 0) were excluded.Thereafter, the HR and RR data were down sampled to 5-min averaged data by calculating the mean of the data within the 5-min windows to match the sampling frequency of the cloud data storage transmissions.

Remote early warning score
As an alternative to the current manually collected early warning scores, the introduction of wearable sensors that automatically collect vital parameters allows the introduction of a remote early warning score (REWS).The REWS was built based solely on automatically collected HR and RR measurements from the wearable.Using the point scoring table of the local MEWS score for the HR and RR measured by the wearable patch, EWS points were assigned at every 5-min interval.(HR < 40 bpm: 2 points, HR 40e50 bpm: 1 point, HR 51e100: 0 points, HR 101e110: 1 point, HR 111e130 bpm: 2 points, HR > 130 bpm 3 points.RR < 9 rpm: 2 points, RR 9e14 rpm: 0 points, RR 15e20 rpm: 1 point, RR 21e30 rpm: 2 points, RR > 30 rpm: 3 points [19].)These scores were subsequently averaged over a 1-h period to avoid major influence of short deviations of vital parameters, resulting in the REWS, a numeric score ranging from 0 to 6.

Data analysis
The REWS was calculated retrospectively for the same time points when a MEWS score was registered in the EMR whenever wearable sensor data were available.If no MEWS was extracted from the EMR, the patient was excluded, as no comparison to the REWS could be made.
Baseline characteristics are expressed as totals and percentages in the case of binominal data or as medians and interquartile ranges (IQRs) in the case of numerical data.The significance of the differences between the deterioration and no-deterioration group was assessed using Fisher's exact test or Wilcoxon test for binominal or numerical data, respectively.
The performance of both early warning scores was then assessed by evaluating whether the patient experienced deterioration (complication CD ! 2) within 24 h after the spot-check measurement.Spotchecks within 24 h after a deterioration are excluded as deviating vital parameters shortly after deterioration are expected to be associated with the recent deterioration rather than an upcoming one.A visual representation on the in-and excluded spotchecks can be found in appendix B. The early warning scores were plotted against the occurrence of deterioration using a receiver operating characteristic (ROC) curve, which describes the discriminative power of the EWS scores for the range of all possible cutoff values [21].The 95% confidence intervals (CIs) around the area under the ROC (AUROC) were computed using 2000 stratified bootstrap replicates.Statistical comparison of the two ROC curves was performed using the Delongs test for correlated ROC curves [22].
The distributions of the early warning scores were plotted as normalized histograms, and the data were split into the deterioration (CD ! 2 complications within the next 24 h) and nodeterioration groups.For the MEWS, the cutoff value was set at ! 3 to match routine practice, as action by the medical team is undertaken when the MEWS rises above this cutoff value.The cutoff value for the REWS was selected to match the number of truepositive cases from the MEWS score.The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and 95% CI were then calculated based on these cutoffs [23,24].The analysis was performed using R (V4.0.5) and R Studio (V1.4.1717) [25,26].

Results
A total of 143 patients were enrolled in this study.After excluding 23 patients to whom no wearable sensor was applied, 2 patients without spot-check data, and 15 patients without wearable sensor data, 103 patients were included in the analysis (Fig. 1).The baseline characteristics and outcomes of the study population are summarized in Table 1.
29 out of the 103 patients(28%) experienced clinically relevant deterioration during the study period.Patients who experienced deterioration were older (median 68 vs. 63 years, p ¼ 0.01) and had a significantly longer length of hospital stay (median 13 vs. 8 days, p < 0.01) than those who did not experience deterioration.Interventions in case of deterioration included unanticipated ICU admission (n ¼ 8), pain medication (n ¼ 1), radiological (n ¼ 5), surgical (n ¼ 4) and antimicrobial (n ¼ 18) intervention.No significant differences in comorbidities were found between the groups (Appendix C).
1909 spotchecks were used in the analysis, of which 665 (35%) originated from patients who experienced deterioration during their hospital stay.A median of 16 spotchecks were obtained for each patient.For patients who experienced deterioration, a significantly higher number of spotchecks was available (median 22 vs. 15, p < 0.01), which can be explained by the significantly longer length of stay.In total, 105 spotchecks (5.5%) were collected within the 24 h prior to deterioration (complication with CD ! 2) and were thus labelled as positive cases.
To match the number of true positives of the MEWS spotchecks, the cutoff for the REWS score was placed at ! 2.4.As shown in Fig. 3 and Table 2, 1816 (95.1%) of the REWS spot-checks did not exceed this threshold.A total of 93 (4.9%) REWS spotchecks did exceed the threshold, of which 21 (22.6%) were followed by deterioration within the next 24 h.The !2.4 cutoff for the REWS results in a sensitivity of 0.20 and a specificity of 0.96.An overview of the outcomes for different REWS and MEWS cutoff values is provided in Appendix D and E.
The MEWS and REWS both exceeded their respective thresholds in 9 patients that experienced a deterioration, 1 patient only exceeded the MEWS threshold and 1 only exceeded the REWS threshold.The e 18 patients that did experience a clinical deterioration did not exceed the threshold in the 24 h prior to a deterioration for both systems.Both MEWS and REWS spotchecks had the same number of true positives by design (21 spotchecks); however, the number of false positives was higher for REWS than for MEWS (72 vs. 65 spotchecks).

Discussion
In the present study, which enrolled patients after major abdominal cancer surgery, the REWS, based on vital parameters measured by a wireless patch sensor (Healthdot) through accelerometry, was compared with routine MEWS spotchecks performed by nursing staff.The cutoff point for the REWS was chosen to match the number of true positives obtained in current practice, which resulted in comparable sensitivity and specificity for the REWS.These findings are important because early warning scores for predicting clinical deterioration are of great interest in current clinical practice.However, the collection of information required for these systems is labor intensive.Therefore, intermittent assessments with relatively large time intervals are used, although this weakens the concept of early warning of deterioration because deterioration may occur between assessments.
In the present cohort, 1816 spot-check measurements did not exceed the REWS cutoff value.These spotchecks would automatically be classified as low-risk and thus would not require any action from the nursing staff.Simultaneously, the number of spotchecks exceeding the alarm threshold increased (MEWS: n ¼ 86, REWS: n ¼ 93), leading to a larger number of patients requiring follow-up care.However, translating these results into an   impact on the net workload, the 1816 automatically and remotely performed spotchecks that did not require follow-up checks by the nurse would amply compensate for the 7 extra checks that exceeded the EWS threshold.While the exact amount of time saved is highly dependent on the future monitoring protocols that are implemented in the general wards, replacing part of or all monitoring in the general ward with an automated early warning scoring system could result in reduction of the manual EWS measurement originating workload of nursing staff.With an estimate of at least 10 min of time needed to manually measure and report early warning score variables (partly) replacing this system with a remote monitoring system could free up time and allow nursing staff to focus on other important tasks rather than on the routine measurement of EWS scores.Another notable finding is that in the present cohort, the REWS based on two continuously measured vital parameters had a comparable AUC as the standard MEWS assessments consisting of seven vital parameters.A previous study evaluating early warning scores supported the finding that HR and RR were the most important predictors for deterioration among the vital parameters in the MEWS and that simplification of the number of parameters could be justified [27].
In the literature, only a few studies have evaluated wearable sensors in a clinical setting [11,12,16].Previous clinical studies investigated comparable wearables in similar populations.Although different outcomes were analyzed, all studies reported promising results of the wearable device in terms of the detection of adverse events, decreased hospital length of stay, and 30-day readmission rate [11,12] .Alternative types of continuous sensors have also been studied, and a study using continuous SpO2 sensor measurements demonstrated improved detection of deteriorating patients [16].Continuous monitoring using a mattress sensor in the general ward decreases the length of hospital stay and reduces medical emergency rates [9].Although promising, the authors encountered significant limitations in the use of such a system when the patients gained mobility.
The present study aimed to assess the performance of REWS compared to MEWS, and other trials comparing continuous measurements with intermittent spotchecks also found promising results.Weenk and colleagues.compared continuous measurements of vital parameters with MEWS and found good agreement between both types of data collection, suggesting that the MEWS system might be replaced [28].However, they did not report the outcomes or predictive values, complicating comparison to our findings.A different and more personalized approach was analyzed by Keim and colleagues, who investigated the deviations of vital parameters from a patient's baseline using continuous measurements compared to intermittent measurements [29].They concluded that continuous monitoring could provide alert thresholds as the degree of change from a patient's own baseline.
The strengths of the present study include its prospective design and the real-world setting in a clinically relevant patient population.The wearable sensor was compared with current clinical practice, while allowing unobtrusive automated EWS assessments.Another potential advantage of wearable monitoring is that patients can move freely, which is important for early post-operative recovery.Furthermore, sleep disturbance for the measurement of vital parameters by the nursing staff is reduced.
A limitation of the present study is that it was performed at a single center and investigated the local performance of the EWS, which might impair external validity.Second, the primary outcome consisted of heterogeneous deteriorations.This heterogeneity challenged the definition of the forecast window and generalizability of patterns in vital parameters before deterioration occurred.A 24 h forecast window length was chosen because changes in vital parameters can be found up to 24 h prior to severe deterioration, and a forecast window of 24 h is commonly used in the literature [30].However, mild deterioration is assumed to impair vital parameters less than severe complications, and their detection could benefit from a shorter forecast window.Additionally, a shorter forecast window could align with the length of working shifts, which are typically 8 h in most hospitals.Third, the population size did not allow splitting into training and validation datasets.Therefore, the performance of the proposed REWS cutoff value should be evaluated using an independent dataset.Fourth, the analysis was challenged by unavailable wearable data in 15 patients.The unavailability of wearable data is caused by preprocessing issues, such as corruption of the internal flash drive of the device and problems with data synchronization.This cause of missing data will be mitigated in future introduction of the wearable device into routine clinical practice and/or its use in follow up trials, as the wirelessly sent data will be used instead of data extraction from internal memory.
The present trial is an important first step toward remote monitoring and wearable-based decision support for patients in general wards.This trial provides a ground for larger scale follow up trials that are needed to optimize and validate protocols for the use of wearable based EWS in clinical practice.In the future, wearable based remote monitoring could be further explored for home monitoring as well.The present analysis focused on the comparison of wearable data collected at the same time as the routine MEWS assessment.However, as wearable patches continuously measure vital parameters, follow-up trials can aim to develop novel, continuous wearable-based early warning algorithms.Considering the current sensitivity of both EWS systems for the detection of deteriorations it will be interesting to assess whether the use of continuous vital parameters can lead to better and/or earlier identification of deteriorating patients.

Conclusions
A remote early warning scoring system based on vital parameters measured by a wearable patch offers a predictive performance comparable to that of the MEWS in patients following major abdominal cancer surgery.The wearable patch detected the same number of true deteriorations without requiring manual spotchecks.Overall, the present study strengthens the idea that remote continuous monitoring with a wearable accelerometer sensor patch has potential as a clinical decision support tool while improving patient mobility and comfort as well as workload of nursing staff.

Fig. 2 .
Fig. 2. Receiver operating characteristic (ROC) curve for the Modified Early Warning Score (MEWS) and Remote Early Warning Score (REWS) to describe the development of a deterioration in the next 24 h.Displaced numbers are the area under the ROCcurves.Dashed line represents the random-guessing line.

Fig. 3 .
Fig. 3. Normalized histograms for the Modified Early Warning Score and Remote Early Warning Score.The colored bars show the distribution of the Early Warning Scores, and the dashed lines indicate the cutoff values.For the MEWS, the cutoff is placed at !3 based on local protocols.The cutoff value for the REWS is placed at !2.4 to match the number of true positives classified by the MEWS.

Table 1
Baseline characteristics, data availability and outcomes table.Continuous data is represented as mean (sd) or median [IQR].

Table 2
Distribution of the patients over the different areas of the Early Warning Scores.The cutoff values match the cutoff values shown in Fig. 3.A Clavien Dindo (CD) score of 0 is assigned when no deterioration is present.CD: Clavien-Dindo score, MEWS: Modified Early Warning Score, REWS: Remote Early Warning Score, PPV: positive predictive value, NPV: negative predictive value.