Advertisement

Control chart methods for monitoring surgical performance: A case study from gastro-oesophageal surgery

      Abstract

      Graphical methods are becoming increasingly used to monitor adverse outcomes from surgical interventions. However, uptake of such methods has largely been in the area of cardiothoracic surgery or in transplants with relatively little impact made in surgical oncology. A number of the more commonly used graphical methods including the Cumulative Mortality plot, Variable Life-Adjusted Display, Cumulative Sum (CUSUM) and funnel plots will be described. Accounting for heterogeneity in case-mix will be discussed and how ignoring case-mix can have considerable consequences. All methods will be illustrated using data from the Scottish Audit of Gastro-Oesophageal Cancer services (SAGOCS) data set.

      Keywords

      To read this article in full you will need to make a payment

      References

        • Grunkmeier G.L.
        • Wu Y.X.
        • Furnary A.P.
        Cumulative sum techniques for assessing surgical results.
        Ann Thorac Surg. 2003; 76: 663-667
        • Treasure T.
        • Lovegrove J.
        • Valencia O.
        • Gallivan S.
        The influence of the surgeon on outcome.
        Best Pract Res Clin Anaesthesiol. 1999; 13: 295-306
        • Spiegelhalter D.J.
        • Grigg O.
        • Kinsman R.
        • Treasure T.
        Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery.
        Int J Qual Health Care. 2003; 15: 7-13
        • Blackstone E.
        Monitoring surgical performance.
        J Thorac Cardiovasc Surg. 2004; 128: 807-810
        • Treasure T.
        • Valencia O.
        • Sherlaw-Johnson C.
        • Gallivan S.
        Surgical performance measurement.
        Health Care Manag Sci. 2002; 5: 243-248
        • Woodall W.
        The use of control charts in health-care and public health surveillance.
        J Qual Technol. 2006; 38: 88-103
        • Thor J.
        • Lundberg J.
        • Ask J.
        • et al.
        Application of statistical process control in healthcare improvement: systematic review.
        Qual Saf Health Care. 2007; 16: 387-399
        • Mohammed M.A.
        • Cheng K.K.
        • Rouse A.
        • Marshall T.
        Bristol, Shipman, and clinical governance: Shewart’s forgotten lessons.
        Lancet. 2001; 357: 463-467
        • Klein A.A.
        • Nashef S.A.M.
        Perception and reporting of cardiac surgical performance.
        Semin Cardiothorac Vasc Anesth. 2008; 12: 184-190
        • Albert A.A.
        • Walter J.A.
        • Arnrich B.
        • et al.
        On-line variable life-adjusted displays with internal and external risk-adjusted mortalities. A valuable method for benchmarking and early detection of unfavourable trends in cardiac surgery.
        Eur J Cardiothorac Surg. 2004; 25: 312-319
        • Novick R.J.
        • Fox S.A.
        • Stitt L.W.
        • Forbes T.L.
        • Steiner S.H.
        Direct comparison of risk-adjusted and non-risk-adjusted CUSUM analyses of coronary artery bypass surgery outcomes.
        J Thorac Cardiovasc Surg. 2006; 132: 386-391
        • Poloniecki J.
        • Sismanidis C.
        • Bland M.
        • Jones P.
        Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups.
        BMJ. 2004; 328: 375
        • Beiles C.B.
        • Morton A.P.
        Cumulative sum control charts for assessing performance in arterial surgery.
        ANZ J Surg. 2004; 74: 146-151
        • Lovegrove J.
        • Valencia O.
        • Treasure T.
        • Sherlaw-Johnson C.
        • Gallivan S.
        Monitoring the results of cardiac surgery by variable life-adjusted display.
        Lancet. 1997; 350: 1128-1130
        • Rogers C.A.
        • Reeves B.C.
        • Caputo M.
        • Ganesh J.S.
        • Bonser R.S.
        • Angelini G.D.
        Control chart methods for monitoring cardiac surgical performance and their interpretation.
        J Thorac Cardiovasc Surg. 2004; 128: 811-819
        • Collett D.
        • Sibanda N.
        • Piolo S.
        • Dradley J.A.
        • Rudge C.
        The UK scheme for mandatory continuous monitoring of early transplant outcome in all kidney transplant centers.
        Transplantation. 2009; 88: 970-975
        • Axelrod D.A.
        • Guidinger M.K.
        • Metzger R.A.
        • Wiesner R.H.
        • Webb R.L.
        • Merion R.M.
        Transplant center quality assessment using a continuously updatable, risk-adjusted technique (CUSUM).
        Am J Transplant. 2006; 6: 313-323
        • Sibanda T.
        • Sibanda N.
        The CUSUM chart method as a tool for continuous monitoring of clinical outcomes using routinely collected data.
        BMC Med Res Meth. 2007; 7: 46
        • Sibanda T.
        • Sibanda N.
        • Siassakos D.
        • et al.
        Prospective evaluation of a continuous monitoring and quality improvement system for reducing adverse neonatal outcomes.
        Am J Obstet Gynecol. 2009; 201: 480.e1-480.e6
        • Lane S.
        • Weeks A.
        • Ccholefield H.
        • Alfirevic Z.
        Monitoring obstetricians’ performance with statistical process charts.
        BJOG. 2007; 114: 614-618
        • Bowles T.A.
        • Watters D.A.
        Time to CUSUM: simplified reporting of outcomes in colorectal surgery.
        ANZ J Surg. 2007; 77: 587-591
        • Choi D.H.
        • Jeong W.K.
        • Lim S.W.
        • et al.
        Learning curves for laparoscopic sigmoidectomy used to manage curable sigmoid colon cancer: single-institute, three-surgeon experience.
        Surg Endosc. 2009; 23: 622-628
        • Dinçler S.
        • Koller M.T.
        • Steurer J.
        • Bachmann L.M.
        • Christen D.
        • Buchmann P.
        Multidimensional analysis of learning curves in laparoscopic sigmoid resection: eight-year results.
        Dis Colon Rectum. 2003; 46: 1371-1378
        • Tekkis P.P.
        • Senagore A.J.
        • Delaney C.P.
        • Fazio V.W.
        Evaluation of the learning curve in laparoscopic colorectal surgery: comparison of right-sided and left-sided resections.
        Ann Surg. 2005; 242: 83-91
        • Ramsay C.R.
        • Grant A.M.
        • Wallace S.A.
        • Garthwaite P.H.
        • Monk A.F.
        • Russell I.T.
        Statistical assessment of the learning curves of health technologies.
        Health Technol Assess. 2001; 5: 1-79
        • East J.M.
        • Valentine C.S.P.
        • Kanchev E.
        • Blake G.O.
        Sentinel lymph node biopsy for breast cancer using methylene blue dye manifests a short learning curve among experienced surgeons: a prospective tabular cumulative sum (CUSUM) analysis.
        BMC Surg. 2009; 9
        • Lerch L.
        • Donald J.C.
        • Olivotto I.A.
        • et al.
        Measuring surgeon performance of sentinel lymph node biopsy in breast cancer treatment by cumulative sum analysis.
        Am J Surg. 2007; 193: 556-559
        • Thompson A.M.
        • Rapson T.
        • Gilbert F.J.
        • Park K.G.
        Hospital volume does not influence long-term survival of patients undergoing surgery for oesophageal or gastric cancer.
        Br J Surg. 2007; 94: 578-584
        • Noyez L.
        Control charts, CUSUM techniques and funnel plots. A review of methods for monitoring performance in healthcare.
        Interact Cardiovasc Thorac Surg. 2009; 9: 494-499
        • Spiegelhalter D.J.
        Funnel plots for comparing institutional performance.
        Stat Med. 2005; 24: 1185-1202
        • Mayer E.K.
        • Bottle A.
        • Rao C.
        • Darzi A.W.
        • Athanasiou T.
        Funnel plots and their emerging application in surgery.
        Ann Surg. 2009; 249: 376-383
        • Page E.S.
        Continuous inspection schemes.
        Biometrika. 1954; 41: 100-115
        • Lucas J.M.
        • Crosier R.B.
        Fast initial response for CUSUM quality-control schemes: give your CUSUM a head start.
        Technometrics. 2000; 42: 102-107
        • Grigg O.
        • Farewell V.
        An overview of risk-adjusted charts.
        J R Statist Soc A. 2004; 167: 523-539
        • Grigg O.A.
        • Farewell V.T.
        • Spiegelhalter D.J.
        Use of risk-adjusted CUSUM and RSPRT charts for monitoring in medical contexts.
        Stat Methods Med Res. 2003; 12: 147-170
        • Steiner S.H.
        • Cook R.J.
        • Farewell V.T.
        Risk-adjusted monitoring of binary surgical outcomes.
        Med Decis Making. 2001; 21: 163-169
        • Birim O.
        • Kappetein A.P.
        • Waleboer M.
        • et al.
        Long-term survival after non-small cell lung cancer surgery: development and validation of a prognostic model with a preoperative and postoperative mode.
        J Thorac Cardiovasc Surg. 2006; 132: 491-498
        • Copeland G.P.
        • Jones D.
        • Walters M.
        POSSUM: a scoring system for surgical audit.
        Br J Surg. 1991; 78: 355-360
        • Parsonnet V.
        • Dean D.
        • Bernstein A.D.
        A method of uniform stratification of risk for evaluating the results of surgery in acquired adult heart disease.
        Circulation. 1989; 79: I-3-I-12
        • Steyerberg E.W.
        • Neville B.A.
        • Koppert L.B.
        • et al.
        Surgical mortality in patients with esophageal cancer: development and validation of a simple risk score.
        J Clin Oncol. 2006; 24: 4277-4284
        • Altman D.G.
        • Vergouwe Y.
        • Royston P.
        • Moons K.G.
        Prognosis and prognostic research: validating a prognostic model.
        BMJ. 2009; 338: b605
        • Royston P.
        • Moons K.G.
        • Altman D.G.
        • Vergouwe Y.
        Prognosis and prognostic research: developing a prognostic model.
        BMJ. 2009; 388: b604
        • Cummins J.
        • McCulloch P.
        ASCOT: a comprehensive clinical database for gastro-oesophageal cancer surgery.
        Eur J Surg Oncol. 2001; 27: 709-713
        • McCulloch P.
        • Ward J.
        • Tekkis P.P.
        Mortality and morbidity in gastro-oesophageal cancer surgery: initial results of ASCOT multicentre prospective cohort study.
        BMJ. 2003; 327: 1192-1197
        • Light R.J.
        • Pillemar D.B.
        Summing up: the science of reviewing research.
        Harvard University Press, Cambridge, MA1984
        • Kunadian B.
        • Dunning J.
        • Roberts A.P.
        • et al.
        Cumulative funnel plots for the early detection of interoperator variation: retrospective database analysis of observed versus predicted results of percutaneous coronary intervention.
        BMJ. 2008; 336: 931-934
        • Kunadian B.
        • Dunning J.
        • Roberts A.P.
        • Morley R.
        • de Belder M.A.
        Funnel plots for comparing performance of PCI performing hospitals and cardiologists: demonstration of utility using the New York hospital mortality data.
        Catheter Cardiovasc Interv. 2009; 73: 589-594
        • Tsang V.T.
        • Brown K.L.
        • Synnergren M.J.
        • et al.
        Monitoring risk-adjusted outcomes in congenital heart surgery: does the appropriateness of a risk model change with time?.
        Ann Thorac Surg. 2009; 87: 584-588
        • Toll D.B.
        • Janssen K.J.M.
        • Vergouwe Y.
        • Moons K.G.M.
        Validation, updating and impact of clinical prediction rules: a review.
        J Clin Epidemiol. 2008; 61: 1085-1094
        • Cook D.A.
        • Duke G.
        • Hart G.K.
        • Pilcher D.
        • Mullany D.
        Review of the application of risk-adjusted charts to analyse mortality outcomes in critical care.
        Crit Care Resusc. 2008; 10: 239-251