Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Systematic review and narrative synthesis of surgeons' perception of postoperative outcomes and risk

Dilaver, N. M., Gwilym, B. L., Preece, R., Twine, C. P. and Bosanquet, D. C. 2020. Systematic review and narrative synthesis of surgeons' perception of postoperative outcomes and risk. BJS Open 4 (1) , pp. 16-26. 10.1002/bjs5.50233

[thumbnail of Systematic review and narrative synthesis of surgeons?.pdf]
Preview
PDF - Published Version
Available under License Creative Commons Attribution.

Download (283kB) | Preview

Abstract

Background The accuracy with which surgeons can predict outcomes following surgery has not been explored in a systematic way. The aim of this review was to determine how accurately a surgeon's ‘gut feeling’ or perception of risk correlates with patient outcomes and available risk scoring systems. Methods A systematic review was undertaken in accordance with PRISMA guidelines. A narrative synthesis was performed in accordance with the Guidance on the Conduct of Narrative Synthesis In Systematic Reviews. Studies comparing surgeons' preoperative or postoperative assessment of patient outcomes were included. Studies that made comparisons with risk scoring tools were also included. Outcomes evaluated were postoperative mortality, general and operation‐specific morbidity and long‐term outcomes. Results Twenty‐seven studies comprising 20 898 patients undergoing general, gastrointestinal, cardiothoracic, orthopaedic, vascular, urology, endocrine and neurosurgical operations were included. Surgeons consistently overpredicted mortality rates and were outperformed by existing risk scoring tools in six of seven studies comparing area under receiver operating characteristic (ROC) curves (AUC). Surgeons' prediction of general morbidity was good, and was equivalent to, or better than, pre‐existing risk prediction models. Long‐term outcomes were poorly predicted by surgeons, with AUC values ranging from 0·51 to 0·75. Four of five studies found postoperative risk estimates to be more accurate than those made before surgery. Conclusion Surgeons consistently overestimate mortality risk and are outperformed by pre‐existing tools; prediction of longer‐term outcomes is also poor. Surgeons should consider the use of risk prediction tools when available to inform clinical decision‐making. Introduction Surgical procedures all carry associated risks. It is therefore important that surgeons are able to make accurate predictions of potential benefit and risk, including immediate mortality and morbidity, as well as long‐term outcomes, to enable balanced decision‐making and fully informed consent. Risks can also be estimated after surgery, based on additional perioperative and intraoperative data, which allows contemporary prediction of outcome. There are numerous risk prediction models that enable the surgeon to quantify risk based on measurable parameters1-5. However, there are inherent limitations in using a generalized risk prediction model, which may not include clinical data pertinent to the individual case in question, leading to variability in model accuracy6-10. As a result, risk prediction tools are generally used in tandem with the surgeon's ‘gut feeling’ of overall risk and anticipated outcome (‘clinical gestalt’). Several disparate factors influence surgeons' perception of outcome: patient factors, such as their perceived fitness, their pathology and planned procedure; setting factors, such as the experience of other members of staff; and surgeon factors, such as clinical knowledge, operative skill, previous significant surgical complications, and inclinations and attitudes11-13. Anticipating surgical risk is subject to multiple biases, which make it challenging. These include the natural tendency toward anecdotal recall and the availability heuristic (the likelihood of making a decision based on how easily the topic or examples come to mind)14, 15. Some studies16-18 support the accuracy and reproducibility of surgeons' predictions, whereas others19-22 demonstrate less favourable results. The complexity of synthesizing risk perceptions is significant and incompletely understood23, 24. The accuracy of surgeons' prediction has not been explored previously in a systematic manner. The aim of this review was thus to determine, from the available evidence, whether a surgeon's gut feeling or perception of risk correlates with postoperative outcomes, and to compare this prediction with currently available risk scoring systems, where available. Methods This systematic review was undertaken in accordance with the PRISMA guidelines25, 26. MEDLINE (via PubMed), Embase, the Cochrane Library Database, and the Cochrane Collaboration Central Register of Controlled Clinical Trials were searched with no date or language restrictions, with the last search date on 9 July 2018. The search term used was (‘Surgeons’[Mesh] OR ‘General Surgery/manpower*’ [MeSH]) AND (‘perception’ OR ‘intuition’ OR ‘predict*’ OR ‘decision making’ [mesh]). There was no restriction on publication type. This search was complemented by an exhaustive review of the bibliography of key articles, and also by using the Related Articles function in PubMed of included papers. Results were restricted to human research published in English. Inclusion and exclusion criteria All studies of patients undergoing surgery in which a preoperative or postoperative surgeon assessment (or proxy assessment) of a postoperative outcome was performed were included. This included articles that reported general risk (such as mortality) or a surgery‐specific risk (for example anastomotic leakage). Studies that made comparisons with established risk scoring tools were also included. Papers or abstracts in English, or non‐English papers with an English abstract, were included. Papers describing the risk assessment of ‘theoretical’ cases, or patient vignettes in a situation distant from clinical practice (such as a conference), were excluded, as were studies in which surgeons' assessment of risk was compared with an established risk scoring tool, without data on actual patient outcome. Data extraction and assessment of study quality Three authors independently extracted data and assessed the methodological quality of the studies, with all data extraction independently checked by the senior author. The following baseline data were extracted from each study: first author, year of publication, data collection period, geographical location, study design and type (single or multiple centres, number of surgeons involved in risk estimation, whether consecutive patients were enrolled), surgical specialty, whether other risk scoring systems were used for comparison and, if so, whether the assessor was blinded to this result. Data extracted regarding the assessment of risk included: risk outcome assessed; timing of risk estimation (preoperative or postoperative); type of risk assessment by surgeons (qualitative, quantitative, continuous scale such as a visual analogue scale (VAS), or composite score); absolute value of risk event predicted by surgeon and by scoring system; absolute value of risk occurrence rate; summary data on outcome reported, including area under the curve (AUC) of receiver operating characteristic (ROC) curves, observed : expected (O : E) or predicted : observed (P : O) ratios, or any other summary data. When data were available, AUCs were extracted with their 95 per cent confidence intervals. AUCs greater than 0·9 were considered as indicating high performance, 0·7–0·9 as moderate performance, 0·5–0·7 as low performance, and less than 0·5 as indicating risk assessment no better than chance alone27, 28. Risk predictions made by pre‐existing tools, such as the Physiological and Operative Severity Score for the enumeration of Mortality and morbidity (POSSUM)1, Portsmouth‐POSSUM (P‐POSSUM)4 or Continuous Improvement in Cardiac Surgery Program (CICSP)5, were compared with outcome when given. Internal prediction models, where authors would derive significant predictive co‐variables from their data set and assess the accuracy of these co‐variables within the same data set, were not evaluated as they lacked validity. Study quality was assessed using the Newcastle–Ottawa (NO) score29, 30. The NO score assigns points based on: the quality of patient selection (maximum 4 points); comparability of the cohort (maximum 2 points); and outcome assessment (maximum 3 points). Studies that scored 6 points or more were considered to be of higher quality. Outcome measures The following outcome measures were defined a priori and refined during data extraction: postoperative mortality (usually defined as 30 days after surgery); postoperative general morbidity (usually defined as 30 days after surgery); postoperative procedure‐specific morbidity; and long‐term outcome (typically operation‐specific). Further comparative analyses of outcomes included comparison of preoperative and postoperative predictions, and of predictions made by consultants and surgical trainees. Narrative synthesis Given the marked heterogeneity in study design, patient population included, method of assessing risk and outcomes assessed, meta‐analysis was deemed not appropriate. A narrative synthesis was therefore performed according to the Guidance on the Conduct of Narrative Synthesis In Systematic Reviews31. Three authors systematically summarized each article using bullet points to document key aspects of each study, focusing particularly on methods used and results obtained. The validity and certainty of the results were noted (whether appropriate statistical comparisons were used and, if so, their effect size and significance). The senior author identified and grouped common themes, divided larger themes into subthemes, tabulated a combined summary of the paper, and synthesized a common rubric for each theme. Consolidated reviewers' comments can be found in Table S1 (supporting information). Results A total of 584 articles were identified from the literature search, of which 48 were retrieved for evaluation. Papers were excluded on the basis of being duplicates (1) and being irrelevant based on the title (497) and abstract (38) (Fig. 1). Twenty‐seven studies16-24, 32-49 comprising 20 898 patients met the inclusion criteria and were included in the narrative synthesis (Appendix S1, supporting information).

Item Type: Article
Date Type: Publication
Status: Published
Schools: Medicine
Publisher: Wiley
ISSN: 2474-9842
Date of First Compliant Deposit: 5 March 2020
Date of Acceptance: 24 September 2019
Last Modified: 05 May 2023 09:54
URI: https://orca.cardiff.ac.uk/id/eprint/130168

Citation Data

Cited 23 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics