Validation of a post operative complication risk prediction algorithm in a non-head and neck squamous cell carcinoma cohort

Published:January 27, 2022DOI:


      Risk-adjusted algorithms in surgical audit attempt to adjust for patient case mix and complexity in order that published outcomes fairly reflect surgical performance and quality of care. Such risk-adjustment models have applied to head and neck squamous cell carcinoma (HNSCC). We test one algorithm, currently embedded in the oncology and reconstruction dataset within the Quality and Outcomes in Oral and Maxillofacial (QOMS) Audit, which is an artificial neural network, for its predictive accuracy on a surgical cohort receiving curative surgery for non-HNSCC pathology. A single centre retrospective case note audit of post operative complications between 2010 and 2020 was conducted on patients having curative surgery for non-HNSCC pathology. The observed complication rate was compared to the predicted probability of complications in order to test the performance of the complication risk-adjustment model. Of 1591 non-HNSCC patients, 58 met the inclusion criteria with a 30-day complication rate of 8/58 (13%). The artificial neural network predicted a complication rate of 20/58 (27%). Sensitivity (0.75), specificity (0.72) and overall accuracy (0.72) suggest acceptable discrimination. Hosmer-Lemershow Goodness of Fit test was good (p = 0.55) suggesting acceptable calibration though over-prediction of complication rate in the highest risk patents was observed. This external validation series suggests the algorithm can be applied to the non-HNSCC cohort, though some refinement of the algorithm is required to account for over-prediction of complications for higher-risk patients. With further analysis a robust means of risk adjusting for the non-HNSCC cohort should be possible.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to British Journal of Oral and Maxillofacial Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


      1. Learning from Bristol. The Report of the Public Inquiry into children’s heart surgery at the Bristol Royal Infirmary 1984-1995. Presented to Parliament by Ian Kennedy QC. Available at

        • Roques F.
        • Michel P.
        • Goldstone A.R.
        • Nashef S.A.
        The logistic EuroSCORE.
        Eur Heart J. 2003; 24: 882-883
        • Quinn M.
        Cancer trends in England and Wales 1950-1999.
        Stationery Office, London2001 (Studies on medical and population subjects No 66)
        • Auluck A.
        • Walker B.B.
        • Hislop G.
        • et al.
        Socio-economic deprivation: a significant determinant affecting stage of oral cancer diagnosis and survival.
        BMC Cancer. 2016; 16: 569
        • Tighe D.F.
        • Thomas A.J.
        • Sassoon I.
        • Kinsman R.
        • McGurk M.
        Developing a risk stratification tool for audit of outcome after surgery for head and neck squamous cell carcinoma.
        Head Neck. 2017; 39: 1357-1363
      2. British United Provident Association (BUPA) Schedule of procedures. Operative Severity Score. Available from URL:

      3. Office of Population Censuses and Surveys Classification of Interventions and Procedures (OPCS-4). 1. Available from URL:

      4. AJCC caner staging manual, 7th Edition. France: Springer; 2010 [cited 2014 Jun12] Avaliable from:

        • Dindo D.
        • Demartines N.
        • Clavien P.A.
        Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey.
        Ann Surg. 2004; 240: 205-213
        • Hosmer D.W.
        • Lemeshow S.
        • Klar J.
        Goodness-of-fit testing for the logistic regression model when the estimated probabilities are small.
        Biometrical J. 1988; 30: 911-924