A novel multimodal deep learning model for preoperative prediction of microvascular invasion and outcome in hepatocellular carcinoma

Published:September 05, 2022DOI:



      Accurate preoperative identification of the microvascular invasion (MVI) can relieve the pressure from personalized treatment adaptation and improve the poor prognosis for hepatocellular carcinoma (HCC). This study aimed to develop and validate a novel multimodal deep learning (DL) model for predicting MVI based on multi-parameter magnetic resonance imaging (MRI) and contrast-enhanced computed tomography (CT).


      A total of 397 HCC patients underwent both CT and MRI examinations before surgery. We established the radiological models (RCT, RMRI) by support vector machine (SVM), DL models (DLCT_ALL, DLMRI_ALL, DLCT + MRI) by ResNet18. The comprehensive model (CALL) involving multi-modality DL features and clinical and radiological features was constructed using SVM. Model performance was quantified by the area under the receiver operating characteristic curve (AUC) and compared by net reclassification index (NRI) and integrated discrimination improvement (IDI).


      The DLCT + MRI model exhibited superior predicted efficiency over single-modality models, especially over the DLCT_ALL model (AUC: 0.819 vs. 0.742, NRI > 0, IDI > 0). The DLMRI_ALL model improved the performance over the RMRI model (AUC: 0.794 vs. 0.766, NRI > 0, IDI < 0), but no such difference was found between the DLCT_ALL model and RCT model (AUC: 0.742 vs. 0.710, NRI < 0, IDI < 0). Furthermore, both the DLCT + MRI and CALL models revealed the prognostic power in recurrence-free survival stratification (P < 0.001).


      The proposed DLCT + MRI model showed robust capability in predicting MVI and outcomes for HCC. Besides, the identification ability of the multi-modality DL model was better than any single modality, especially for CT.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to European Journal of Surgical Oncology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Bray F.
        • Ferlay J.
        • Soerjomataram I.
        • Siegel R.L.
        • Torre L.A.
        • Jemal A.
        Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.
        CA A Cancer J Clin. 2018; 68: 394-424
        • Reveron-Thornton R.F.
        • Teng M.L.P.
        • Lee E.Y.
        • et al.
        Global and regional long-term survival following resection for HCC in the recent decade: a meta-analysis of 110 studies.
        Hepatol Commun. 2022; (00): 1-14
        • Sapisochin G.
        • Bruix J.
        Liver transplantation for hepatocellular carcinoma: outcomes and novel surgical approaches.
        Nat Rev Gastroenterol Hepatol. 2017; 14: 203-217
        • Mazzaferro V.
        • Bhoori S.
        • Sposito C.
        • et al.
        Milan criteria in liver transplantation for hepatocellular carcinoma: an evidence-based analysis of 15 years of experience.
        Liver Transplant. 2011; 17: S44-S57
        • Shindoh J.
        • Makuuchi M.
        • Matsuyama Y.
        • et al.
        Complete removal of the tumor-bearing portal territory decreases local tumor recurrence and improves disease-specific survival of patients with hepatocellular carcinoma.
        J Hepatol. 2016; 64: 594-600
        • Zhong X.P.
        • Zhang Y.F.
        • Mei J.
        • et al.
        Anatomical versus non-anatomical resection for hepatocellular carcinoma with microscope vascular invasion: a propensity score matching analysis.
        J Cancer. 2019; 10: 3950-3957
        • Rodriguez-Peralvarez M.
        • Luong T.V.
        • Andreana L.
        • Meyer T.
        • Dhillon A.P.
        • Burroughs A.K.
        A systematic review of microvascular invasion in hepatocellular carcinoma: diagnostic and prognostic variability.
        Ann Surg Oncol. 2013; 20: 325-339
        • Lee S.
        • Kang T.W.
        • Song K.D.
        • et al.
        Effect of microvascular invasion risk on early recurrence of hepatocellular carcinoma after surgery and radiofrequency ablation.
        Ann Surg. 2021; 273: 564-571
        • European Association for the Study of the Liver
        Electronic address eee, European association for the study of the L. EASL clinical practice guidelines: management of hepatocellular carcinoma.
        J Hepatol. 2018; 69: 182-236
        • Heimbach J.K.
        • Kulik L.M.
        • Finn R.S.
        • et al.
        AASLD guidelines for the treatment of hepatocellular carcinoma.
        Hepatology. 2018; 67: 358-380
        • Banerjee S.
        • Wang D.S.
        • Kim H.J.
        • et al.
        A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma.
        Hepatology. 2015; 62: 792-800
        • Min J.H.
        • Lee M.W.
        • Park H.S.
        • et al.
        Interobserver variability and diagnostic performance of gadoxetic acid-enhanced MRI for predicting microvascular invasion in hepatocellular carcinoma.
        Radiology. 2020; 297: 573-581
        • Zhang X.
        • Ruan S.
        • Xiao W.
        • et al.
        Contrast-enhanced CT radiomics for preoperative evaluation of microvascular invasion in hepatocellular carcinoma: a two-center study.
        Clin Transl Med. 2020; 10: e111
        • Yang L.
        • Gu D.
        • Wei J.
        • et al.
        A radiomics nomogram for preoperative prediction of microvascular invasion in hepatocellular carcinoma.
        Liver Cancer. 2019; 8: 373-386
        • Chong H.H.
        • Yang L.
        • Sheng R.F.
        • et al.
        Multi-scale and multi-parametric radiomics of gadoxetate disodium-enhanced MRI predicts microvascular invasion and outcome in patients with solitary hepatocellular carcinoma </= 5 cm.
        Eur Radiol. 2021; 31: 4824-4838
        • Hepatocellular Carcinoma B.E.-S.H.
        N Engl J Med. 2011; 365: 1118-1127
        • Kim H.D.
        • Lim Y.S.
        • Han S.
        • et al.
        Evaluation of early-stage hepatocellular carcinoma by magnetic resonance imaging with gadoxetic acid detects additional lesions and increases overall survival.
        Gastroenterology. 2015; 148: 1371-1382
        • Zhang Y.
        • He K.
        • Guo Y.
        • et al.
        A novel multimodal radiomics model for preoperative prediction of lymphovascular invasion in rectal cancer.
        Front Oncol. 2020; 10: 457
        • Esteva A.
        • Kuprel B.
        • Novoa R.A.
        • et al.
        Dermatologist-level classification of skin cancer with deep neural networks.
        Nature. 2017; 542: 115-118
        • Parmar C.
        • Barry J.D.
        • Hosny A.
        • Quackenbush J.
        • Aerts H.
        Data analysis strategies in medical imaging.
        Clin Cancer Res. 2018; 24: 3492-3499
        • Liu F.
        • Liu D.
        • Wang K.
        • et al.
        Deep learning radiomics based on contrast-enhanced ultrasound might optimize curative treatments for very-early or early-stage hepatocellular carcinoma patients.
        Liver Cancer. 2020; 9: 397-413
        • Nam J.Y.
        • Lee J.H.
        • Bae J.
        • et al.
        Novel model to predict HCC recurrence after liver transplantation obtained using deep learning: a multicenter study.
        Cancers. 2020; 12
        • Wang M.
        • Fu F.
        • Zheng B.
        • et al.
        Development of an AI system for accurately diagnose hepatocellular carcinoma from computed tomography imaging data.
        Br J Cancer. 2021; 125: 1111-1121
        • Jiang Y.Q.
        • Cao S.E.
        • Cao S.
        • et al.
        Preoperative identification of microvascular invasion in hepatocellular carcinoma by XGBoost and deep learning.
        J Cancer Res Clin Oncol. 2021; 147: 821-833
        • Xu X.
        • Zhang H.L.
        • Liu Q.P.
        • et al.
        Radiomic analysis of contrast-enhanced CT predicts microvascular invasion and outcome in hepatocellular carcinoma.
        J Hepatol. 2019; 70: 1133-1144
        • Song D.
        • Wang Y.
        • Wang W.
        • et al.
        Using deep learning to predict microvascular invasion in hepatocellular carcinoma based on dynamic contrast-enhanced MRI combined with clinical parameters.
        J Cancer Res Clin Oncol. 2021; 147: 3757-3767
        • Roayaie S.
        • Blume I.N.
        • Thung S.N.
        • et al.
        A system of classifying microvascular invasion to predict outcome after resection in patients with hepatocellular carcinoma.
        Gastroenterology. 2009; 137: 850-855
        • Pencina M.J.
        • D'Agostino R.B.
        • Sr .,
        • D'Agostino Jr., R.B.
        • Vasan R.S.
        Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.
        Stat Med. 2008; 27 (; discussion 207-112): 157-172
        • Pencina M.J.
        • DAgostino Rb Sr
        • Steyerberg E.W.
        Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers.
        Stat Med. 2011; 30: 11-21
        • Zhu Y.
        • Xu D.
        • Zhang Z.
        • et al.
        A new laboratory-based algorithm to predict microvascular invasion and survival in patients with hepatocellular carcinoma.
        Int J Surg. 2018; 57: 45-53
        • Nitta H.
        • Allard M.A.
        • Sebagh M.
        • et al.
        Prognostic value and prediction of extratumoral microvascular invasion for hepatocellular carcinoma.
        Ann Surg Oncol. 2019; 26: 2568-2576
        • Zhang Y.
        • Shu Z.
        • Ye Q.
        • et al.
        Preoperative prediction of microvascular invasion in hepatocellular carcinoma via multi-parametric MRI radiomics.
        Front Oncol. 2021; 11633596
        • Peng J.
        • Zhang J.
        • Zhang Q.
        • Xu Y.
        • Zhou J.
        • Liu L.
        A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis B virus-related hepatocellular carcinoma.
        Diagn Interv Radiol. 2018; 24: 121-127
        • Li X.
        • Qi Z.
        • Du H.
        • et al.
        Deep convolutional neural network for preoperative prediction of microvascular invasion and clinical outcomes in patients with HCCs.
        Eur Radiol. 2021; 32: 771-782
        • Wei J.
        • Jiang H.
        • Zeng M.
        • et al.
        Prediction of microvascular invasion in hepatocellular carcinoma via deep learning: a multi-center and prospective validation study.
        Cancers. 2021; 13: 2368
        • Meng X.P.
        • Wang Y.C.
        • Zhou J.Y.
        • et al.
        Comparison of MRI and CT for the prediction of microvascular invasion in solitary hepatocellular carcinoma based on a non-radiomics and radiomics method: which imaging modality is better?.
        J Magn Reson Imag. 2021; 54: 526-536
        • Wang G.
        • Zhu S.
        • Li X.
        Comparison of values of CT and MRI imaging in the diagnosis of hepatocellular carcinoma and analysis of prognostic factors.
        Oncol Lett. 2019; 17: 1184-1188
        • Lee C.M.
        • Choi S.H.
        • Byun J.H.
        • et al.
        Combined computed tomography and magnetic resonance imaging improves diagnosis of hepatocellular carcinoma </= 3.0 cm.
        Hepatol Int. 2021; 15: 676-684
        • Jin C.
        • Yu H.
        • Ke J.
        • et al.
        Predicting treatment response from longitudinal images using multi-task deep learning.
        Nat Commun. 2021; 12: 1851