Introduction

Gastrointestinal stromal tumors (GISTs) as mesenchymal tumor originated in abdominal cavity or retroperitoneal Cajal cells1. The modified criteria of National Institutes of Health (NIH) are the widely used to evaluate the prognosis of GISTs2, which divided the tumor into very low-risk, low-risk, moderate-risk and high-risk group3. Different risk groups have different recurrence rates, ranging from 2.4 to 69.8%4. In the NIH criteria and National Comprehensive Cancer Network (NCCN) guidelines, tumor size and mitotic count have received the greatest endorsement as predictors of malignant tumor. Mitotic count is a significant indicator in the NIH standard, which mainly rely on pathologist experience, and the results may vary between different doctors. Furthermore, mitotic count merely reflects mitotic phase of cell proliferation, which may affect the accuracy of risk stratification5.

Ki-67 as a significant immunohistochemical indicator may reflect tumor heterogeneity and cell growth, which is highly expressed in most proliferating phase, it can represent the proliferative activity of tumor cells6. High Ki-67 expression is associated with adverse pathological features and invasive behavior, which can be used as an independent prognostic indicator of GISTs. Ki-67 index can be used as a supplement to the NIH risk stratification criteria to evaluate the prognosis of high-risk GISTs. High Ki-67 expression of GISTs patients indicates a shorter survival time and poorer response to molecular targeted therapy7. Besides, there is a significant correlation between high Ki-67 expression and KIT mutation or platelet-derived growth factor receptor alpha (Platelet - derived growth factor receptor alpha, PDGFRA) mutation, which may facilitate individual multidisciplinary planning of gene-targeted therapy8,9. As the widely application of targeted therapy, NIH showed a certain limitation10. Ki-67 as a complementary factor may make better risk stratification of NIH, and its high consistency makes it possible to replace mitotic count counts and integrate with other parameters into a new prognostic stratification11. In clinical practice, Ki-67 index is obtained by surgery or needle biopsy, while these operations are traumatic and may cause tumor ulceration or rupture, increasing the risk of tumor spread. In addition, due to the heterogeneity of tumor, the evaluation of Ki-67 expression level by biopsy cannot truly reflect the overall tumor proliferation activity of GISTs12. Therefore, it has great clinical significance to establish a noninvasive technique to preoperatively predict Ki-67 expression of GISTs patients.

Radiomics is a field of medical imaging analysis that extracts numerous quantitative features from CT, MRI, and PET scans, which can be used to reflect the biologic behavior of tumor. Zhang et al.13 studied the radiomics model of arterial phase to predict the expression of Ki-67 in GIST patients, and the AUC of prediction model in validation group was 0.754, which had good prediction performance. Zhao et al.14 used Random Forest (RF) classifier to establish a Ki-67 prediction model, and the AUC of validation group reached 0.784. However, due to the “black box effect” of radiomics models, the prediction results and decision-making process of these research models lack interpretability and transparency. Shapley Additive Explanations (SHAP) analysis is a method for explaining the predictions of ML models, which aims to assign a “contribution” to each feature and explain the impact of each feature on prediction model. In this study, we constructed and validated a machine learning model based on CT semantic features and radiomics features to preoperative predict Ki-67 expression level of GISTs patients.

Materials and methods

Clinical data

This retrospective study was approved by the Ethics Committee of Affiliated Huaian No.1 People’s Hospital of Nanjing Medical University (KY-2022-045-01), and written informed consent was waived by the institutional review board. The procedures performed in this study was in accordance with the Declaration of Helsinki. We retrospectively collected 200 GIST patients proved by surgical pathology from Jan 2017 to Dec 2022. Inclusion criteria: (1) postoperative immunohistochemical analysis (2) complete clinical data (3) preoperative contrast-enhanced CT examination. Exclusion criteria: (1) low quality image affecting observation and region of interest (ROI) delineation (2) received anti-tumor treatment before surgery (3) with recurrence or metastasis after surgery. Finally, this study included 149 GISTs patients. Based on previous studies15, we divided GISTs patients into 2 groups: high Ki-67 expression group (Ki-67 ≥ 10%) and low Ki-67 expression group (Ki-67 < 10%).

CT scanning protocol

We used SIEMENS SOMATOM definition dual-source CT scanner to perform CT scanning

The patients were required to fast for six hours prior to the examination and to consume 600–1000 milliliters of water 40–60 min before the scan to ensure adequate filling of the gastrointestinal tract. The scanning was carried out ranging from diaphragm to pubic symphysis with supine position, and the examination was completed in the breath-hold state. Tube voltage was 100–120 kV; tube current was 300 mA; collimation width was 64 × 1.25 mm; pitch was 1.5:1; matrix was 512 × 512; layer thickness: 5 mm, layer spacing: 5 mm. Enhanced CT scanning was performed by bolus injection of iohexol through the cubital vein with a high-pressure syringe. Trigger scanning was conducted when the aortic trigger threshold attained 170 HU, the arterial phase images at the time of trigger and the venous phase images at 30 s after trigger were obtained. The original images were transferred to Syngo CT 2012B imaging station for multi planner reformation (MPR), and the reconstructed slice thickness was 5 mm.

Analysis of CT image features

CT images were independently evaluated by two radiologists who were blinded to the pathological outcomes. Any discrepancies between their assessments were resolved through consensus. CT image features were recorded, including tumor ___location, size, margin, growth pattern, enhancement pattern, necrosis or cystic degeneration, calcification, surface ulceration, intra-tumoral gas, and enlarged vessels feeding or draining the mass (EVFDM). The CT values of the tumor parenchyma were measured in plain scan, arterial phase, and portal vein phase, avoiding blood vessels, gas, and necrosis. CTN, CTAP and CTVP were recorded on plain scan, arterial phase, and portal vein phase, respectively. Using CTN as reference, the differences of CT value (ΔAP, ΔVP) and contrast enhancement ratio (CER) at arterial and venous phase were calculated based on the following formula:

$$\begin{gathered} \Delta {\text{AP}} = {\text{CT}}_{{{\text{AP}}}} - {\text{CT}}_{{\text{N}}} {\text{ }}\;\;\;\;\;\;\;\Delta {\text{VP}} = {\text{ CT}}_{{{\text{VP}}}} - {\text{CT}}_{{\text{N}}} \hfill \\ {\text{CER}}_{{{\text{AP}}}} {\text{ = }}\Delta {\text{AP/CT}}_{{\text{N}}} \;\;\;\;{\text{CER}}_{{{\text{VP}}}} {\text{ = }}\Delta {\text{VP/ CT}}_{{\text{N}}} \hfill \\ \end{gathered}$$

Image segmentation and radiomics features extraction

ITK - SNAP software (www.itksnap.org) was used by two radiologists. We manually draw the outline of tumor edge step by step on venous-phase enhancement CT images, 3D volume of interest (VOI) was reconstructed and radiomics features were extracted. We use FAE software to extract radiomics features from the segmented tumor images. We extracted 1316 radiomics features. Z-score was used to standardize all feature parameters according to the data of training set. We used intra-and inter-class correlation coefficients (ICCs) to assess intra -and inter-observer reproducibility of extracted features.

Features selection and development of machine learning model.

First, features with high stability (ICCs > 0.80) in intra-observer and inter-observer consistency tests were retained for further analysis. Second, we used MRMR to select top 20 radiomics features. Finally, LASSO was used to screen the optimal radiomics features. The optimal weight parameter λ was selected by 10-fold cross-validation method to obtain radiomics signature by linear combination formula. 6 ML algorithms were used for model construction, including logistic regression (LR), light gradient boosting machine (LGBM), Gaussian naïve bayes (GNB), multilayer perceptron (MLP), support vector machine (SVM), and K-nearest neighbor (KNN). Drawing ROC curves to assess the predictive efficacy of 6 ML models for Ki-67 expression of GISTs patients, and then selected the optimal prediction model. In order to further understand the risk prediction model, we use the SHAP to analyze and study the contribution of different clinical variables and risk threshold. The flowchart of model construction process was illustrated in Fig. 1.

Fig. 1
figure 1

The flowchart of model construction process.

Full size image

Statistical analysis

Using SPSS software and R − 22.0 version 3.6.3 statistical package for statistical analysis. Using Shapiro-Wilk test to test the normality of data. Measurement data (such as age, tumor diameter) conform to normal distribution were expressed in x ± s, and independent sample t test was used to compare the difference between two groups. Count data (such as gender, ___location of lesion) were expressed with (%), and comparison between the two groups by chi-square test. P values <0.05 were considered significantly different. Using 6 ML algorithms to build models, and receiver operating characteristic (ROC) curves were used to assess the predictive ability of these models. SHAP analysis was used to analyze and study the contribution of different variables and the risk threshold.

Results

General data characteristics of the patients

149 patients were enrolled in our study, with 70 males and 79 females. The median age was (59.93 ± 0.89) years. We divided the patients into high Ki-67 expression group (≥ 10%, 53 cases, 35.6%) and low Ki-67expression group (< 10%, 96 cases, 64.4%) by Ki-67 expression level. Significant differences were found in tumor diameter, ___location, margin, growth pattern, EVFDM, surface ulcer and intra-tumoral gas between the two groups (p < 0.05). While no statistically significant differences were found in gender, age, enhancement mode, calcification, CERAP, CERVP and cystic necrosis (p > 0.05). Details in Table 1.

Table 1 General clinical data.
Full size table

The value of radiomics features for prediction of Ki-67 expression in GISTs

We extracted a total of 1316 radiomics features from venous phase-enhanced CT images, including 18 first-order features, 554 texture features and 744 wavelet features. After the repeatability test with ICCs > 0.80, 978 features were retained for further analysis. First, we integrated the top 20 radiomics features of MRMR score, and then we used LASSO algorithm to further reduce the dimension of feature parameters, 12 radiomics features were finally selected. (Fig. 2A) After determining the number of features, the most predictive feature subsets were selected and their corresponding coefficients were calculated. Radscore was calculated by a weighted sum of the coefficients of the selected features to calculate the final result. Radscore is a comprehensive index that integrates multiple radiomics features into a single value. The main function of radscore is to simplify a large number of complex radiomics features into a single indicator that holds biological significance and clinical application value. The radscore formula is as follow: Radscore = -0.4*square_glcm_MCC + -3.635*square_firstorder_Minimum + 3.784*square_glcm_Imc1 + -2.826*square_glcm_Idmn + 2.656*square_glszm_ZoneVariance + 0.79*exponential_glszm_SizeZoneNonUniformityNormalized + -square_glcm_Idmn + 2.656*square_glszm_Zone Variance + 0.79*exponential_glszm_SizeZoneNonUniformityNormalized + -4.708*square_GrayLevelNonUniformity + 7.616*square_ngtdm_Strength + -0.032*logarithm_glszm_LargeAreHighGrayLevel.

Fig. 2
figure 2

(A) Weighted graph of radiomics features. (B) ROC curves of Radscore for prediction of Ki-67 expression in training and validation set.

Full size image

Emphasis+-0.391*log-sigma-1-mm-3D_glszm_ LargeAreaLowGrayLevelEmphasis + 0.499* squarerot_firstorder_Kurtosis + 0.279*wavelet-LLL_ngtdm_Busyness + -1.198. In training set, radscores of high and low Ki-67 expression group were (5.50 ± 8.27) and (-2.16 ± 5.56), respectively. Statistically significant differences were found between the two groups (p < 0.001). In validation set, radscores of the two group were (2.15 ± 1.71) and (-3.43 ± 6.90), respectively, and the difference between the two groups was statistically significant (p < 0.001). (Table 2). ROC curves were employed to evaluate the predictive ability of radscore in determining Ki-67 expression among GISTs patients. The results showed radscore had good predictive efficiency in predicting Ki-67 expression of GISTs patients in both training and validation set. AUC of training set was 0.749 (95% confidence interval 0.649–0.848), and 0.729 in validation set (95% confidence interval 0.560–0.898). (Fig. 2B)

Table 2 Radscore of the two groups in training and validation set.
Full size table

The development and evaluation of ML models

AUC of LGBM, KNN, LR, and SVM had better performance and LGBM performed best in training set. However, in test set, SVM had the highest AUC (Fig. 3A,B). LGBM may have overfitting phenomenon, while SVM have relatively good stability. AUC is focus on the predictive accuracy of a model instead of its clinical practice. Hence, decision curve analysis (DCA) and calibration curve analysis were further performed in this study. Calibration curve indicated SVM model has a high prediction accuracy (Fig. 3C). DCA also showed bigger advantage of SVM in clinical applications (Fig. 3D). Taking these results into account, SVM was considered to be the optimal model.

Fig. 3
figure 3

(A, B) ROC curvers of 6 ML models in training and validation set. (C) Calibration curves of validation set. X-axis represents the average predicted probability, and y-axis represents the actual probability. Dashed diagonal is on behalf of reference line, while other smooth solid lines are on behalf of different fitted lines of the models. The closer the fitted line to reference line, the higher precision the prediction model. (D) DCA of validation set. Black dotted line is on behalf of the hypothesis that all GISTs patients with high Ki67 expression, while red dotted line is on behalf of the hypothesis that there was no GISTs patient with high Ki67 expression. The rest of solid lines are on behalf of different models, and different colors indicate different models.

Full size image

The construction and evaluation of the optimal ML model.

SVM machine learning algorithm was used for model construction. In the total sample, we randomly selected 44 cases (30.00%) as test set, and the rest samples as training set for 10-fold cross-validation, and AUC of validation set was 0.767(0.466–0.999). AUC of the final model in test set was 0.832, and the accuracy was 73.3%. (Fig. 4A–C). Given AUC performance of validation set did not exceed test set or the exceeding ratio was less than 10%, the fitting results were considered satisfying, and SVM model could be used for classification modeling task. The learning curve showed strong fitting ability and stability in training and validation set (Fig. 4D). The results indicate that we can use SVM model for classification modeling of data.

Fig. 4
figure 4

(A–C) ROC curvers of SVM model in training, validation and tset set. (D) Learning curve. The red dashed line on behalf of training set, and blue dashed line on behalf of validation set.

Full size image

Explanation of SHAP

SHAP was used in this study to illustrate how these variables contributed to the prediction model. Figure 5A shows the distribution of SHAP value of the selected features, ranking from top to down according to the importance of each feature. X-axis position represents SHAP value of the feature in the same row. Different colors of dots were used to mark the attributions of all patients to the outcome, and red dashed line on behalf of high risk value, while blue dashed line on behalf of low risk value. Figure 5B shows the model interpretation of the top 7 risk factors according to the mean absolute SHAP values, and the X-axis was on behalf of the importance of predictive model. Figure 5C,D shows the force plots of personalized features attributions for 2 representative patients in this study cohort. Figure 5C shows a GISTs patient with high Ki67 expression has higher SHAP score (0.64). While Fig. 5D shows a GISTs patient with low Ki67 expression has lower SHAP score (0.15).

Fig. 5
figure 5

(A) Bee-swarm plots of the contribution of each SHAP variable. (B) Bar graph of the average SHAP value for each predictive factor; Higher SHAP values represent greater prediction effect of the model. Force plots of personalized feature attributions of GIST patients with high Ki67 expression (C) and low Ki67 expression (D). The arrow was used to represent contribution of each feature, the arrow pushes either a decrease (negative contribution) or an increase (positive contribution) in outcome probability. The red color indicates positive contribution, while blue color indicates negative contribution. The length of each arrow is proportional to the SHAP value of the feature.

Full size image

Discussion

In our study, we developed a radiomics signature based on venous phase enhanced CT images, combined CT semantic features to construct an interpretable ML model, and compared six ML models to determine the optimal ML for preoperatively predicting Ki-67 expression in patients with GISTs. Besides, we used SHAP to visualize the model prediction process. The result demonstrate that the combined ML model is a feasible tool for evaluating Ki-67 expression, with SVM being more accurate and robust.

Many studies have shown that there is a close relationship between tumor radiological features and Ki-67 index. Traditional CT semantic features have important predictive value in preoperative prediction of Ki-67 expression of GISTs patients16,17. Our study also suggested that tumor diameter, ___location, margin, growth pattern, EVFDM, surface ulcer and intra-tumoral gas have significant difference between high and low Ki-67 expression of GISTs patients. In recent years, more and more studies showed that radiomics based on enhanced CT images could be used to precisely predict Ki-67 expression of GISTs patients15,18,19. The enhanced degree of venous phase is affected by the biological behavior of tumor in GISTs patients, while Ki-67 may accelerate its growth and metastasis of tumor cells. Therefore, the radiomics features selected according to venous phase CT images are more helpful to predict Ki-67 expression level of GISTs patients. In this study, 12 selected radiomics features successfully demonstrated the feasibility of CT radiomics features to predict Ki-67 expression in GISTs patients. The selected radiomics features included 2 first-order texture features and 10 higher-order texture features. First-order texture features are primarily extracted based on the gray-level histogram of the image, used to reflect the intensity distribution characteristics of individual voxels (or pixels) in the image. Higher-order texture features included GLSZM (n = 5), NGTDM (n = 2) and GLCM (n = 3). GLSZM quantifies gray level zones in an image. By analyzing the size and distribution of these zones, GLSZM features capture information about the homogeneity, texture coarseness, and complexity of the image. NGTDM, also known as the summation and difference histogram, considers the difference in gray levels between a pixel and its neighbors. It quantifies the spatial relationship of pixel intensities, providing insights into the coarseness of the texture and the spatial frequency content. GLCM is a statistical method that analyzes the spatial relationship between pairs of pixels or voxels with specific gray level values. It characterizes texture by considering how often pairs of pixel values with a specific spatial relationship occur in an image. All the radiomics features could precisely reflect the heterogeneity of tumor. Our study showed radiomics model had good prediction performance in both training and validation set, with AUC of 0.749 (95% CI: 0.649 0.848) and 0.729 (95% CI: 0.560 0.898), respectively. Previous studies have also used radiomics features to predict Ki-67 expression of GISTs13,14, but the prediction results of these research models lack interpretability, which causes certain troubles for clinical application and decision-making. However, there is no relevant study on preoperatively predicting of Ki-67 expression level in GISTs patients based on interpretable ML model.

In this study, we used six ML models to thoroughly evaluate the performance in predicting Ki-67 expression level in GISTs patients, which were widely used and showed excellent performance in medical image classification and prediction task. For example, LR has high computational efficiency and performs well for linearly separable datasets. LGBM performs well in dealing with structured data. GNB was good at classification problem based on feature independent assumption. MLP is a ML model based on artificial neural network, which can learn more complex nonlinear relationships, making it perform better when dealing with complex datasets. KNN is good at dealing with multiple category problems, especially in small datasets or small feature space, its performance is usually good. SVM has good generalization ability for small sample data, effective in high-dimensional space and has good effect when the sample size is not particularly large20,21. Due to the small sample size of our study, we use SVM to construct the prediction model which can improve generalization ability of the model. In our study, AUC of SVM model in predicting Ki-67 expression of GIST patients were 0.840 ,0.767 and 0.832 in training, validation and test set, which showed great prediction accuracy. In order to deeply understand the risk prediction model, we analyzed and studied the contribution of different variables using the SHAP method. SHAP analysis offers a general tool for evaluating the features importance of ML models. Feature importance of each point was determined by Shapley values, which was a game theory concept firstly introduced in 1950s22. In interpretable ML model, we used SHAP values to explain model output by calculating contribution of each input feature across all samples in dataset. We used visualized SHAP values of global and local forms to study the influence and interaction between variables. If the SHAP value is positive, indicating it has positive contribution to model prediction and vice versa. In our study, SHAP analysis showed that radscore, tumor diameter and growth pattern were the top 3 features with the most information. In addition, radscore and tumor diameter has a high positive impact for outcome prediction. A higher radscore (risk threshold = -0.1175) indicates a greater likelihood of high Ki-67 expression and increased risk of postoperative recurrence or metastasis. Our study also found that tumor diameter was another important predictive factor of high Ki-67 expression. When the tumor diameter exceeds 5.5 cm, the likelihood of higher Ki-67 expression increases, which is similar to previous studies23. Because the tumor size can reflect the growth of tumor cells. In risk classification of GISTs, larger tumor size tends to have more aggressive biological behavior, and the worse prognosis. Wei et al.24 showed that the larger the tumor diameter, the higher probability of postoperative recurrence and metastasis, which was similar to our study.

Some limitations were found in our study. Firstly, this was a single-center study, in the future, multi-center data are needed for external validation to improve the generalization of the model. Secondly, we extracted radiomics features from venous phase CT images in this study, and the radiomics features of multi-phase CT images may improve the prediction performance of the model. Lastly, gene mutation was not considered in the study, different types of gene mutation may require different targeted drugs. Therefore, it is worth looking forward to explore the relationship between radiomics features and different c-kit or PDGFRA mutations in future radio-genomics studies.

Conclusion

In conclusion, the interpretable ML model based on CT semantic features and radiomics features can be used to predict the Ki-67 expression of GISTs patients individually before surgery, providing reliable imaging biomarkers for clinical individualized treatment decisions.