Haq Nawaz1, Chen Yi-Fan3, John Kloke2, Asif Khalid1, Kevin McGrath1, Douglas Landsittel2 and Georgios I Papachristou1
1Division of Gastroenterology, Hepatology and Nutrition and 2Institute for Clinical Research Education, University of Pittsburgh Medical Center; 3Department of Biostatistics, University of Pittsburgh. Pittsburgh, PA, USA
Received March 9th, 2013 – Accepted June 5th, 2013
Context The optimal approach to pre-operative imaging assessment of pancreatic cancer is unknown. Objective The aim ofthis meta-analysis was to assess accuracy and performance characteristics of EUS in determining nodal staging, vascularinvasion, and prediction of resectability of pancreatic cancer. A secondary aim was to perform head to head comparison ofperformance characteristics between EUS and CT for nodal staging, vascular invasion and resectability. Design Data fromEUS studies were pooled according to bivariate generalized random effects model. Pooled estimates for CT were obtainedfrom studies which performed head to head comparison between EUS and CT. Patients Patients with pancreatic cancerundergoing pre-operative imaging assessment. Intervention EUS. Main outcome measure Pooled sensitivity, specificity,positive and negative predictive values of EUS for nodal staging, vascular invasion and resectability. Results Forty-ninestudies were considered of which 29 met inclusion criteria with a total of 1,330 patients. Pooled summary estimates for EUSnodal staging were 69% for sensitivity and 81% for specificity. For vascular invasion, sensitivity was 85% and specificity was 91%. The sensitivity and specificity for resectability was 90% and 86%, respectively. CT scan showed lower sensitivity than EUS for nodal staging (24% vs. 58%) and vascular invasion (58% vs. 86%); however, the specificities for nodal staging (88%vs. 85%) and vascular invasion (95% vs. 93%) were comparable in studies where both imaging techniques were performed. The sensitivity and specificity of CT in determining resectability (90% and 69%) was similar to that of EUS (87% and 89%). Conclusions EUS is an accurate pre-operative tool in the assessment of nodal staging, vascular invasion and resectability in patients with pancreatic cancer.
Endosonography; Neoplasm Staging; Pancreatic Neoplasms; Sensitivity and Specificity; Tomography, X-Ray Computed
Endoscopic ultrasound (EUS) is frequently used in making the cytological diagnosis of pancreatic cancer; however, its role in the pre-operative staging of such patients is not entirely clear [1, 2]. Data on the performance characteristics of EUS utilizing the electronic curvilinear array echo endoscope and its relative advantage if any over dual-phase computed tomography (CT) in the preoperative assessment of pancreatic cancer is limited [2, 3].
Pancreatic cancer is the fourth most common cancer in the United States and one of the leading causes of cancer-related mortality . It has a poor 5-year survival rate of 4%, which has remained largely unchanged over the past 20 years . This is primarily because the majority of patients with pancreatic adenocarcinoma progress to either metastatic or locally advanced disease in the asymptomatic phase. Surgical excision is the ultimate treatment with a 5-year survival rate after resection around 20% . In recent studies, neoadjuvant chemoradiation has revealed promising results in patients with locally advanced disease .
As treatment of pancreatic cancer continues to evolve, accurate staging becomes increasingly important. The challenge for preoperative imaging studies is to accurately stage pancreatic cancer and determine resectability, hence avoiding unnecessary laparotomies in patients with locally advanced disease and directing them for neoadjuvant chemoradiation. This approach may offer an opportunity for surgical cure in a select group of patients with loco-regional disease .
This is not an easy task; a recent study showed that complete resection with negative margins could only be achieved in 50% of patients with suspected loco-regional disease on routine imaging . Recent advances in imaging technology including EUS and CT have shown promise in assessing loco-regional and distant spread of the tumor and improving prediction of resectability . Currently, EUS is widely utilized in such patients, but there is no consensus on the role of EUS in the pre-operative imaging assessment of pancreatic cancer.
The primary aim of this meta-analysis is to review published literature to assess the accuracy and performance characteristics of pre-operative EUS in determining nodal staging (NS), vascular invasion (VI), and prediction of resectability (R) for pancreatic adenocarcinoma using surgery as the gold standard. A secondary aim is the head-to-head comparison of the accuracy of EUS vs. CT scan in studies that assessed both imaging modalities.
We used the PRISMA statement for quality assessment in reporting meta-analysis  (Supplementary Table ).
Inclusion and Exclusion Criteria
The focus of this meta-analysis was on nodal staging, vascular invasion, and/or resectability of pancreatic adenocarcinoma. Studies solely on tumors with other histologic sub-types and ampullary tumors were excluded. In studies where data were heterogeneous every effort was made to dissect data for pancreatic adenocarcinoma.
EUS studies in which nodal staging, vascular invasion, and/or resectability was confirmed by surgery or clinical follow-up were selected. Only studies from which a 2x2 table could be constructed (i.e., actual counts for the number of false positive, false negative, true positive, and true negative was available) for any of the following: nodal staging, vascular invasion, and/or resectability, were included in the analysis. Abstracts and articles published in languages other than English were excluded. In case of studies from same authors with overlapping time periods of investigation only the larger study (greater number of patients) was considered in the analysis to avoid duplication of results.
EUS criteria used for lymph node involvement included: round shape, hypoechogenecity, smooth borders, and short axis size greater than 5 mm in most studies [10, 11]. EUS criteria used for vascular invasion included : peripancreatic collateral vessels around a pancreatic mass with obliteration of the normal anatomic location of one of the major adjacent vessel, tumor within vessel lumen, distortion of vessel contour, and loss of hyperechoic interface between the vessel and the tumor.
An independent review of all studies assessing CT scan in preoperative staging of pancreatic adenocarcinoma was not performed. However, in EUS studies with available CT data, a sub-analysis was performed to compare the accuracy of EUS and CT in predicting nodal staging, vascular invasion, and/or resectability.
Two authors (H.N. and G.I.P.) independently conducted search for studies using the MEDLINE and Embase data base using the following search terms: “pancreatic carcinoma”, “pancreatic cancer”, “vascular invasion”, “resectability”, “TNM staging”, “endoscopic ultrasound”, “endoluminal ultrasound” and “EUS”. “Related articles” for the articles selected using the initial strategy were also searched in MEDLINE. The last search was performed in May 2013. The reference list of citations in the shortlisted articles was reviewed to identify additional papers pertaining to the current study.
Data pertaining to calculation of sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for nodal staging, vascular invasion and resectability was extracted from individual studies by H.N. and G.I.P and differences were resolved by discussion. In cases of sufficient doubt we communicated with authors of individual studies to obtain additional data for clarification. The data was recorded on piloted forms and then transferred to a spread sheet in Microsoft Excel. Additional data that was recorded included name of first author, study duration, year of publication, type of study (retrospective vs. prospective), location of tumor (pancreatic vs. ampullary), histologic sub-types of tumors if reported, name of journal and type of EUS (radial vs. linear).
Criteria Used to Assess Study Quality
Study quality was assessed using the following seven criteria as described by Dewitt et al. . One point was given for each criterion that was fulfilled by an individual study: 1) consecutive series of patients to minimize selection bias; 2) sufficient description or standardization of EUS and CT to enable replication of technique; 3) independent or blind interpretation of both tests; 4) unbiased determination of resectability (0.5 point was given if decision for resectability was independent of either EUS or CT scan); 5) independent or blind comparison with reference standard (surgical pathology); 6) report of operating characteristics such as sensitivity, specificity, PPV and NPV or description of sufficient data to enable calculation of same; and 7) clinical follow up in cases which were managed non-operatively to confirm diagnosis of pancreas cancer.
The main performance measures in this study were sensitivity, specificity, and positive (PPV) and negative (NPV) predictive values. For evaluating the heterogeneity among studies, Q statistic and Isquare statistic were used . Considering the correlation between paired outcomes and the heterogeneity, a bivariate generalized linear random effects model with logit link was utilized for EUS and CT scan on nodal staging, vascular invasion and/or resectability [14, 15]. This model estimated the pooled summary of each outcome, its standard error, and 95% confidence interval. The diagnostic performance of the sensitivity and specificity across studies was measured by the summary receiver operating characteristic curve (SROC), which incorporated with the covariance of paired outcomes . The area under the curve (AUC) was used as a summary for the diagnostic accuracy.
We evaluated for small study effects by using Deek's funnel plot asymmetry test . A sensitivity analysis, where each study is removed from the analysis and a separate pooled estimate is calculated, was carried out to assess the influence of each individual study on outcomes. All the analyses were conducted using STATA Version 12 (StataCorp LP, College Station, TX, USA). A two-tailed P value less than 0.05 was considered statistically significant except for Deek’s funnel plot asymmetry test where significance level was chosen at P<0.10.
A total of 49 studies were considered for inclusion. The manuscripts of shortlisted articles were retrieved and assessed. Twenty studies were excluded [18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37] and 29 were included in the analysis based on the criteria mentioned above and summarized in Figure 1 [8, 10, 11, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63]. The first author, year of publication, number of patients included, and type of echo endoscope used for each of the 29 selected studies are presented in Table 1. Of these 29 studies, 12 studies reported performance data on CT as well.
In respect to classification of pancreatic cancer (i.e., pancreatic vs. ampullary and adenocarcinoma vs. other histologic sub-types) data solely on pancreatic adenocarcinoma were extracted from 9 studies [8, 11, 43, 44, 45, 47, 51, 52, 61]. Eleven studies included data on tumors of other histologic subtypes [10, 38, 42, 46, 49, 53, 54, 57, 58, 60, 63] and 5 studies included data on ampullary tumors [10, 48, 49, 55, 60]. In studies with heterogeneous data the number of patients with tumors of other histologic subtypes and ampullary tumors was relatively small (less than 10).
Criteria Used to Assess Study Quality
None of the studies met all the criteria used to assess study quality as described in Table 2. Twelve studies reported consecutive series of patients [8, 10, 11, 38, 43, 44, 45, 47, 53, 54, 55, 58], 5 studies did not report EUS or CT techniques in sufficient detail to be reproducible [40, 47, 49, 51, 59]. Nine studies reported independent or blind interpretation of both imaging modalities [8, 10, 11, 38, 43, 46, 47, 54, 55]. None of the studies reported unbiased determination of resectability with respect to both EUS and CT scan as reported in Table 2. Only four studies reported unbiased determination of resectability with respect to EUS and therefore received 0.5 point [11, 38, 47, 61]. In all studies the gold standard for comparison was surgical pathology and all studies either reported the operating characteristics or data that could be used to calculate the operating characteristics. Nine studies reported details of clinical follow-up where applicable [8, 11, 42, 43, 44, 45, 50, 52, 56].
Performance Characteristics of EUS in Assessment of Nodal Staging, Vascular Invasion and Resectability
The primary aim of the meta-analysis was to assess the diagnostic performance of EUS in assessment of all three primary outcomes: nodal staging, vascular invasion and resectability. Of the 29 studies included in this analysis, 16 studies examined nodal staging [10, 11, 38, 41, 43, 44, 45, 47, 49, 51, 53, 54, 56, 61, 63] for a total number of 512 patients, 25 examined vascular invasion [8, 10, 11, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 51, 52, 53, 54, 55, 56, 57, 58, 61, 62] for a total number of 886 patients, and 9 examined resectability [8, 46, 47, 48, 53, 54, 55, 59, 60] for a total number of 377 patients. There was moderate to high degree of heterogeneity in the studies for each of the conditions as is evident in the forest plots in Figure 2. The tests of heterogeneity were all highly significant with I2 values ranging from 57.8 to 89.6. Because of the moderate to high degree of heterogeneity we used the random effects model for pooled estimates. Our sensitivity analysis indicated that no single study resulted in a substantial change in the pooled estimates. The Deek’s funnel plot asymmetry test failed to demonstrate a small study effect (Figure 3).
Figure 3. Deek’s funnel plot to assess for small study effects. This is a method used for detecting small study effects based on a linear regression of log odds ratios on inverse root of effective sample sizes. When no small study effect exists, the test of the non-zero slope coeffcient has P value >0.1, as was shown in our study.
The pooled summary estimates for EUS nodal staging were: 69% (95% CI: 51-82%) for sensitivity; 81% (95% CI: 70-89%) for specificity; 81% (95% CI: 72-88%) for PPV; and 65% (95% CI: 56-73%) for NPV. The pooled summary estimates for vascular invasion were: sensitivity of 85% (95% CI: 76-91%); specificity of 91% (95% CI: 85-94%); PPV and NPV of 87% (95% CI: 81-92%) for both. The pooled estimates for resectability were: sensitivity of 90% (95% CI: 71-97%); specificity of 86% (95% CI: 63-96%); PPV of 85% (95% CI: 66- 94%); and NPV of 90% (95% CI: 78-96%) (Figure 2 and Table 3).
A sub analysis on studies that utilized curvilinear echo endoscopes was also performed. From the original 29 studies, only 8 studies used linear EUS, either alone or in combination with radial EUS (total of 384 patients) [8, 47, 52, 56, 57, 58, 60, 62]. Since data were not presented in various studies, as well as the data available were heterogeneous because the authors of most of the studies did not use linear EUS in isolation and often they used radial EUS in combination with linear EUS (therefore, data for linear EUS could not be extracted), pooled statistics could not be performed for this subgroup. Results are as presented in Table 4.
Performance Characteristics of EUS and CT in Assessment of Nodal Staging, Vascular Invasion and Resectability
A comparison of EUS and CT was performed using data from 12 studies that assessed both imaging techniques as shown in Table 5. Eight studies [10, 11, 38, 43, 47, 54, 55, 63] assessed for nodal staging, 12 studies [8, 10, 11, 38, 43, 46, 47, 52, 54, 55, 56, 62] for vascular invasion, and 6 studies [8, 46, 47, 54, 55, 59] for resectability. CT scan showed lower sensitivities for nodal staging (24%, 95% CI: 16- 34%) and vascular invasion (58%, 95% CI 45-69%) when compared to EUS (nodal staging 58%, 95% CI: 44-70%; vascular invasion 86%, 95% CI: 70-94%). The sensitivity of CT for resectability (90%; 95% CI: 77-96%) was comparable to EUS (87%; 95% CI: 63- 96%).
Following an exploratory data analysis approach by examining SROC curves (Figure 4), the AUCs of CT scan appear to be lower for nodal staging (51%, 95% CI: 47-56%) when compared to EUS nodal staging (79%, 95% CI: 75-82%). However AUCs for CT vascular invasion (91%, 95% CI: 88-93%) and resectability (90%, 95% CI: 87-92%) were comparable to EUS vascular invasion (95%, 95% CI: 93-97%), and resectability (94%, 95% CI: 92-96%). Further the AUC CIs for EUS and CT do not overlap. This can be regarded as a conservative test of hypothesis so we can conclude that EUS performs better than CT based on the available data.
A sub-analysis comparing EUS and CT scan in assessing venous and arterial invasion was also performed. Of the 12 studies with EUS and CT data, only 5 provided separate data on venous and arterial invasion rather than pooling this information together as vascular invasion [10, 38, 59, 61, 62]. Due to the small number of studies with available data, pooled statistics could not be performed. The results of these studies are presented in Table 6.
Accurate pre-operative staging of pancreatic cancer is important in directing patients with resectable disease to surgery and to avoid resection in those with more advanced disease. The role of EUS with FNA in making a tissue diagnosis in patients with a pancreatic mass is well established. However, the role of EUS in pre-operative staging of disease and its impact on management remains unclear. Prior studies have yielded conflicting results primarily due to small studies with heterogeneous patient populations. Any information gleaned from older studies may not be applicable today due to improvements in computed tomography and EUS, with the universal use of linear array scopes for pancreatic disease.
In this meta-analysis, all EUS data in pre-operative staging of pancreatic cancer available to date were extracted using strict inclusion criteria, so as to provide summary estimates on the performance of EUS in assessing nodal staging, vascular invasion, and/or resectability. When compared to the study by DeWitt et al. , which was a systematic review of literature in this field without pooled statistics, our study reported pooled statistics using the random effects model to account for moderate to high degree of heterogeneity in the data collected. When compared to the meta-analysis by Puli et al. , which was limited to EUS staging of vascular invasion in pancreatic and ampullary tumors, our study reported pooled statistics for all three pertinent outcomes in the pre-operative imaging assessment of pancreatic adenocarcinoma including nodal staging, vascular invasion, and resectability.
In addition, we performed a sub-group analysis directly comparing EUS with CT scan in studies that reported results for both modalities. Our results are in concordance with Puli et al. , but in addition demonstrate that EUS is accurate in the assessment of nodal staging and resectability.
In our analysis, EUS was found to perform with high accuracy in all three domains including nodal staging, vascular invasion, and resectability. Since current practice in expert centers is the use of linear echo endoscopes when assessing patients with suspected pancreatic cancer for both tissue sampling and staging purposes we attempted to perform a sub-analysis using only linear EUS data. However, data on linear EUS was limited precluding the performance of summary statistics.
The accuracy of EUS for detecting lymph node metastasis in the selected studies ranged from 44 to 100% with a pooled sensitivity of 69% and specificity of 81%. The pooled sensitivity was rather low and this could be explained by peri-tumoral inflammation and/or large tumor size, which may potentially hinder detection of lymphadenopathy . EUS-FNA in addition to diagnostic EUS has the potential to increase the sensitivity of nodal staging. One study reported sensitivity of 82% when EUSFNA was used to confirm malignant lymphadenopathy .
At present, loco-regional lymphadenopathy does not preclude surgical intervention, especially for tumors involving the pancreatic head, since these can be removed en bloc with the tumor [61, 65]. In contrast, involvement of celiac lymph nodes portends an unresectable state and careful survey of this region is therefore warranted . The accuracy of CT for lymph node staging ranged from 38 to 77% with a pooled sensitivity of 24% and specificity of 88%. Based on our head-to-head comparison, EUS was more sensitive and accurate in detecting malignant lymph nodes.
Current practice in expert centers has evolved to surgical excision of pancreatic adenocarcinoma with venous invasion accompanied by venous reconstruction . Furthermore, recent studies support the use of neoadjuvant chemoradiation in locally advanced disease with arterial vascular invasion . Therefore, accurate assessment of vascular invasion is critical for avoiding unneeded laparotomies and guide initial treatment. CT scan, which has been traditionally considered the mainstream pre-operative staging tool, can be falsely negative for vascular invasion in about 20% of patients .
In our analysis, EUS accuracy in the detection for vascular invasion ranged between 62 to 100% with a pooled sensitivity of 85% and specificity of 91%. In a previous meta-analysis by Puli et al. , EUS revealed a sensitivity of 73% (95% CI: 69-77%) and specificity of 91% (95% CI: 88-92%) in diagnosing vascular invasion. The higher sensitivity seen in our analysis could be explained by the inclusion of additional studies [61, 62] with high sensitivity and the exclusion of ampullary cancer data, which could have increased the homogeneity of our study population. The accuracy of CT for diagnosis of vascular invasion ranged from 62 to 95% with a pooled sensitivity of 58% and specificity of 95%. Based on our head-to-head comparison, EUS appeared to be more sensitive in detecting vascular invasion when compared to CT scan. Both CT and EUS revealed comparable specificities.
Four different EUS criteria have been used in studies to define vascular invasion. Peripancreatic venous collaterals and tumor within vessel lumen are usually straightforward, easy to document with a high specificity . However, they are less prevalent and less sensitive when compared to abnormal vessel contour and loss of the vesselparenchymal interface. Furthermore, the diagnostic accuracy may differ among different vessels; the superior mesenteric vein is long considered the most difficult to image . It is noteworthy that criteria for arterial invasion have not been standardized . Whereas arterial involvement is obvious in cases of vessel wall irregularity or stenosis, loss of hyperechoic interface may not be considered an absolute contraindication for surgical resection in some centers .
EUS has been suggested to be more sensitive in detecting venous invasion, whereas CT is considered more sensitive in detecting arterial invasion in the pre-operative staging of patients with pancreatic cancer [10, 35, 38, 59]. Since current management in expert centers differs in locally advanced pancreatic cancer with venous and arterial invasion, we attempted to perform a subgroup comparison between EUS and CT scan in assessing venous vs. arterial invasion. Only scarce comparative data were reported especially for arterial invasion, which precluded the performance of summary statistics. It appeared though that EUS and CT scan perform similarly in assessing venous invasion and CT scan may be superior in assessing arterial invasion.
The accuracy of EUS for detecting resectability in our analysis ranged between 62 to 94% with a pooled sensitivity of 90% and specificity of 86%. The accuracy of CT scan ranged from 60 to 92% with a pooled sensitivity of 90% and specificity of 69%. Based on our head-to-head comparison, EUS and CT scan performed similarly in predicting resectability. It is important to mention that resectability largely depends on surgical practices. Some surgeons resect almost any pancreatic tumor, including performing vascular reconstruction for venous invasion, while others operate only in very early stages of loco-regional cancers referring the rest of the patients to neoadjuvant chemoradiation .
In addition to EUS and CT MRI has emerged as a complementary imaging modality for staging of pancreas cancer [36, 43]. Soriano et al. showed that for N-staging MRI had a lower sensitivity (15%) but higher specificity (93%) when compared with EUS (36% and 87%, respectively) and CT (37% and 79%, respectively). For vascular invasion MRI had sensitivity of 59% and specificity of 84% in comparison to EUS (42% and 97%, respectively) and CT (67% and 94%, respectively) . In a meta-analysis published in 2005 comparing prediction of resectability between MRI and helical CT, the sensitivity was comparable at 82% vs. 81% and the specificity was 78% vs. 82%, respectively . Other studies have shown that MRI may be superior than CT in detecting liver metastasis and peritoneal implants [68, 69]. Pooled estimates comparing MRI with EUS and CT could not be performed in our study due to small number of studies (n=3) providing operating characteristics [43, 54, 55].
This meta-analysis has several limitations. First, a moderate to high degree of heterogeneity was found in the selected studies for each of the three primary outcomes. Despite efforts to extract data only on pancreatic adenocarcinoma some studies had a small proportion of patients with ampullary tumors and/or tumors with other histologic subtypes. Second, the selected studies were published in a wide time range from 1988 to 2012. During the last 25 years, EUS and CT scan technology has evolved. Thus, the technology used in the earlier studies likely does not reflect current practices. We attempted to focus our analysis on linear EUS; however, this subgroup analysis could not be performed, because of scarce availability of data on current technology. Third, all studies were relatively small in size; none of the selected studies exceeded a sample size of 100 and only one third enrolled more than 50 patients. Finally, even though most of the included studies met majority criteria of quality assessment, the documentation of quality criteria was found to be suboptimal in a few studies.
In conclusion, this meta-analysis, despite its limitations, clearly showed that EUS is an accurate pre-operative tool in the assessment of nodal staging, vascular invasion and resectability in patients with pancreatic adenocarcinoma. On headto- head comparison, EUS appeared to perform better than CT scan in respect to nodal staging. Based on these results, we conclude that EUS staging should be the standard of care along with state-of-the-art CT scan for the preoperative evaluation of patients with pancreatic cancer. However, large, prospective, multicenter studies that will directly compare the accuracy of linear EUS vs. new generation CT scanners are still needed.
The project described was supported by the National Institutes of Health through grant numbers UL1RR024153 and UL1TR000005
The authors do not have any conflict of interest to report