Clinical auditing as an instrument to improve care for patients with ovarian cancer: The Dutch Gynecological Oncology Audit (DGOA) European of Surgical Oncology

of auditing for (DICA) c are a registration based on a set of quality indicators. Results of quality indicators are and benchmarked information is to the cation was Results: Between December 31, 6535 patients with were registered. The case ascertainment was 98.3% in 2016. The number of patients with ovarian cancer who start therapy within 28 days decreased over time from 68.7% in 2014 to 62.7% in 2018 (p < 0.001). The percentage of patients with primary cytoreductive surgery decreased over time (57.8% e 39.7%, P < 0.001). However, patients with complete primary cytoreductive surgery improved over time (53.5% e 69.1%, P < 0.001). Other quality indicators did not signi ﬁ cantly change over time. Conclusion: The Dutch Gynecological Oncology Audit provides valuable data on the quality of care on patients with ovarian cancer in the Netherlands. Data show variation between hospitals with regard to pre-determined quality indicators. Results of ‘ best practices ’ will be shared with all participants of the clinical audit with the aim of improving quality of care nationwide.


Introduction
In the last decades, clinical audit registries have been introduced for various conditions in European countries containing population-based data [1e3]. For gynecological malignancies, audits have also been established. A European example is the Swedish Quality Registry for Gynecologic Cancer; this population-based registry covers about 95% of all gynecological malignant tumors and was initiated in 2010. This registry includes patients with ovarian, cervical, endometrial, and vulvar cancer [4]. Such registries use a set of quality indicators concerning quality aspects of the diagnosis and treatment of patients with gynecologic cancers. Clinical registries have been acknowledged as an essential tool for quality assessment and provide feedback to participating hospitals, leading to improved patient outcomes [5].
In recent decades, various steps were taken in the Netherlands to improve the quality of care. Centralization of care was implemented since the mid-1980s for cervical and vulvar cancer [6,7]. For advanced ovarian cancer, centralization was initiated in 2010, following the requirements set by the national health authorities [8]. Centralization of ovarian cancer treatment came with quality standards in which only certified gynecologic oncologists were allowed to perform cytoreductive surgery within hospitals with a minimum of 20 cytoreductive surgery procedures annually [8]. Centralization aimed to improve the result of the cytoreductive surgery since survival has a direct relation to the completeness of the cytoreductive surgery [9]. Following the centralization of surgical care in the Netherlands, the number of hospitals performing cytoreductive surgery for advanced ovarian carcinoma reduced from nearly 90 before 2010 to 23 in 2019 [10]. Another restriction by the health authorities involved the staging procedures for low stage ovarian carcinoma, which was only allowed in those hospitals performing at least 20 cytoreductive surgery per year [11].
The surgical care for patients with a gynecological malignancy is currently organized within eight regional cancer networks. Although the number of hospitals performing cytoreductive surgery decreased, variation between these regional cancer networks concerning effectiveness may still exist. To evaluate differences in quality of care between providers, a nationwide gynecologicaloncology audit was initiated to gain more insight into patient outcomes in daily practice [3].
At the end of 2013, following an increased demand for insight in the variation of care and its influence on the quality of care delivered to patients with cancer in the Netherlands, the Dutch Gynecological Oncology Audit (DGO) was initiated by the Dutch Gynecological Oncology Working Group (WOG) and facilitated by the Dutch Institute of Clinical Auditing (DICA) [5]. The registry's primary goal is to evaluate the results of treatment given to patients with a gynecological malignancy and identify factors that could improve the outcomes in these patients. This article illustrates the key elements and processes of the Dutch Gynecological Oncology Audit and the results of the first four years of this clinical audit on ovarian cancer.

Nationwide clinical audit
The Dutch Institute of Clinical Auditing (DICA) was founded in 2010 and since then they facilitated several medical societies in The Netherlands to initiate population-based audits, using the colorectal audit as a blueprint [12]. The key element to their clinical auditing model is that healthcare professionals themselves determine the audit objectives and decide which aspects of care should be registered to gain insight into the outcomes of patients [13].
The Dutch Gynecological Oncology Audit scientific committee consists of four gynecologic oncologists, two radiation oncologists, one representative of the Netherlands Comprehensive Cancer Organization, the Netherlands (NCCN), and two members from the gynecological oncology patient advocate organization "Stichting Olijf". The scientific committee meets four times a year and determines the audit objectives and the dataset's content for each tumor type.

Dataset of the Dutch Gynecological Oncology Audit
The Dutch Gynecological Oncology Audit has been a mandatory registry since January 2014 and contains detailed clinical data of all patients treated with any form of therapy for ovarian-, cervical-, endometrial-or vulvar cancer in the Netherlands. Since 2018 a registration for patients treated with radiotherapy was added (DGOA -Radiotherapy). The registered items are divided into the following categories: patient identification, tumor characteristics, surgical and pathology items, radiotherapy and chemotherapy items. Follow up data are registered for five years after primary treatment. Approval of an Institutional Review Board was not needed since all data are anonymized and processed by a third party.

Data collection and analysis of the data
Data are either prospectively collected via a web-based survey from the medical record by gynecological oncologist or delivered by trained data managers from of the Netherlands Comprehensive Cancer Organization; the division in supply is almost equally divided. All data delivered are checked by the treating physician before it is sent to a third party, Medical Research Data Management (MRDM), who anonymizes the data. All patients seen by the gynecologic oncologist are captured in the data. The anonymized data are sent to DICA, where analysis of the data takes place. The data science team of the Dutch Institute of Clinical Auditing provides results of benchmarked quality indicator to the participating hospitals through a secured web-based dashboard. Hypothetically, it might be that patients were registered more than once, especially when treatment was performed in more than one hospital (i.e., surgery in a center hospital and chemotherapy in a non-center hospital). In order to prevent this, the registry website specifies specific rules which hospital should register a patient. Registration is usually performed by the gynecological oncology center, where the most extensive surgery was performed, e.g. such as staging or cytoreduction for ovarian cancer [14].

Data verification in the Dutch Gynecological Oncology Audit
The data entered in the Dutch Gynecological Oncology Audit was verified in 2016. This data verification process aimed to determine if all patients meeting the inclusion criteria were correctly registered in the registry and if data were accurate. The scientific committee of the Dutch Gynecological Oncology Audit selected the items to verify. All participating hospitals received an invitation to participate in this voluntary verification process, after which hospitals were randomly selected. Data verification was performed by trained data managers and verified by a third party consisting of representatives of the scientific committee of the Dutch Gynecological Oncology Audit, the Dutch health and youth care inspectorate, the patient federation, and a statistician. Data in the Electronic Health Records of the selected hospitals were compared with the registered data in Dutch Gynecological Oncology Audit. The accuracy of the data was measured through any discrepancies found against the total number of patients checked [15].

Quality indicators
In 2014 the quality indicator set of the Dutch Gynecological Oncology Audit consisted of four structure indicators and five process indicators (Appendix A1 & A2) [16]. This was evaluated over the years, and in 2018 the set included one structure indicator, two process indicators, and four outcome indicators. At the start of the registry the main focus was on developing quality indicators for ovarian cancer with a future perspective to develop quality indicators for the other malignancies. This resulted in a quality indicator set mainly focusing on ovarian cancer. The scientific committee evaluates this quality indicator set each year. New quality indicators are developed in collaboration with representatives of the Dutch Society of Obstetrics and Gynecology, the Federation of Medical Specialties, the Dutch Health and Youth Care Inspectorate, health insurance companies and patient organizations.

Transparency
A goal of the Dutch Gynecological Oncology Audit at its initiation in 2014 was to improve the quality of care by public transparency of reliable hospital-specific outcome information. Quality indicators are used to achieve this goal by making the results transparent to the specialists through benchmarked data in funnel plots. The results of publicly available quality indicators are discussed in annual meetings where all parties involved in national healthcare participate [2].

Analyses of the Dutch Gynecological Oncology Audit data
Patient, tumor, and treatment characteristics are reported for ovarian cancer from January 2014 to December 2018 collectively. The results of the process and outcome indicators are compared over the years using the chi-square test. Hospital variation is shown for a process and an outcome indicator using funnel plots. R statistical package version 1.2.5019 (R Foundation for Statistical Computing, Vienna, Austria) was used to analyze the data.

Results
Between January 01, 2014 and December 31, 2018, a total of 6535 patients with ovarian cancer were registered in the Dutch Gynecological Oncology Audit. Patient, tumor, and treatment characteristics are depicted in Table 1. Over the years, the registry became more mature, which resulted in a higher number of registered patients in more recent years than at the start of the registry (Appendix:A3).

Data verification results
According to the data verification in 2016 completeness of data included in the registry was 98.3%. Accuracy on selected items was checked, and most items had an accuracy ranging from 95 to 98%. However, 'FIGO classification' and 'date of discharge' had an accuracy of 87.8% and 27%, respectively (Appendix: A4).
To check if there were no double entries, an analysis of valid social security numbers was done by Medical Research Data Management in 2019. The analysis was performed on the entire DGOA database (September 2019). At that moment, the Dutch Gynecological Oncology Audit included 22.801 treatment records for all tumors (treatment records do not correspond to unique patients as patients could receive multiple treatments). The analysis was only possible on data supplied through direct entry by the gynecologist, which was 46.9% (10689 records) and 97.9% of these records were found to have unique social security numbers. This analysis showed that double registration in the database is not a frequently occurring problem and reinforces the data's quality. The data supplied by the Netherlands Comprehensive Cancer Organization by batch (52.1% of the data) is prevented from doubling since they create a separate unique patient number. Table 2 shows the results over time for the various indicators including the specific inclusion criteria for each indicator. Percentages are national averages calculated over the group of patients included in the nominator of the indicator. The Quality indicator set and is shown in Appendix A1. The indicator 'Percentage of patients with ovarian cancer with less than 28 days waiting time before start treatment' significantly decreased over time (2014: 68.7% -2018: 62.7% P < 0.001), meaning that more patients waited longer than 28 days to get any form of initial treatment. In addition, the indicator regarding 'The percentage of patients who underwent primary cytoreductive surgery' decreased over time as well (57.8% % -39.7%, P < 0.001), though patients with a complete primary cytoreductive surgery' improved over time (53.5%e69.1%, P < 0.001). Complete staging and mortality fluctuated over the years, but changes were not statistically significant. Fig. 1 shows the funnel plot for waiting time (<28 days) between the first visit at the hospital of treatment and the start of treatment collectively for the years 2014e2018. Positive as well as negative outliers were identified: three hospitals had a significantly shorter waiting time before the start of treatment, and four hospitals had significantly longer waiting times. Fig. 2 shows hospital variation in the percentage of patients with complete primary cytoreductive surgery between 2014 and 2018. For hospitals treating more than 50 patients during the study period, the percentage of a complete primary cytoreductive surgery (defined as no residual disease) varied between 39% and 84%, with three hospitals having significantly less favorable results on this outcome indicator than the national average.

Discussion
The Dutch gynecological clinical audit was created in order to register data from gynecologic malignancies (ovarian, vulvar, cervical and endometrial cancers) and to measure outcomes. Up till now, the analysis of quality of care has been analyzed over a period of 4 years focusing on ovarian cancer. The primary purpose of the audit was not only to establish the status of care provided but also to identify elements of the clinical care process amenable for improvement. At the end of 2018 a total of 6535 patients with a gynecological malignancy were registered in the Dutch Gynecological Oncology Audit, and case ascertainment reached almost 100% for most items. Outcome of quality indicators for ovarian cancer fluctuated over the years and showed variation between regions, indicating that there is room for overall improvements. In the United States, tumor-specific quality indicators are formulated by the SGO [17,18]. In 2015, Liang et al. published results on compliance on ovarian cancer quality indicators by the SGO in a cohort of 123 patients. In the ovarian cancer quality indicator set, the indicator 'complete staging of patients with early-stage ovarian cancer' appears most similar to ours [19]. Unfortunately, a comparison between results is not possible since the indicator is slightly different from ours as the SGO only included patients with FIGO I-IIIB [18], whereas the Dutch Gynecological Oncology Audit included all stages. To our knowledge, there are no populationbased results of similar quality indicators for gynecological oncology yet which makes international comparison not possible yet.
The first analysis of the ovarian quality indicators of the Dutch Gynecological Oncology Audit shows that they vary over time and there are differences between hospitals. The indicator "Waiting time between diagnosis and start treatment" shows that over time patients waited longer to start with treatment after being diagnosed with ovarian cancer. A previous report comprising one region in The Netherlands showed conflicting results; in this particular region, the waiting time improved after the  centralization of ovarian cancer [20]. Besides the fact that this study was only single-centered, it was also performed shortly after centralization. Since the prolonged waiting time could be influenced by the nature of the initial therapy (surgery or chemotherapy), a sub-analysis was performed on patients starting with neoadjuvant chemotherapy. This analysis showed an increase in waiting time; from 34% of patients starting with neoadjuvant chemotherapy within 28 days in 2015 to 16% in 2018. This finding did not meet our expectations, and one hypothesis could be that centralization entailed organizational problems, which, for example, resulted in longer waiting times. Another explanation could be that diagnostic procedures are more often undertaken to ensure the diagnosis ovarian carcinoma before start treatment. For this reason, we added items in the survey since 2020 to provide details on the diagnostic process. The indicator "percentage of complete primary cytoreductive surgery" shows a significant increase over the years. Although not statistically significant, the percentage complete interval cytoreductive surgery also increased over time. This trend can be explained by centralization. By establishing volume standards in 2010, hospitals were required to annually perform a minimum of 20 cytoreductive surgery. A stricter selection of patients for primary cytoreduction, as reflected by an increase in the number of patients treated with interval cytoreductive surgery, may also have influenced these results.
To fulfil the purpose of the registry, its scientific committee will focus on the reduction of hospital variation, through the identification of best practices and revealing the mechanisms responsible for the better outcomes ( Figs. 1 and 2). In addition, data quality should be maintained by registration directly from the electronic patient records, which also reduces the registry burden for medical specialists. Furthermore, next to a core dataset per tumor type, the focus of the registry should be dynamic and guided by developments in international literature and incorporate these in quality indicators, which are case-mix corrected. Lastly, if similar indicators are used internationally, collaboration with other countries will enable us to compare outcomes between countries and might identify best practices and improve the quality of care on a much broader scale. The ESGO quality indicators for cytoreductive surgery for ovarian carcinoma are a good example of an internationally accepted set of quality indicators making comparisons between countries possible [21].
The strengths of this study include the population-based nature without exclusion of any subgroups. Therefore, it gives insight in the results of the "real world". With this "real world" data, clinical auditing aims to improve the standard of care through identifying hospitals that are outliers to both sides. From this feedback, information improvement programs can be initiated to clarify underlying mechanisms [12]. An additional strength is the use of data verification reports to test the accuracy and completeness of its data, making it more reliable. One of the examples is the visible low percentage of 'date of discharge' at the start of the registry. Following the data verification, this variable became mandatory since it was used to calculate indicator 5 "Complicated course". This resulted in an almost complete entry of date of discharge of 98% in the years after (data not shown).
Nevertheless, the audit has its limitations. Although populationbased, some patients with gynecological malignancies were not registered because these patients were not seen by a gyneoncologist but by the medical oncologist only who do not provide information in the survey. The Netherlands Comprehensive Cancer Organization registers all patients with malignancies of whom a pathological diagnosis is available. With their data, it is possible to get a more complete overview and identify how many of these patients were not registered. (Appendix: A2). The differences in numbers are minimal for ovarian cancer.
In conclusion, the initiation of the Dutch Gynecological Oncology Audit registry came with hurdles and still allows for further improvement. Although, recurring issues regarding registration burden, these results give an insight into ovarian cancer outcomes in the Netherlands with future perspectives for the other malignancies as well. Presenting benchmarked data to the individual hospitals will result in a discussion on how to decrease hospital variation and evaluate the quality of care with the aim to improve outcomes for patients with ovarian cancer. This process will be used in the future as a template to gain insight and improve quality of care for patients with cervix, vulva or endometrial cancer.

Funding
There is no external funding.

Author contributions
Dr. M.W. Wouters: Active participation in reviewing of manuscript (first through last version).
Prof. Dr. R. Kruitwagen: Active participation in reviewing the manuscript (first through last version).
Dr. M. van Ham: Active participation in reviewing of manuscript (last versions).
Dr. W. Van Driel: Active participation in reviewing of manuscript (first through last version).

Declaration of competing interest
There are no conflict of interest.

Appendices A1
Quality indicators of the Dutch Gynecological Oncology Audit present in 2018 [22].