Statistician Jack Wilkinson speaking on behalf of the Cochrane group highlighted multiple anomalies in data reporting as reasons in support of core outcome measures for randomised trials.
Published 04 July 2019
Substantial heterogeneity exists in the reporting of outcomes of fertility trials, with inconsistencies in numerators, denominators and definitions making results difficult to interpret, compare and combine in meta-analyses. Jack Wilkinson presented the rationale for a core outcome set of measures for fertility trials on behalf of the Cochrane Gynaecology and Fertility Group and the COMMIT initiative in Vienna.
More than 800 combinations of numerator and denominator were reported as outcomes in 142 IVF randomised controlled trials (RCTs) published in peer-reviewed journals in the space of just one year, according to a statistician from the Cochrane group. Important outcomes such as clinical pregnancy were defined in over 60 different ways or were not defined at all, thus causing methodological frailty in the evidence base.
This was the prelude to an important session in Vienna organised by the Cochrane Gynaecology and Fertility group which would ultimately call for outcomes defined by 'core' consistent measures related to birth rate in final reports.
IVF treatments, said statistician Jack Wilkinson, include a succession of bottlenecks which would be used incorrectly as denominators when randomisation had occurred earlier. Thus, outcomes such as miscarriage may be reported per clinical pregnancy or per embryo transfer when women were in fact randomised prior to oocyte pick-up. Fewer than half the RCTs reporting live birth actually presented a correct calculation which included all randomised participants or only excluding protocol violators.
One of the important consequences of these inaccuracies and inconsistencies in reporting trial outcomes, said Wilkinson, is imprecision. The median width of the 95% confidence intervals calculated for live birth in meta-analyses was 21 percentage points, which, he said, would lead to a failure to detect plausible improvements in live birth rate.
Selective reporting has proved another problem. This occurs when authors selectively reveal or suppress information in order to present the trial findings in a more favourable light. One in five trials provided primary outcomes in their manuscript that were inconsistent with their original trial registry description - and this may be even more worrying in trials which are not even registered.
Wilkinson was thus keen to explain the option of 'registered reports' as one potential solution in which ‘trials can be peer reviewed before they happen on the basis of their importance and methodology’. This, he said, would remove ‘the incentive for selective reporting of results’. ‘Fertility and Sterility should be congratulated for leading on this development, and I look forward to other journals in our field embracing the format,’ he added.
Thus, defining a set of core outcome measures to be used by all trials would allow more trials to be included in meta-analyses and consequently increasing their power and precision. This should be supported by the use of statistically sound methods and measurements in fewer, but larger fertility trials.
The COMMIT initiative was a collaboration of 378 healthcare professionals, researchers and people with infertility who reached a consensus for the most relevant core outcome measures to be reported by infertility trials. What's more, from the discussion it seems the main fertility journals already support the principle for reporting and publishing such studies.
* There's more on this in our interview with the Cochrane group's Cindy Farquhar at www.focusonreproduction.eu/article/ESHRE-Meetings-Farquhar19