Where the Data Comes From

At the beginning
If ever confirmation that there was need for ODEP came to us at our first meeting when we realised there was hardly any comprehensive and validated evidence to support the ongoing use of the majority of the hip replacements used in the UK. When ODEP started the only registry data available was from the Scandinavian registries, NJR was only about to start. Even for the Charnley THR, the most commonly used THR in the UK, published data was not available to easily be merged into the agreed ODEP submission form. Initially it appeared that we would have difficulty supporting the use of the 3 major THRs (Charnley, Stanmore and Exeter) used in a way that would fit with the NICE guidelines.

Thus in the absence of Registry data we accepted results from RCTs, data published in peer reviewed journals, data from podium presentations and manufacturers validated in house data sources.

Nowadays
We continue to accept all these forms of data provided we are convinced of its worthiness. Particularly in the early life of an implant Registry data is not as reliable as it is later on and so ODEP welcomes even small studies are in support of early registry data. This is consistent with our primary objective, to ensure manufacturers monitor the performance of their products.

In 2023, ODEP implemented a new guideline requiring manufacturers to ensure that any in-house data used for ODEP submissions complies with ISO 14155:2020 standards. This provides ODEP with extra reassurance about the quality of the submitted data.

Registry Data
“It is generally agreed that registries have provided us with enormous benefit in the monitoring of implants, surgeons, hospitals etc. They allow us to contact patients, when this is required, maybe because of a problem with an implant, at the press of a button”

Advantages of observational (registry) studies
Registries affords the security of being able to contact patients who have received a faulty device

Provided they are > 95% complete, 95% compliant and have > 95% linkability between primary and revision procedures they are reliable for many devices.

They collect data from every surgeon that uses the device and not just a “chosen few”

Studies can be nested in and run by registries such as “Beyond Compliance”
The implant is externally assessed and the data externally validated

Limitations of Registries
Most Registries report annually on data accumulated up until about 6 months before they publish. This means that there can be a significant time lag before worrying features are noted.

Many joint replacement registries rely on revision to be the metric to judge success or failure of an implant. If the patient is too unfit for a revision or even when the implant has failed it is not removed (for one reason or another) then “revision” underestimates failure.

They are less sensitive in the early years of an implant. Except for infection, repeated dislocation or #, surgeons are reluctant to revise until there is little chance of spontaneous improvement.

They can camouflage bad results if they are not sufficiently granular.

They are not all equally reliable.

Not all registries will make their data about implants, freely available and they are not always prepared to “cut” the data for a particular implant at a designated time point.

ODEP would be delighted if a greater number of registries made implant data available to manufacturers in a standard way that would answer the questions posed in an ODEP submission.

ODEP is grateful to the following registries who do make data available to manufacturers on a regular basis

NJR, the Australian Registry, the New Zealand Registry, RIPO, LROI, EPRD, the Norwegian Registry.

ODEP will only accept contemporary data from a registry. It must be recent to the registries most recent annual report.

There are many other excellent registries in the world but manufacturers find that they will not “cut” their data about an implant in the desired way.

ODEP mainly use International Society of Arthroplasty Registries (ISAR) type 1 registries (the details may be found on the ISAR website).

Here is a list of registries which manufacturers have successfully used in 2022: –
1. NJR
2. RIPO (Italy)
3. AUSTRALIAN
4. NEW ZEALAND
5. LROI (Netherlands)
6. NORWEGIAN AND FINLAND
7. SHAR & SKAR (Sweden)
8. SIRIS (Switzerland)
9. EPRD (Germany)
10. SAR (Slovakia)

11. AJRR (USA) – Under review

Could a manufacturer falsify their data?

This is a question that is often asked. It is a question in the back of the minds of the panellists as they assess ODEP submissions.

So far we have not had any definite evidence that we have been submitted data that is inaccurate and which shows an implant in a better light than it deserves.

1. Manufacturers have to answer the question at the bottom of the front sheet

Is the clinical data submitted for this hip/ knee/ shoulder representative of all the results of all studies conducted in relation to it Yes/ No.

Please enter below details of and reasons for data that has been omitted that does not indicate similar or better results than that submitted on the attached clinical data sheets.

2. Manufactures will all have governance structures within their organisation and it would be very foolish to purposefully falsify a submission for unlawful gain
3. Time goes by and manufacturers will have to make a further submission for the next time point benchmark. Inconsistencies would be noted.

Thus ODEP could never be sure that a manufacturer is not making a dishonest claim – we think it is unlikely, as it is much easier to complete a submission form truthfully.

Combining and Bundling Data
Combining

As stated, ODEP welcomes data from more than one source, particularly for the earlier benchmarks. There are occasions where a manufacturer will not have enough cases from one source but when these cases are combined to those from another source the cohort will be adequate to reach the number required for a particular benchmark. An example would be where the “X” TKR has a 250 patient cohort in the Australian Registry and 250 in NJR at 10 years. Combining them would give a cohort of 500 patients would be adequate for a 10A rating and if 400 had gone past the benchmark the cohort would be big enough for a 10A* provide the submission met the other requirements. This arrangement only applies where the implant being used and the way it is being used is identical. A separate submission form is needed to be completed for each cohort.

Bundling

“Bundling” is the term given to the practice of allowing cohorts of implants that are different, albeit usually only slightly, to be bundled together to make up numbers for a benchmark. It is practiced when the individual cohort would not be large enough to gain the benchmark in its own right and it is only allowed in a limited number of closely regulated situations. These include:

• cohorts where there are significant variations within a brand, such as a long stems (for an example see hip methodology)
• where an addition to a construct is optional such as patella resurfacing (for an example see knee methodology)

It is essential that the subgroup is of a certain size and the revision rate in the subgroup is consistent with the demands of the benchmark.