Highlights

• • We illustrate the case for a separate and stand-alone process of deduplication for study records from trials registry resources.

• • We illustrate the methods for deduplication using the bibliographic tool EndNote.

• • By the end of the tutorial, a reader will be able to deduplicate records from WHO ICTRP and ClinicalTrials.gov prior to merging with the separately deduplicated bibliographic search results for study selection.

1 Background

After navigating the eternal challenge of searching the WHO International Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov (CTG),Reference Cooper, Court, Kotas and Schauberger ¹ the researcher is faced with the grim realisation that their battle with the registers is not yet ended. The records must be exported and deduplicated.

Searching trials registers is a mandatory item (C27) in the Methodological Expectations of Cochrane Intervention Reviews (MECIR) standards.Reference Higgins, Lasserson, Thomas, Flemyng and Churchill ² The Cochrane HandbookReference Lefebvre, Glanville, Briscoe, Higgins, Thomas and Chandler ³ (Chapter 4.3.3) specifically recommends searching both CTG and the WHO ICTRP and further recommends that they should be searched separately, despite the overlap in records. ICTRP contains trial records from CTG and many other trial registries across the globe. ⁴ Further details about the value of searching trial registers and the reasons for searching CTG and ICTRP separately, including the evidence base for these recommendations, are well described in Chapter 4 of the Cochrane Handbook of Systematic Reviews of Interventions.

2 The issue at hand–why are records from trials registry resources tricky?

Registry records represent a challenge in deduplication compared to deduplicating studies reported in journals and exported from bibliographic databases such as MEDLINE. Registry records are non-bibliographic in nature—registry resources report data in different and often more/non-standard fields compared to journals—and the same registry record can be found in multiple sources yet be formatted differently and so appear ‘differently’ when presented for deduplication.

To demonstrate, Figure 1 shows how the same study record (NCT01777542) appears in EndNote when exported from CTG and ICTRP. This is the same study, but these two records would not be identified as duplicates because none of the relevant fields (such as title, author, etc.) have similar data.

Figure 1 The same record (NCT01777542) exported from CTG and ICTRP and imported into EndNote 20.1–Windows version.

More than one title-related metadata field exists for a given reference type. The ‘Generic’ reference type, which is what ICTRP records are formatted for, has the following title-related fields: title, short title, and alternate title. On the other hand, CTG records are formatted as ‘Online Database’ records and have the title, short title and translated title fields. Trial registry records can have more than one title, specifically, a short title or public title and a longer scientific title. For ICTRP records, when imported into EndNote, the public title (which is the short title) goes in the title field, and the scientific title ends up in the alternate title field (Figure 2). Whereas, in CTG records, the short title is found in the short title field and the scientific title in the title field. Furthermore, the URLs in these two records are not identical, even though both lead to a record in CTG (see Figure 1).

Figure 2 Screenshot of the metadata in EndNote for NCT01777542 imported from ICTRP showing the title, short title, and alternate title field.

This makes it harder to identify duplicate study records from trials registry resources when using automated or even manual methods to deduplicate.

Many systematic review screening tools include a deduplication feature which may be based on the comparison of a preset combination of fields. For example, Covidence software (Covidence.org), a commonly used systematic review software, states that it deduplicates based on exact matches of the title, year, and volume, and sufficiently similar values in the author field. ⁵ Based on the example provided in Figures 1 and 2, it is clear that Covidence would not identify the two records as duplicates of each other.

Recent studies exist that compare the deduplication performance of systematic review software/tools (such as Covidence) against other automated tools and reference managers (such as EndNote). In a 2021 study, comparing the deduplication efficiency of various tools for bibliographic records from health databases, Covidence scored well, with an accuracy of 96% and a sensitivity of 90%.Reference McKeown and Mir ⁶ A more recent study used six datasets to test deduplication in four tools, of which four datasets included records from CTG and ICTRP.Reference Janka and Metzendorf ⁷ Covidence demonstrated 100% precision (all duplicates identified were true duplicates) and 76.8% recall (an indicator of the number of missed duplicates). In comparing their result to the 2021 study, Janka and MetzendorfReference Janka and Metzendorf ⁷ specifically noted that the difference in accuracy of deduplication across the two studies may be due to the inclusion of trial registry data in the latter study’s dataset.

We have, however, experienced some similar and some different results for registry records in terms of both precision and recall. Since trial registry records lack metadata in fields such as journal, volume, page numbers and DOI, when a current Research Information Systems (RIS) export from a trial registry is uploaded to Covidence, it displays only a limited number of fields, namely the title and year for both ICTRP and CTG records and the author and abstract fields for ICTRP records. Given the differences in organization of metadata across fields between CTG and ICTRP records, and with so little metadata available in the records, one risks removing false positives and including false negatives during Covidence deduplication (hence reducing the efficiency of the deduplication process). In a recent test, we came across two examples of false positives being removed; one such example is shown in Figures 3 and 4.

Figure 3 Complete metadata of two records from CTG that are identified as duplicates by Covidence.

Figure 4 Data available in the records of NCT06265857 and NCT06328842 when imported into Covidence forcing them to be inaccurately identified as duplicates.

In Covidence, the deduplication is automatic and cannot be turned off, which is especially problematic if false positives are removed automatically. It should be noted that Covidence does have a manual review option, where researchers can look at all removed duplicate pairs individually, but a researcher would proactively have to know to look for this feature and be familiar with its use.

During the course of our work, in August 2024, comparing an original search (657 records) and an update search (817 records) in CTG, the method proposed in this tutorial accurately identified 576 duplicates. Covidence’s automated performance for these records was less stellar. It mistakenly flagged two records from the original search as duplicates, even though they were not (Figures 3 and 4). This left 655 records from the original search. When the new file containing 817 records was uploaded, 92 duplicates were removed, leaving 725 new records. The deduplication process resulted in a screening set containing 1,380 records, of which 2 missing records inaccurately flagged as duplicates (false positives) were removed, and 484 missed duplicates (false negatives) remained.

In a test conducted on 1 December 2024, using the same search string in both CTG and ICTRP, 68 and 22 records were retrieved, respectively (Appendix). An initial scan of the ICTRP records showed that it contained seven records from CTG, as recognised by the unique format of CTG study IDs. Importing both files into Covidence resulted in one duplicate being identified. To ensure that this wasn’t only an issue in Covidence, we used the same files in another freely available deduplication tool, the SRA deduplicator,Reference Forbes, Greenwood, Carter and Clark ^{8 , 9} and neither the focussed or relaxed modes found any duplicates between the two files. Following the process presented in this tutorial, however, resulted in the identification of all seven duplicates (100% precision and 100% recall), including the one found by Covidence. Analysis of the one duplicate record identified by Covidence showed that this particular record (NCT05036603) used the same title for the short/alternate title and scientific title, causing a title match between these two records and allowing this duplicate to be recognised.

Having demonstrated what the problem is, and why it happens, let us now turn our attention to the solution.

3 Our process

An external intermediary method (i.e., before Covidence, or other tools) is therefore necessary to efficiently deduplicate registry records. Since trials in registries have a unique study ID, this makes for a reliable unit for comparing duplicates. The format of the study ID varies across the different trial registries. In CTG, the study ID starts with NCT followed by eight digits (e.g., NCT05036603). In another registry, such as the Australian New Zealand Clinical Trials Registry, the study ID starts with ACTRN followed by 14 digits (e.g., ACTRN12608000601336).

Software such as EndNote (desktop version), a reference manager, which has the functionality to batch edit records and manipulate deduplication parameters is ideal for this task. Note that the web version of EndNote does not have the functions required to carry out this type of deduplication.

This method works for records from any trial registry where the unique trial number or study ID is present in a metadata field or as a fragment of the URL. We anticipate that this method will be of greatest use where multiple searches are made of multiple registry resources (e.g., CTG, ICTRP, CENTRAL) and/or where multiple searches are made of the same registry to break up complicated search syntax but for the same condition or intervention (e.g., Search 1, Search 2, Search 3, and so on). This method could also suit a model of searching which is focussed on trials reported in registers or via CENTRAL, as a separate step from the bibliographic searching.Reference Cooper, Premji, Worsley, Tomlinson, Dawson and Prentice ¹⁰ An additional evaluation of this method is presented separately.Reference Zabzuni, Anzola, Almuallem, Worsley, Cruz, Smith and Cooper ¹¹

Furthermore, with only the limited metadata available in screening software such as Covidence (Figure 4), screening such records would likely require accessing the registry record from the original source in order to verify eligibility criteria. This has been our experience, more often than not, where most study records have to be screened in full. The direct URL to the registry record does not appear in the Covidence record, despite being in the RIS file exported from CTG; this is due to the limited fields displayed in Covidence. This tutorial therefore also includes instructions on moving the URL for the record into the DOI field, which is more likely to be a standard displayed field in screening software.

4 Tutorial aim

In this tutorial, we present a method to deduplicate trials registry records. Our aim is to improve the efficiency and effectiveness with which duplicate trial records are removed in preparation for study selection. We aim to ‘clear out’ some of the mess.

We focus on trial records from two of the most commonly searched trial registry resources: CTG and ICTRP, with initial deduplication in EndNote (any version).

Registry records from databases such as Cochrane CENTRAL could also be included in this process since they can be isolated for download from the overall search of CENTRAL. We detail the specifics for including registry records from Cochrane CENTRAL (Wiley interface only) in this process as an addendum below.

5 Who is this tutorial for?

This tutorial is intended for researchers searching trials registers for evidence synthesis reviews. We assume that in reading this tutorial, people will already be familiar with the need for and general process of deduplicating study reports for evidence synthesis. An evidence-informed summary is available in Section 4.3 of the Technical Supplement to Chapter 4 of the Cochrane Handbook,Reference Lefebvre, Glanville, Briscoe, Higgins, Thomas and Chandler ¹² so we do not elaborate further here.

6 The deduplication method

Conceptually, the deduplication process has the following steps:

1. 1) Download records from trial registry sources.

2. 2) Import records into EndNote.

3. 3) Move a copy of the trial number data to the label field.

4. 4) Edit the data for consistency (if needed).

5. 5) Alter deduplication parameters in EndNote.

6. 6) Execute the deduplication.

7. 7) Celebrate with tea.

6.3 Step 3: Moving a copy of the data

The objective of this step is to locate the field that contains the study ID and move a copy of the study ID into a different field as we prepare for deduplication.

The data being sought (the study ID) may be in the Author field, Accession number field, or as a fragment of the URL field depending on the source registry and date of the records.

The destination field has to be one that is available in the deduplication field options and can handle numerical data. A complete list of fields can be found in the ‘Duplicates’ tab of the ‘Preferences’ menu in EndNote. We recommend the Label field, as it is a user-defined field and does not typically contain any values from the incoming records themselves. In discussion with folks at EndNote, we understand that the Author field does not handle enumerated data, making it imperfect as the field to deduplicate against—especially when moving data between EndNote libraries (even though ICTRP output already uses this field).

Copy data in the URL, author or accession number field to the Label field, thusly:

1. a) Control A to select all records within the relevant group.

2. b) Go to: ‘Library’ (or ‘Tools’ in other EN versions) in the menu bar, and select ‘Change/Move/Copy Fields’.

   • • For older CTG imports that only have the study ID as part of the URL:

     1. i. Toggle ‘Change Fields’ to ‘Move/Copy Fields’ and then switch buttons to Copy Field’.

     2. ii. Toggle ‘From’ to URL and ‘To’ to Label (see Figure 5).

     3. iii. Select ‘Replace entire field’, and click ‘OK’.

   • • In newer CTG imports that have the study ID in the Accession Number field:

     1. i. Toggle ‘Change Fields’ to ‘Move/Copy Fields’ and then switch buttons to Copy Field’.

     2. ii. Toggle ‘From’ to Accession Number and ‘To’ to Label.

     3. iii. Select ‘Replace entire field’, and click ‘OK’.

   • • In ICTRP imports that have the study ID in the author field:

     1. i. Toggle ‘Change Fields’ to ‘Move/Copy Fields’ and then switch buttons to Copy Field’.

     2. ii. Toggle ‘From’ to Author and ‘To’ to Label.

     3. iii. Select ‘Replace entire field’, and click ‘OK’.

Figure 5 Copying data from URL to Label field.

It is important to copy data between fields rather than replace as the URL field is later used to source the registry record when selecting studies (because registry data are not bibliographic in nature, it is often necessary to ‘screen’ the record in full at title/abstract stage rather than waiting until full-text).

A point to flag is that the registry version of the record can change over time where the trial manager edits as the trials evolves. It is worth saving a copy of the PDF (or using other methods for archiving web content) in the study selection stage for comparison to any later trial record.

6.4 Step 4: Editing data

The objective of this step is to ensure that the ‘Label’ field has only the study ID. ICTRP and newer CTG records do not need to be edited, as the study ID has been copied and is already available in the Label field.

This step therefore applies to cases where the study ID is only available in the URL field of CTG records. The URL data are now in both the URL and Label fields. It is therefore time to edit the data in the Label field for deduplication.

1. a) Control + A to select all records within the relevant group.

2. b) Copy (Control + C) the following, standardised text: https://clinicaltrials.gov/study/.

3. c) Go to: ‘Library’ (or ‘Edit’ in other EN versions), then ‘Find and Replace’.

4. d) Toggle ‘in’ to Label.

5. e) Populate the ‘find’ box with the text you copied earlier (via Control + C).

6. f) Leave ‘Replace with’ blank, and

7. g) Click the ‘Change’ button (Figure 6).

Figure 6 Trimming the URL fragment to isolate the study ID.

This removes the URL text preceding the unique NCT number, leaving only the unique NCT number per record in the Label field.

6.5 Step 5: Altering deduplication parameters

In the EndNote preferences, in the top left of the EndNote screen, change the default settings:

1. a) Go to ‘Edit’ (in the top menu bar), then ‘Preferences’.

2. b) In the window that pops up, click on the ‘Duplicates’ tab.

3. c) In ‘Compare references based on the following fields’, select ‘Label’ (chose only this field and no other).

4. d) select ‘Exact Match’, and

5. e) Click ‘Apply’ (or ‘Save’ in other EN versions) and then ‘OK’ (See Figure 7).

Figure 7 Changing the deduplication parameters.

7 Deduplicating Cochrane CENTRAL registry records as part of this process

The process for deduplicating trial registry records from CENTRAL (Wiley interface only) against CTG and ICTRP records requires some minor modifications. Not all steps require modifications. Those that do are detailed below. All other steps of the tutorial can proceed as written.

1. I. For ICTRP records in CENTRAL, copy (Control + C) the following text that is enclosed in quotation marks (note that the inclusion of the space is deliberate): ‘ICTRP’.

2. II. For CTG records in CENTRAL, copy (Control + C) the following text that is enclosed in quotation marks (note that the inclusion of the space is deliberate): ‘CTgov’.

8 Additional useful modifications

If the intention is to take the resulting deduplicated trial registry records and screen them alongside bibliographic records using systematic review software (such as Covidence), then it is helpful to make some additional modifications to the records.

9 Concluding remarks

The method presented will remove duplicate study records under the same study ID. We note that registry records are editable study reports, updated as the study in question evolves. Once eligible studies have been identified during the search stage, the trials registry record should be regularly checked for any update on study progress.

Any update searches made, using this method of deduplication, will remove a duplicative study record which might mean that a researcher will miss a study update associated with the study ID. Therefore, we recommend that researchers re-check study records for studies that were not already ‘complete’ when screening records during a search update.

Using the method demonstrated in this tutorial, the deduplicated records can be incorporated for study selection with the search results from bibliographic databases and other search methods.