Clinical Trial Site Metrics

Do clinical research sites know the extent to which big pharma and CROs collect and monitor metrics about their site?

I’m in favour of the collection and analysis of metrics and key performance indicators (KPIs) in clinical trials. What I wish is that the information was shared with sites. There should be complete transparency on which metrics are being recorded. The results should then be disclosed to the sites. How are sites supposed to improve if they don’t know how they compare with other sites or where there is room for improvement?

There should be complete transparency on which metrics are being recorded. The results should then be disclosed to the sites.

I’ve been fortunate to work closely with many large sponsors and CROs and have had the privilege to view and discuss site metrics. I’m so grateful for this opportunity.

Here’s a secret that I discovered when reviewing the metrics. Sometimes, and I know you’re going to be shocked; those metrics aren’t accurate. Decisions are being made based on those metrics, but if there are data entry errors, the site is being judged unfairly.

The top two errors that I’ve seen are the target enrollment number for the site and the site activation date. Those are two key items to get wrong. The target enrollment number may be discussed during feasibility in a conversation with the CRA, never confirmed in writing with the site, so the site is not aware of the decided target number. The sponsor often records the site activation date as the date of the SIV but frequently there is a delay between the SIV and when the site is given the green light to start screening. This delay can be a week or more if the site doesn’t have the supplies they need.

Without the site metrics being shared with the sites, there’s no opportunity to correct inaccurate information.

Imagine all the possible things that could be tracked, measured, and compared against other sites. Here’s what is often recorded, analyzed, and assessed:

• Time from selection to activation
• Time from activation to screening the first candidate
• Target enrollment (even if it is not shared with the site, there’s always a target)
• Percentage of enrolled participants from site target
• Number of queries issued
• Number of open queries
• Number of days between enrolled participants
• Time from participant visit to data entered in CRF
• Budget per participant
• Number screened
• Number enrolled
• Number screen failures
• Protocol violations
• Time between global start and site start

And my favorite:

• participants enrolled per site per month

This last one is extremely important for planning future clinical trials. It is used to help establish recruitment timelines.

The sites are compared with other sites in the country and across the whole study. Sites are given ratings and metrics of concern are flagged.

Occasionally, sites will get a glimpse of a metric – like when a site is contacted to let them know that there are X# of queries outstanding or when the CRF completion is averaging > than a week. There are also the recruitment graphs shared during the recruitment period. I’m a fan of these so that sites can see how they stack up against other sites. Besides these things, the site probably won’t hear about the specifics of how the site is performing.

So, if a site is wondering why they weren’t selected for a study and they’ve worked with the sponsor or CRO in the past, it could be that there’s room for improvement in site metrics.

Here’s my advice for sites:

• Confirm in writing with the sponsor what the target enrollment number is for your site for every study.
• If you’re able to record metrics on your own site, start recording them. Over time review them and assess where there might be room for improvement.
• Ask the sponsor/CRO if they collect metrics on your site and if they do, request to see them. It’s probably not a good use of time for every study but it might be useful for large, important studies.
• If you are able, start to collect metrics on sponsors and CROs. You may discover where roadblocks occur and have an opportunity to escalate issues and improve some of those metrics. I’ll have a future post on this topic.

Here’s my advice for sponsors and CROs:

• Confirm in writing with the site the target recruitment number.
• Consider sharing site metrics with the site. For sites that want to improve, this can enhance the relationship with the site. The site may also work to improve some of the metrics and then everyone wins.
• Use your judgement, but for most studies, sharing the recruitment graphs fosters healthy competition.
• Publicly recognize sites that are performing well in a key area. The newsletter is a good place for this, and it doesn’t have to be recruitment related. It could be the site that is the fastest at data entry following the visit. People like to see their efforts recognized and by doing this, it also communicates that metric is being tracked.

Advice for sponsors, CROs and sites: confirm target recruitment numbers in writing

When collecting metrics, transparency is key. Metrics are limited in their value if only used as historical comparisons and not shared with sites. For metrics to increase in value, they need to be shared with sites. The historical metric then transforms into a future target for improvement. If we want to make clinical trials better, starting with simple communication is a good step in the right direction.