Trials with Maya Z

Episode 2
[00:00:00]
Maya: Hello everyone and welcome to Trials with Maya Z! Today I'm especially excited because I have one of my old and good friends, Dana, here with me. Dana is the CEO of August Research, an amazing clinical research organization, expert in conducting clinical trials, especially in Europe.
And the reason why I invited Dana to join me in this discussion about recruitment rates is because [00:01:00] she's been one of the people that told me amazing stories about clinical trials and how things actually are in the reality. And I know that she has a lot of things to share with us, and probably we won't have enough time to share everything, but I really hope that she'll give us a sneak peek of what she experienced before.
But Dana, first to you. Can you please introduce yourself to our audience?
Dana: Sure. Thanks, Maya, for the invitation to come on your podcast today. It's exciting to talk about a topic that I'm definitely passionate about with somebody who's equally enthusiastic about improving clinical trials. So, I am Dana Niedzielska. I am the CEO and co-founder of August Research. I have been running clinical trials in Europe for over 20 years.
Obviously, you can tell from my accent that I am originally from the United States, but I moved to Bulgaria, where Maya is from, [00:02:00] in the late 90s, worked there for a few years and then started my first CRO in Europe back in the year 2000. In my first company, we focused only on Central and Eastern Europe.
We sold that business to a global CRO and after a three year non-compete, we restarted as a small CRO working across Europe. We really focused on providing excellent service for small and medium-sized farm and biotech companies.
Maya: A real entrepreneur, Dana, thank you for this introduction. So, Dana, to the key question here, how much can we trust recruitment rates when we plan clinical trials? What do you think?
Dana: Well, certainly, it's a great question and I think we could probably talk about this for hours. But I think, when we talk about recruitment rates and when you talk about using historical rates to [00:03:00] predict future enrollment, it's certainly a tricky area and it's something that is very hard to pin down.
There is a saying that if you've run one clinical trial, you've run one clinical trial. And I think there is a certain amount of truth to that. Although we have a lot of data, this is unfortuantely not something that we can we can't just replicate trials one after another.
They're run in a real-world environment with a lot of external factors affecting enrollment and performance. So, I think that when a clinical trial sponsor is starting to plan a trial and they want to predict their own enrollment rates, the place to start before just trying to pull a prior study off the shelf and just replicating that exactly, is really to look at the overall external factors that are going to affect the [00:04:00] enrollment for your trial. And the kind of the factors, I mean here are, "What is the overall environment in the countries that you're potentially targeting?," because those are the big picture things that really drive underlying enrollment rates. And what I mean there are even basic things like the organization of the healthcare system of that country. In countries where the government is providing the healthcare, there are certain assumptions that you can make about that country versus countries like the United States. We are not having a government-provided healthcare system. That type of setup affects things like, standard of care, which is a big driver of selecting the appropriate countries for a trial.
If you have a trial that exceeds standard of care in certain countries, it could be extremely attractive and drive enrollment. If you have a study that [00:05:00] is inferior to standard of care, look somewhere else, it's not going to work. So, standard of care is sort of the baseline.
You want to make sure that your study is attractive in that environment. There's also generalizations that you can make about certain regions even within Europe. We do a lot of work in Eastern Europe and one structural difference between Eastern Europe and Western Europe is that in Eastern Europe, the majority of investigators are paid directly - a large portion of the investigator grant to their bank account, and then a smaller portion is paid to the hospital. So, often that split could be 80% to the investigator, 20% to the site. Whereas in Western Europe or the USA, a 100% of the grant is paid directly to the hospital and the hospital decides how they allocate that money and what they will compensate the doctors for their time on that trial. [00:06:00] Obviously, paying people directly for their work translates into more work, more attention and more commitment to that project. For trials that are compatible with standard of care and the environment of Central and Eastern Europe, that drives a lot of the differences in that we see in enrollment rates.
So, that's the other kind of factor that you should consider when you're targeting countries because that would really make a difference. I think those are the big picture things that you can look at when you first start to consider countries and even look at countries where a trial was run before.
But then you have to look at the standard of care, then you also need to look out onto the horizon about how that standard of care might be changing. And I know, Maya, that you actually have products that address this topic about reimbursement rates for different drugs.
Maya: I was actually speaking with [00:07:00] lots of people that it's super interesting when you plan your clinical trial - it might be couple of months, even a year before you start. And then by that time the landscape can change. So we're even rethinking our products because we have the software for planning of trials, but how can we actually make sure that people know how the landscape changes so that when the time comes to start, the people behind it, clinical operations people and project managers, know how much is different than when they did the plan. So yeah, absolutely relevant.
Dana: Yeah, it can change dramatically. We have plenty of studies where even during the conduct of a trial a comparator may be reimbursed, where it wasn't previously, and then the trial becomes completely unattractive in a certain country and even potentially unethical because there's a better treatment on the market and there's no reason to put patients in experimental treatments. It is a real issue and certainly sponsors know [00:08:00] generally the competition and the environment of their own drugs, but it's something that needs to be watched very closely especially in areas such as oncology, where you do have a lot of expensive treatments, in which the reimbursement and the access can be changing all the time.
Maya: So when should you actually consider recruitment rates?
Dana: As a CRO we have to consider them right upfront, because all clients want to budget and they want to know the timeline. So, we always have to make some assumptions and we try to do that through looking at the background situation, but also doing a lot of detailed feasibility because the things that affect ongoing recruitment all the time are competitive trials that might be running in the same indication, or competitive trials that may also have [00:09:00] different reimbursement rates or just a more attractive protocol design. So that is something that can obviously be flushed out by speaking to investigators about what other trials they've committed to in the next 12 months, if the trial design would be attractive to them compared to other studies that might be running at their site. So, that is information that can certainly be identifying through conversations with stes. But that is something that certainly has to be done on that level. You don't know until you speak to investigators directly. You don't know what other trials that they have been contacted about they're considering. And investigators are certainly key opinion leaders who are contacted all the time, make their own decisions about why they may or may not want to participate in a trial. They may have had a bad experience with a sponsor in the past, they may just not like the drug in the trial, it may be too much work. Maybe they're [00:10:00] short- staffed and they want to pick a trial that has fewer procedures or a shorter timeframe. That's why if you run one trial, you've run one trial, everything needs to be considered individually for what it is and have those conversations to ideally identify investigators who are excited, motivated, and who want to participate in your trial, because you can tell right off the bat when you're pulling teeth from investigators if they're not really committed to the study during the feasibility phase.
They're probably not going to do a during the enrollment, and those are things that you can't see from a list of names of investigators. You have to speak to them to find out.
[00:11:00]
Maya: So if I'm to summarize, you need to have a recruitment rate in order to calculate the budget at the end of the day.
But in order to understand whether you should be more optimistic or the other way around, you should pay attention to these other factors like competition, standard of care, the landscape, all of these things that contribute to the final recruitment rate at the end of the day.
And make sure that you have a realistic expectation of what should be the speed of your clinical trial.
Dana: If I could just add one thing. I think being realistic for sure, but I think it's also important to build in some sort of risk management into that, because even when you're being realistic, you may think that you've taken all your information and you're not being overly optimistic, but as we said, still things can change. [00:12:00] Wars start and countries you expected to participate in your trial just are not available anymore. So, even when you've come up with your best case, it's always good to still apply some risk management analysis to that and still consider if there are other things you could do to better guarantee meeting that timeline, whether it's adding in a few extra countries or adding a few extra sites in a country just to make sure that you have some cushion and that you're not relying on everything going right even in your scenario, because, obviously, there are shocks and things can change external shocks that you can't predict.
Maya: So if I'm to translate this message to biotech founders and biotech managers, make sure that you have some extra budget and don't always pick the lowest budget offer, but pay attention to the one that really makes [00:13:00] sense. And I guess, a recommendation for when it comes to this interaction between sponsors and CROs is just to be transparent. Things happen, so we need to be prepared for all sorts of situations. I want to bring you back to the conversation around the sites and the keeping leaders. You said something that I absolutely believe and it is that if an in investigator really believes in your study and wants to be a part of your study, you can definitely tell.
But one of the things that a lot of our customers are asking are, for example, site level recruitment rate because it's perceived that it's closer to the reality if the site is going to perform or not. So what's your opinion? How can you trust a little bit more when we speak about site level recruitment?
Dana: The challenge with site level recruitment rates is whether or not they have really read [00:14:00] very carefully the protocol before giving their own estimates. And if they really understand the requirements of the trial, because the biggest disappointment is that the site says: "We'll enroll one patient per month, that'd be 12 patients a year." And then when we get to the site initiation visit and we start discussing and they finally understand the trial , they tell you 8 reasons why they're not even going to do half of that enrollment. Because I think a lot of times we try to make things easy for sites. We don't want to send them 15-page questionnaires about the trial. So we try to summarize inclusion, exclusion, and the things that we think look most difficult. A lot of times it's really important to focus the investigators on the challenges of the trial when you're conducting feasibility and not just what's attractive about the [00:15:00] trial, because that's where a lot of the disappointments, misunderstandings, and hurdles appear later.
When they finally really understand what types of procedures they need to do and how much time things will take, then they're already disappointed that they agreed to do the trial at that price, and now they understand what it really is. And now their excitement has turned to disappointment.
So that's during those initial feasibility conversations - in order to have more reliable information, the best way to do it is really to have an actual conversation and not just rely on written information that you send because you just don't know how much people read. If you really want to be able to trust those numbers, then the way to do it is to really have an actual conversation with a principal investigator and as I said, highlight the challenges of [00:16:00] the protocol.
And really try to push back. Because generally speaking, we usually try to push back on the initial numbers that a site gives us. Some CROs use a shorthand of we just cut it in half or whatever.
Maya: I've heard these stories as well.
Dana: Yeah, but I think actually spending the time, and time is money, but it's worth investing that time in the front end to have a real conversation with a site and understand, and make sure they understand the study so that the numbers they're providing are reliable.
Maya: I think that having some historical numbers for how the site has performed before does help you. I agree very much with all these people that I've been discussing this with, but at the same time, I think that you still need this conversation, because, if you were successful yesterday, it may be for various reasons and that doesn't mean you're gonna be successful tomorrow as well.
And I love what you said. You have to sense that this investigator [00:17:00] is really excited about your project and you know why, but also that they understand what's behind the clinical trial, what the challenges are, and make sure that you adress that upfront, and same with when sponsors and CROs are discussing the clinical trial, are they both aware of what the opportunities, but also the challenges are and how they're going to overcome this together?
Dana: Yeah, I mean, I could say on that point, we have worked on a study for a small sponsor who felt that their study, their protocol was very similar to a study that a large pharma had run previously that had finished, and they had the information about which sites participated in that and they could see how the trial enrolled.
So, they really expected that their study would run the same way and they had expectations of very good enrollment from particular sites in the UK in this case. They basically tried to replicate the study and they got almost zero enrollment in the [00:18:00] UK and in the prior study they had done something around 500 patients.
And actually what happened was in the time between the first trial and the second trial, that principal investigator retired.
Maya: Ah.
Dana: and the people who came, who took over the site, totally different people.
Maya: That's a great story. Can you tell me another story where you were misled by the recruitment rates and it turned out that it's not gonna be what you expected?
Dana: Well, unfortunately, that does happen pretty often.
Actually, we just completed a study recently where it goes to both sides of being upfront a very complicated study with dialysis patients. The study was looking at different types of lesions, and the protocol was written in a way that the investigators estimated the patients that they believed they had.
But when it came to actually running the [00:19:00] protocol, the sponsor, when they saw images, they rejected a lot of the patients as their condition was not severe enough. And certainly some of the sites were frustrated. They felt that the patients complied with the protocol and they wanted to give them the treatment.
But the sponsor had different criteria for judging those images. And so it ended up that the enrollment was not what was expected. The sites were frustrated because they thought they had more patients, but the fact is that a lot of times we see that type of situation where protocols don't necessarily have the last detail of precision and expect one thing out of the patients or the trial and investigators read the protocol maybe in a more broad way and then there becomes a [00:20:00] mismatch. So, that is one situation we had recently. We also sometimes have cases where, obviously inclusion, exclusion criteria can have a range. There could be ranges depending on what type of lab assessments are used to enter the trial in the inclusion, exclusion criteria, there's generally some sort of ranges and sometimes investigators will put patients who are very close to the edges of that, and these patients meet the criteria, and then the sponsor may say, "Oh, but we didn't really want those patients, we wanted more severe patients." And the investigators will often push back and say, "But they comply with the protocol."
A lot of times that will lead to a protocol amendment, where a sponsor realizes that their protocol wasn't really necessarily that precise or they realize that they actually do need a different patient population than they [00:21:00] originally thought.
So, there's lots of challenges even once you get started, because it's really rare that there is a protocol with zero amendments.
Maya: Yeah. Well,
we all know the numbers.
Dana: An amendment, obviously, your recruitment is probably gonna change in some way.
Maya: Well, that explains again that failing to plan is planning to fail. And maybe we should spend a little bit more time to do our homework prior to starting the clinical trial.
One thing that I hate about the industry is exactly like this. The very early days of communication between clinical research organizations and sponsors, this RFP process, it has these such short deadlines. And these deadlines are usually required by the sponsors, by the way. And that's not in their favor actually.
They're not in their favor at all because the less time you have on assessing a protocol and the clinical research landscape, the more mistakes you can [00:22:00] make. Some of the stories that you're now sharing with us actually describe exactly the situation and I'm sure there are just so many other stories to tell.
I wonder, was there any case where you knew about their historical recruitment rates, but you knew that's not gonna happen? And you started a trial and you were proven right.
Dana: We don't want to be proven right when we can prove the sponsor wrong. We try to counsel them on the front end about whether we think it's unrealistic. But I have a study right now actually where we were brought into the trial later and the sites were working directly with the sponsor and the sites gave them incredibly high enrollment figures, which as a CRO, just understanding how sites work, I knew that they could only do a [00:23:00] fraction of that. But the sponsor believed them because they're the principal investigator and they know their patients and they set up the trial with expectations to get those patients.
And we don't want to be the pessimistic ones, we don't want to be the people who are not being up to or enthusiastic to try to help reach these goals. Of course, we want to, but we try to explain to sponsors that sometimes sites will make calculations. They use very deductive numbers if it's just asking how many patients you treat per month. They are starting with a big funnel and then ask how many patients are applicable for this trial.
How many could you enroll? It sounds very logical to make that sort of funnel, but a lot of times there's too many that drop patients out, and some of them we've talked about, but a lot of times nobody wants to say, "I have 300 patients and I'm gonna enroll [00:24:00] 2."
That sounds ridiculous, but the fact is that the amount of effort and the amount of staff they have to put the patients in the trial, even if they had 300, still may only be 2 because it just takes a lot of time. So sometimes sites can be misled by looking at this top line number, but the fact is sites are so strapped for resources right now.
So, even if they technically have the patience, they don't have time to consent those patients, to talk to them about the trial and then the requirements. So it could be a very dangerous way to make estimates. But a lot of times, even sites will on their own. We'll forget about all the work and forget about what really needs to be done. And they just think, "Yeah, I have the patience." So, we have a situation like that now, which is obviously not [00:25:00] a happy one. We have added a lot more countries to try to bring up the enrollment. But sometimes you can see right from the get-go that things are just not possible.
Maya: And let's not forget that, at the end of the day, it's within the sites' experience and effort and resources to get the patients, but it's also the patient's choice, right? And sometimes that cannot be easily predicted, especially if you haven't done a proper job understanding the patient journey, the patient needs and challenges and barriers and all sorts of things.
And I like to say that clinical trials are like black swans. There is one great book by Nassim Taleb, it's called the Black Swan. And it has so many similarities to clinical trials. Sometimes black swans can occur when you conduct a clinical trial.
And you can't predict that the only thing you can do is just [00:26:00] be flexible. We've discussed it so many times that the key to success to every clinical research organization is exactly being flexible and trying to change your strategy, even if you have to do it in the middle of the trial.
But just follow your goal and be flexible and try to be transparent with all the stakeholders.
Dana: Yeah. I mean, I think as you said, people see a black swan and think, "Wow, well, I'm going to be the next one."
Maya: Yeah, you don't know.
Well, Dana, that's been a really insightful and interesting conversation. I hope that everyone really enjoyed our conversation and if you want to continue this discussion with Dana, you can reach out to her on LinkedIn and find out more about August Research and all the incredible stories that you guys have.
Thank you so much for being with me today.
Dana: Thanks for the opportunity, Maya. It was great to see you.