Trials with Maya Z

[00:00:00]

Maya: Hello everyone and welcome again to Trials with Maya Z. I'm today here with a very, very interesting person, uh, Jennifer Tinger. And why she's extra interesting is because she comes from the broadcasting industry and now she's in the clinical, in the clinical research space, which is amazing.

Uh, and it also validates my hypothesis that [00:01:00] we can learn from. The other industries are a lot. So first, let me give the word to Jennifer to present herself and then I can jump with my questions, around what makes or breaks clinical trials and challenges around our industry. So Jennifer, thanks for coming.

Jennifer: Great. Thank you, Maya. Thank you. It's great to be here and I just love that you're doing this, uh, these interviews and these videos. I think it's wonderful. We need to continue this conversation, uh, to help us all improve and, you know, just. Always get better, right? As, as an industry and its people. Um, I currently serve as president of Narrativa, which is an AI tech company, and a tech software company.

And, um, you're, you're totally right. I came out of the broadcast. Uh, I have a background in television and in film. Um, I worked in the media industry for close to 17 years, um, and I loved it, but at the same time, I was always interested in [00:02:00] tech. I, I had a great, um, drive to be more innovative and even, you know, close to.

Almost 20 years ago, um, I actually worked with the guys who started Hulu. Um, I was part of kind of their first launch. Um, so even then I knew technology always is changing industries and just as technology changed the media industry as we now experience it, uh, with O t T over the top platforms, um, for content, I knew when I started with Nativa, Um, in 2019 came on board to help launch Nativa into us, uh, in, you know, the US market.

Um, I really was excited because I said, this is how technology is going to improve and even help many more people. In the life sciences industry. Um, so it's been a wonderful journey. I still feel like I'm so green and new to the industry, but like you, Maya, [00:03:00] I have made so many wonderful friends, people who are very excited, open to new things, and even open to new faces, to new people who can come in, bring a fresh perspective, and we can collaborate and build together.

So it's been really a great, exciting journey so far.

Maya: I agree with you, Jennifer, that um, this industry is super important for every one of us, and not many people recognize that this is the industry that defines our next. Like the future of medicine and, and, and, and, and, and healthcare as a whole. Um, so I'm also like when I first started in clinical research, I also came from a completely different background.

So let's say I, I, yeah, I have a similar story like yours, uh, but I know how important it's to bring more people like you to the clinical research industry and. Yeah, we need more change-makers like you, let's put it that way. Speaking about change makers, um, I would love to learn more about narrative and [00:04:00] your technology there.

And I, of course, have some background information. I know that you guys worked on, uh, one really exciting project with us and I think that that can transform. Many stages of the clinical research space. So yes. Can you just tell us a little bit more about the technology first And if you can give us this case study, that'd be great.

Jennifer: Absolutely. Absolutely. Yeah. So the first thing Narrativa was started just as an AI tech company to literally automate content. founded in 2015, but then, in 2019, we adapted and moved into the clinical trial space, because we had quite a few large sponsors, pharma sponsors who were asking us, Hey, can you automate?

Regulatory documentation for clinical trials. And we were like, well, maybe we don't know yet. That's a very big question. but we took our, proprietary AI software. That, [00:05:00] is really, fast and amazing. I mean, we've perfected it over 9 years now, and now, since 2019, we have been automating patient narratives.

now we're automating TLFs tables, listings, and figures. Even now cross-reporting and then CSR. the clinical study reports, and what's amazing is that. We are, we have some really phenomenal use cases, like with the Leukemia and Lymphoma Society, which just became a client of ours, at the end of 2022.

And now we just completed, um, a trial with them and automated all the patient narratives for a trial, a clinical study with them. And now this will be the actual first time a whole study is submitted to the FDA. With AI-generated reporting, that's a huge breakthrough. I mean, people need to understand that.

This is amazing. This is not just great for l l [00:06:00] s, this is great for the whole world. So I always say this, there are always opportunities to improve processes. And the number one question I get whenever I'm talking with clients or even with an audience, um, I get a lot of questions from people saying, Hey, that sounds like a great automation tool, but it's going to take our jobs.

Maya: That was my next question. What do people feel about it?

Jennifer: Yeah, and I have medical writers. Um, I like I spoke on a panel last summer. At a conference with a lot of medical writers, and I mean, it looked like when I started sharing about what we were doing to automate medical writing and, even automating much of the work that biostatisticians and programmers do, um, they were pretty angry at first.

They, I got a lot of very intense, uh, questions because people were like, no, you're gonna take our jobs, et cetera. And I continually reminded them. No, this is an AI medical writing assistant. This [00:07:00] is your friend. So if you want to have a life, maybe go walk your dog, be able to pick up your kids from school or you know, have an extra hour of sleep.

Um, this will help you. This is like saying, taking, you know, what we used to do on typewriters, right? And a year, decades ago, you know, then now we had computers, right? And that fast-tracked everything in our lives. So I say it's the same kind of adaptation. It's just a new evolution of technology. So you just adapt and it's just going to increase your process so you can actually work better, more efficiently, and actually have more time for yourself.

And the medical writers from L L S, the Leukemia and Lymphoma Society, were an absolute shock at how well the patient narratives were not just generated quickly, but the quality of the output was better than what they had originally done. Before. So, the wonderful thing is we work with the medical [00:08:00] writers to make sure the output is exactly what they want.

You know, because they've set the bar. I always say I need great medical writers. I need the top professionals to make sure that the output is the quality that we need, you know, for submission. And so they really loved it. Um, they said it even helped open up their eyes to see there were actually, you know, data gaps.

And, uh, there were actually some errors in the data that they didn't even know about. They had not seen so, We're going, you know, I think we're going in the right direction. Um, it just takes a while, you know, for everyone to get on board because this is a little bit of a disruptor, right? In a process. But I think we're going in the right direction.

[00:09:00]

Maya: Congratulations on that. I'll definitely share in my, in a comment when we share the episode, uh, the article with like more information about this case study because I think it's, it's very exciting. So yes. Uh, for everyone who's interested, you can learn more about it. Um, I wonder now that we started speaking about technology in people and how do they think about technology? now speaking with different companies and different stakeholders, um, do you think they're more, um, let's say, trying to embrace technology? Or do you think there is this technology fatigue out there?

Jennifer: that's a great question actually. I think both if you look at it, Many people at this point I think are much more open. And excited, maybe even some of them excited to embrace new technologies like AI and Chat, G P T and Dolly and all [00:10:00] these new fun things you can play with. Um, but at the same time, I know there are a lot of people who are very weary and, and scared right?

Of, of the. The cons, right? Not just the pros, but the cons that can happen when you have a technology that moves too quickly. And we've seen that. I think every technology out there has good and bad. I mean, look at social media. I always say social media was created for the community, but it's also been very destructive, right?

And many people's lives. So, I'm really proud of the life sciences industry because I think we are taking our time. I'm not upset about taking my time. In fact, I. I feel like sometimes we go so fast. I want a little bit more time to make sure we're perfecting a product that has the reg, you know, the compliance issues settled.

Um, you know, that we're talking about even, you know, the protection of data, right? With PIIi, personal identifiable information, how are we [00:11:00] protecting that data? Our team personally, on a, a personal note with Nativa, our team has worked extra hard to make sure that our software, our AI software actually only uses, um, like, uh, birds and, uh, GT three, well, no, sorry, GPT two because GT three, we tried GPT three, but with compliance issues, it's still an open AI model, so there's no control.

Okay, over where that data's gonna go, and that can actually be dangerous, right? So we have actually scaled back a little bit and used other types of artificial intelligence models that are not open. So that we can make sure to maintain, um, our compliance regulations. You know, with p i I, um, so there are a lot of people very concerned.

I am even concerned with other industries like, you [00:12:00] know, images and, and people, you know, doing fake, uh, fake images of, you know, who could make a video of our president saying something that's scary, right? Um, but. Now I think there are people actively working hard to, uh, make sure that there are, you know, guidelines, I would even say boundaries set up.

And in fact, I'm gonna give a little shout-out to the New England Journal of Medicine. Um, I just was talking with them last week. They have a whole new division that they are launching in June called their, uh, NEJM AI Journal and it's gonna be all about just ai. And I am really excited about this because I think the more people talk about the risks and talk about the good and bad, then the more we can be prepared, right?

We can safeguard ourselves, um, and not be going in blind. I never want us to go into something blind. I think we need to [00:13:00] have as much information prepared ahead as we can ahead

Maya: Mm. Yeah. Yeah, you're right. Um, you partially answered my next question, but I still want to ask that because I wanna see if there are other, um, another perspective. So, Generally speaking, what are these challenges, um, when we speak about AI technologies entering life, the life sciences industry, like, which are, which are these main challenges for the, for the AI technology?

Jennifer: Yeah, I mean, in general, it depends. I mean, ai, some people think it's, it has a brain of its own. And yes, it can do, it can do reasoning, but it will only do as much as you feed it. I always say it's like a brain. You're programming a brain, um, a knowledge graph and it can only do as much as you program it to do.

Okay. So it will. Uh, people think that it's gonna become like its own being right? Like a human being. I said, no, it, there are limitations [00:14:00] still to ai, but the problem is, um, we are seeing, I think there is a little bit of a tech overload at this point. I think one of the problems is that AI. we, we should not become too dependent on it.

I know, and I'm, that's strange because I run an AI tech company. Uh, but I think we should always make sure that people, human beings have the final say. I've had clients who want everything automatically published from our software. That's great. But even my team, I always tell them, to listen.

My, um, editors or my automation editors, let's always make sure there are at least one or two QA quality assurance steps in each process. Sure. That the output is exactly as we want it because it's still a machine. Like yes, it's doing it quickly. It's doing it a thousand times faster than we could. Um, and it's not getting tired like we do, uh, as humans, but I [00:15:00] still think our minds.

We have what's called intuition, and we have what's called morality. Now, that's, I'm touching on something bigger. Okay. AI does not have morality. Okay. It has a little bit of intuition, like, you know, patterns. It can, it can pick from, But AI does not have morality and ethics. So when you're com talking about business and especially clinical trials, especially life sciences, there is something in us as human beings that we have a moral code and hopefully, we still have it.

Cause looking at our world, it's a little scary, but I always say, listen to your moral code cause. Maybe you'll get great output, let's say, you know, reporting, uh, for patient narratives. Okay. but something in you as a human being will look at all this great output for the patient narratives that were generated.

But you'll have to be like, oh, you know what? I'm gonna just go [00:16:00] back and double-check that one more time. So I need a great medical writer who has had many years of experience and says those two numbers. Yeah. They were generated from the data. But guess what? I still think there's something missing. And sometimes, and the AI will catch any data gaps, but a human being with just intuition, you know, and with this gut check, they'll be like, the, you know, the numbers are not adding up here correctly.

Let me just go double-check. And so that's why in our process we love ai. I think, you know, we're an AI company, but we actually put in the, um, patient narratives that are generated. A whole sidebar on the Word documents that are outputted, and on the sidebar it will actually put little notes or highlighted notes for the medical writers to check because the AI knows something's missing.

But I still need a great human being to [00:17:00] go through and say, oh, yep, that's what was missing, or drug was taken, you know, at the opposite time it should have been, or at the wrong time, it should have been taken. So I, I see the good, I see the bad. Now when it comes to one of the other cons, okay. About ai, one of the bad areas of AI is I think we should never become so dependent.

On software, we forget the human touch. Now I'm gonna move a little bit more over to med devices. Okay. There's a lot going on about, you know, medical devices or, um, even decentralized trials and how we can use AI for all kinds of, Uh, data collection. Right. Which is great. I, I love data collection and I love wearables.

I think there's, it's a huge industry now. Um, it's def definitely changing clinical trials and how they're conducted and the data that we actually can use to help improve, you know, um, [00:18:00] people's lives and help improve studies. But at the same time, everyone needs people. We need a human touch. We cannot replace human beings with technology because as we learned during covid, I mean when we're not around people, we suffer.

We suffer mentally, emotionally, spiritually, and physically, right? And I mean, even, you know, I'm on Zoom calls all day long for my job. My job's a hundred percent remote. I still need people. I need someone to touch, someone to hug. And especially when I'm sick. And I think of, you know, I have loved ones and family members who even now have cancer and I tell them, listen.

You need to hug someone today, you need that physical touch because it actually helps heal us. So we cannot forget. AI is wonderful. It has many great benefits. At the same time, we cannot forget the human element. The human benefit of touch is very vital. So [00:19:00] that's where I see like people are scared of ai, but I think we, there are some good things that AI is doing, but we always need to make sure

Maya: Yeah, agree. So if I'm to summarize, when speaking about challenges for AI entering the clinical research industry, life sciences in general, one of the challenges, um, let's say people's fear that that will replace their jobs or everything basically in the human touch. And I agree with you, that we will always need the human touch.

At least people will need people. So like, definitely. But what about the challenge of the old habits, do you think that old habits are a barrier to AI entering more into the life sciences industry?

Jennifer: It could be. I don't think the industry will completely adopt what we're doing with, AI reporting. The whole industry, it will take at least five to seven years, right, for the whole industry to adopt it. But it could go faster. We don't know. But at the same [00:20:00] time, I think, um, there's a generation Of, leaders and medical writers, they're going two directions and I've met them. There are either those leaders who are going in the direction of innovation and those who want to stay the same and they want to stay in their same processes and not have any disruptors. Um, so I think that you know, with ai, it's going to be very difficult, you know, for some people and.

They, I hate to say it, with their jobs, um, they're going to have to either adapt or they may lose their job. But I would encourage everyone to adopt because learning something new, you know, like learning Microsoft Word, I remember when, you know, or when Outlook came out or any of this new software, Salesforce, right?

anything that's new. You have to learn it to adapt. And I say the more open you are to change, the [00:21:00] stronger you will be and you'll have greater job security regardless of your age, regardless of your age. I have close friends who are in their sixties and seventies, they're brilliant and they are still adapting.

And you know what, it's, it's amazing because they're inspiring to me. They just keep going with this new, you know, any new change and saying, yep, okay, there's always a better way to do something.

We, we should never say we've arrived. Never. We should always say there is always something to be improved. Um, so that's what I think we should look at, I hope I answered that in the way you wanted, but I

think that should look at it.

Maya: Interesting. What about data, do you think that com, because AI is like, needs data at the end of the day, like, especially for what, what you guys are doing with narrative, uh, you are structuring data and, and creating, generating. Like insights and, and information out of that. Do you think, uh, and what's are, what are [00:22:00] your observations, do you think companies and their internal, existing processes and, and like, and data, existing data, are they prepared for changing to an AI approach?

Jennifer: Hmm. That's a great question. Um, a lot of the issues we're finding with data because yes, uh, AI needs data o even run. I mean, you can't, we, when we first started in clinical trials, Um, it was hard for us to even create something for clinical trials because we had to first have data because it's machine learning and it can't learn unless it's, um, constantly ingesting more data.

So that's why we ran quite a few trials with small, medium, and large sponsors. Um, and that data helped build the whole software system that we have now. But I would say the big issues we're seeing now is that We keep kind of going upstream [00:23:00] in our solutions for clinical trials because we first started with CSR, the clinical study report, and we were trying to automate sections one through nine, section 10, section 12.

But some of the big issues we were finding were with data. The data because by the time we got down to the end of the stream with the CSR, every, SDTM was locked, right? We couldn't, there were issues with the data. Huge gaps in the data errors, and so we were like, well, let's take a step back. So we were automating patient narratives.

Then we needed to take another step back. Now, we gotta automate TFS because they're issues tables. So now we're taking another step back and we actually now are automating, well, we're actually gonna be working on a new tool. This is kind of big, um, that will be a data analysis tool on our platform on the EVA platform so that even in mid-trial [00:24:00] teams can upload there, their SDTM or there, or their data sets and they can upload 'em and actually have run like a cleaning process to see what's missing.

So it's a whole analysis process to say, Hey, you're missing, you know, these dates or these dates are wrong. These, you know, seem out of order. Or this person, you know, there's this person's 400 feet tall. How, how does that, you know, so you always have those issues, right? With data collection and. Um, from your site, uh, leaders.

And so when you're having all this information coming in from sites, um, you know, you definitely need greater data analysis so that the AI can. Smoothly. So we're actually looking at quite a few other companies who are great with data collection. We're looking at maybe even partnering with them. Um, there's no announcement yet, but we think there's definitely some synergy and complimentary [00:25:00] tools that can work together to make that a more seamless, easier process.

Mm-hmm.

Maya: Hmm. That's great. Well, I like your approach, how you're kind of like seeing the vision for the AI in the life sciences industry in your particular, uh, space, but then you kind of like go a couple of steps. Before that, make sure that you accommodate for what's feasible at the moment for the industry and kind of like guide the industry to the next thing, and then the next thing, and then the next thing, until you really kind of like together achieve this vision for what can AI do for, for the industry.

And I'm pretty sure that along the way you're gonna learn a lot more things. Um, but that raised, raised another question. when companies hear about these solutions, especially AI and innovation and, and technology that usually like, kind of like, uh, comes with these pictures of expensive and money and you know, dollar sign.

So do you think that AI and these solutions [00:26:00] are just for the big companies, the ones with the big pockets, or do you think it's the opposite? It makes these solutions like AI makes these solutions to be accessible even to the smaller ones with smaller budgets.

Jennifer: Yes. That's a great, great question. I love that. Our solution so far has been most quickly. Adopted and used by SMBs, SMB biotechs, and pharma. I mean, even now we're talking with CROs and what's wonderful is, I mean our, clients that are using it, the average client is, you know, maybe smaller, medium-sized biotech.

some of them run maybe 2-3 trials a year, some run 25, you know, or, but really, what we're seeing on average, Now, I think as of this year we ran, some studies that the average client is seeing at least a 40% reduction in cost and at [00:27:00] least a 65% reduction in time. It's amazing.

So yes, the large sponsors, we are still talking with them. If they adapt, you know, want to bring our technology in-house, that's wonderful. but we really want to. Hit the majority of the market. I mean, that sounds crazy, but I want everyone to say, yes, this makes our lives easier. And, um, it, you know, helps our budgets, it helps our time.

Um, it just makes our teams more effective. So that's why we really have been hitting hard, the SMB market, you know, the small market. Um, but you know, we have quite a few large sponsors, uh, coming on board with us, and they're. They're excited. But as you know, with those larger sponsor entities, it takes a while.

They're slow, you know, in, um, and there's lots of people that have to sign off, a lot of stakeholders, so it just takes time as everyone knows that.

Maya: Well, it's exciting and it's [00:28:00] important when we speak about technology, to also make sure that people understand that technology is not a luxury. It's actually the, it, it can be a commodity. It's actually something that everyone can leverage, and especially ai, um, and all these new technologies, they just make it easier for more people, more companies, businesses.

Like for single individuals to actually access things that we've never been able to access before. So it's great that you target this group of businesses. Um, I know they're a lot more, I can't say open, but at least they're more agile in making their decisions. So, I really hope that you can create more and more of these case studies and success stories and, I hope next time when we can discuss another success story.

But, um, it's definitely inspirational to see what you guys are doing, and I just cross my fingers for you. Thank you so

much, Jennifer, for sharing this story with us.

Jennifer: Thank you. Great talking with you. Thank you so much, Maya.

[00:29:00]