LinkedIn – Connect with Heather.
Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.
Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.

[INTRODUCTION]

[0:00:03] HC: Welcome to Impact AI, brought to you by Pixel Scientia Labs. I’m your host, Heather Couture. On this podcast, I interview innovators and entrepreneurs about building a mission-driven machine learning-powered company.

This episode is part of a mini-series about foundation models. Really, I should say, domain-specific foundation models. Following the trends of language processing, domain-specific foundation models are enabling new possibilities for a variety of applications with different types of data, not just text or images. In this series, I hope to shed light on this paradigm shift, including why it’s important, what the challenges are, how it impacts your business, and where this trend is heading. Enjoy.

[INTERVIEW]

[0:00:49] HC: Today, I’m joined by guest, Paul Hérent, Co-Founder and CEO of Raidium, to talk about a foundation model for radiology. Paul, welcome to the show.

[0:00:58] PH: Thank you, Heather.

[0:00:59] HC: Paul, could you share a bit about your background on how that led you to create Raidium?

[0:01:03] PH: Yeah, sure. My background is first, radiologist. Radiologist by training in France, where I was trained even years training, so quite long. I also have a cognitive science degree. I posed my residency doing two years to learn about neuroscience, knowing that I wanted to do AI. With the weak signals, we have seen occur in the late after-images, so in 2012. This is where the AI story began for me. I joined a little startup called Owkin. After this post, I started to work on AI quickly, so I made my medical thesis on this topic in 2018, and this is where the story began. No, I’m not an employee. I’m a co-founder since three years.

[0:01:53] HC: What does Raidium do, and why is it important for healthcare?

[0:01:56] PH: Raidium is a new AI-based on foundation model. We will deploy this, define more processes, what we mean by that. The goal for us is to have AGI for radiology, if we had a one-liner to frame. By meaning, that we want to try it to the first generation of AI, though one AI for every radiological use cases. As a radiologist today, I don’t choose any AI existing solutions. To give some context, in radiology, we have seen many, many first AI solutions for medicine. So, there is quite a few years in the field. But no, maybe apparently, crowded market with many, many AI solutions, but in practice, there is still a few adoption, because every solution solve only a tiny part of what radiologist do. What do radiologist, it’s a wider job, you do a diagnosis on vertical modalities. It can be 2D images, 3D images. It’s a very growing demand, so it’s key specialty in medicine.

We want, as a radiologist, to have one tool to rule all modalities. Today, we use a so-called viewers. So, it’s a console that allows us to see every kind of imaging, whatever, it’s a CT, MRI, X-Ray, you use your viewer. What we do at Raidium is we build an AI native viewer that allows to interact with the AI and control what the AI has to do to simplify the life of the radiologist. Beyond simplifying workflows, we have a strong willingness to build, continue biomarkers for precision medicine.

[0:03:44] HC: You mentioned foundation models as the core piece of this. Why is a foundation model important here?

[0:03:51] PH: Why? If you try generic AGI applications, such as ChatGPT, or [inaudible 0:03:58] app, you can put images, and you can put medical images into it. For instance, if you put some basic X-rays, if we say, okay, it’s an X-ray, you can sometimes make assumptions of what it’s seen, but it’s still a gadget today. It’s not as a level of performance required for imaging used in the clinical, or research conditions. This is why we are relevant. We build a dedicated specialized foundation model. Why traditional AGI solution doesn’t work? It’s trained on Internet data, so it’s – you have some images, medical images in these data sets, in Google images, for instance. But it’s very, very few compared to the amount of data we have in hospitals.

On our side, we have the chance, since the day one of Raidium, of having a massive diverse data set with our first clinic that we partner with, that are with us to have at least one billion images, plus a radiological report to train the model. We have what is required to do that. It’s not triggered to manage such amount of data, but we think this data set is an asset for us, because we already have very convincing benchmarks that show it works. When doing it, if I rephrase, go back to the question. We think the model need to see a diverse set of data, so we need to see what’s in the given organ from the head to the feet. We need to recognize the different patterns, different shapes, not more on pathological ones. Learn to be good in a very wide state of clinical situations. It’s the backbone of what could be AGI in the future, the very holistic representation of a humanitarian.

[0:05:51] HC: What types of data go into training this model? You mentioned the CT, EMR, and radiology reports. Are these the core components, or are there other pieces to it?

[0:06:00] PH: Basically, this is it. You have many competitors that try to do foundation model today. Mostly of them are doing 2D X-ray, for instance, foundation models, or model ET, CT foundation model. What we do is we have a diverse set of CT, EMR, X-ray in the app. We have a particular focus on complex 3D imaging data, the CT and MRI, notably. Because this is where is the main need in precision medicine. We want to prove that we can solve complex use cases with this approach. This is, we think, the added value of foundation models, you can train in self-supervised way the model, so we don’t need to make a huge annotation work, compared to the previous supervised machine learning architectures.

For us, it’s a game changer, because with this data, we already have benchmarks in the very wide set of applications from the road, to on-code, to gather metabolic disease. We get confident. We have a very, very interesting result.

[0:07:07] HC: What does it take to build a foundation model? We’ve talked about the data, but what are the other components that go into building this?

[0:07:14] PH: Yeah. When you train this kind of model, it’s changing for many aspects. First, you need to hire a good team of engineers. It’s very challenging to do it alone. Maybe impossible. This is the first need we had, to hire the core R&D team. Raidium was founded in 2022. Now, we are 18 people. We have a significant team involved in this, in order to manage the complexity of this training. This is the first requirement. Also, requirements is that having access to enough compute. In France, we had a chance to have low carbon, free access for research to HPC. We could start to train since day one, the model.

[0:08:00] HC: What are some of the challenges you encountered while building the foundation model?

[0:08:05] PH: The first challenge was to make it work. We started with a few amount of data, curate the data before the training, or so, to anonymize the data, to process the data that is very wide. We have huge constraints on the legal side also, so we have to do that. It’s a pending process. It’s not ended, because we want to have a more, I would say, expansion. We want to expand the data set we have. It’s never ended. Legal challenge, compute challenge, and use cases change is a key aspect is, okay, you want to train the foundation model, but where is the added value compared to supervised machine learning approach? Can it do quite well to work for simple cases?

We wanted to show that foundation model are relevant for unmet needs. For instance, one of the first demos that we have is in oncology. In oncology, you have patients with different lesions in different organs. If you want to train a modern supervision in a way, you have to train one word for each organ. It’s very, very challenging. The first competitors that tried to do that focused on strategic organs, such as liver on end. We have done a preliminary work on this, and as per abstract, no preference, that is pending for peer-review publication, so stay tuned. Basically, we have a very good algorithm that works on a different set of at least five organs. It’s the first proof point we have delivered on the medical application.

[0:09:47] HC: This may sound like a fairly basic question, but how do you know whether your foundation model is good?

[0:09:54] PH: Let’s back in 2020-ish. We showed when we were at Owkin’s, the weak signals of the, again, the paradigm shift, whether they be in self-supervised learning approaches. Papers like Moco, Latino architectures, on vision, for instance, on transformers in general, showed that in the self-supervised learning way, you could outperform super managing learning approach. This was the weak signals before the GPT moment on the research side. That motivated us to start the company entirely.

We had the conviction that could build very different products, thanks to these properties. With one model, we can do many, many applications. One model for every use case is typically, if you see a ChatGPT, you can do an as many things to an LLM. We want to translate this ID for a generic specialty in medicine.

[0:10:54] HC: Do you have a process for benchmarking your foundation model?

[0:10:57] PH: Yeah, sure. Benchmarks in [inaudible 0:10:59] are very present. As said, we had a chance and readily to be pioneer. We have many, many category competitions, other conference, some generics, or BK, which are machine learning conference, which we have a strong interest in medical imaging. Many benchmarks in consequence. We can access the properties of our foundation model on the existing benchmarks.

For us, give more value of what we do. We don’t have to build the benchmark. At the beginning, we can show that we outperform the existing ones with a different set of existing metrics. We have coming papers on this in a few months, foundational paper to tease it.

[0:11:43] HC: As the hype for AI has taken off, there’s been an increasing concern for bias, especially medical applications. How does the potential for bias compare between traditional supervised models and this new approach of foundation models?

[0:11:56] PH: Yeah. Bias is a big concern in every machine learning algorithm. You can see the bias LLMS, for sure. You can see images bias in imaging algorithms, such as Dall E. It has been proven that some gender bias, or even racist bias can be present. For medicine, we are not exempted from that. The way to tackle this is to very diverse and massive data first. This is a way to manage this bias, because you have reached your distribution, basically. This is where foundation models are better than supervised machine and approach in the sense, because more data.

Another aspect is they are more resilient to bias compared to supervised approach, because they don’t use the labor to train the representation. When you put the labor, such as, I don’t know, images annotation, you induce a behavior in the model. When you do self-supervised learning, you are more systematic. The goal is to learn things that are very basic. Maybe not useful, but that could regularize the things that has to be seen that are relevant. This is the first way to mitigate this.

Another aspect to mitigate the bias is to have a human-in-the-loop approach. Once the pre-training is done, so we can refine, interact with the viewer we did with the model inference. Even if the model is wrong, we can see it and we can correct it. We have, in a sense, continuous learning that is possible with the design of the product.

[0:13:38] HC: How are you currently using your foundation model and how do you plan to use it in the future?

[0:13:42] PH: Today we are still in the first R&D phase. We have the first version of the model that is trained, the dataset. We are going multi-model. Today, it was large vision model. But we are going to a large multi-model there. We are including text. This is on the foundation model part, significant milestone to achieve, because it’s not trivial to do that.

On the medical applications, today we are three tracks that are already started. We have NASH disease biomarkers. NASH is a fatty liver disease. So, you need to assess some biomarkers are considered virtual biopsy. You can, for instance, quantify how much fat, or much fibrosis is there in the liver. This is very relevant for imaging and pricing clinical trials. We have done that for the first-time client. I’ve mentioned oncology. This is for some very key application, because it’s an innate need. It’s a pain point for radiologists when you do a clinical evaluation of patients and their treatment. You need to monitor them efficiently. Today is still manual measure. With our approach, we are much more efficient, much more precise. We have other applications coming in neuro and cardio, but not disclosed yet.

[0:15:08] HC: How do you plan to commercialize your foundation model?

[0:15:11] PH: The commercialization is quite challenging. Today, we start in life science business. Our first clients are not radiologists yet. It’s far more into these three medical devices in the three, where imaging is key for the given application. I’ve mentioned oncology for instance. Oncology is the first lead of imaging endpoint in clinical trials. We chose to crack it first. The way we choose and work is to not do only life science purpose, but to consider the medical application in routine for a second time. We build a viewer with a product that has an intended user in practice at the end. This is all we started to start for research and go to practice after.

[0:16:02] HC: Are there any lessons you’ve learned in developing foundation models that could be applied more broadly to other data types?

[0:16:08] PH: Yes. First data is key. If you have good data, not only to build data set, but proprietary data, challenging data, rare data in a specific domain. It’s very valuable, because the architecture is not particularly innovative. In a sense, we work with transformers. We are still a recipe for adapting it through 3D imaging data, complex processing before training. Data is the first key aspect for considering an application with foundation models. All to have massive data, the first question you should ask to when you want to start a project.

The other thing is have a developed ability with other existing models. For instance, we don’t want to reinvent the wheel when something is already solved. LLM for us, for instance, is an umbrella. We don’t want to train or fine-tune specifically some LLM that already works. Use existing APIs, instead of training by yourself, or fine-tune by yourself, if relevant, the second lesson. Third lesson is being a real need. Maybe not just do a take push, but see where you can paint points. It’s something that is not new on a specific foundation model, but we can see today many, many LLM for X reason emerging. The question is, is it useful?

[0:17:49] HC: What do you think the future of foundation models for radiology looks like? This could be at Raidium, or it could be more broadly.

[0:17:56] PH: Yeah, so foundation models are emerging everywhere now. We have the same approach for histology, the same approach for genetics, complex, transcriptomics data, or even multi-omics data now. The future is multi-model. This is not only radiology. In radiology, we have multi-model in the medical imaging domain itself. Many, many things to do that are not solved yet. If we see as a map of existing biomarkers required for precision medicine today, only a few tiny percent of these biomarkers are solved with AI. Many work to do.

The result back is the future of product based on foundation models. Beyond the model, we need to design, to think about the UX change, the life of the users you target so far. For instance, in radiology, we want to build an AI NTC world, which is the must-have tool for radiologists. We hope we will see adoption in that sense.

[0:19:03] HC: Thinking more broadly about your role as a founder, is there any advice you could offer to other leaders of AI-powered startups?

[0:19:10] PH: Yeah, sure. Don’t be alone. Build a team with people you trust. Entrepreneurship is not trivial. Be complementary. Even if you have an AI-native startup, it’s good to have maybe an expelled domain in the founder team. My case is I’m a radiologist, so it’s quite trivial, but we need so much time in defining the use cases you target, having an expert at home. This is something we can see in every domain. If you do legal, or maybe have a lawyer, or people walking in this domain, it’s very, very key, not only as an advisor. I think it’s good to have maybe expert domain COO, or key people in each future application. The first, advisor I will get.

[0:20:00] HC: Finally, where do you see the impact of Raidium in three to five years?

[0:20:04] PH: Yeah. The dream of Raidium is to build something that has a huge impact on patient’s life. Radiology is key. You can make period diagnosis. You can make acute labor treatment if you do this fighting cancer. If we have an impact on patients, it will be a middle bar. I mean, it will be mainly well. At the end, there is capable product. Since, this is why I mentioned AGI in radiology. Adding an expert in silico that again, solves the needs of medical imaging, much more holistically than today, would be very, very good for medicine in general. Because radiologists are long to train, even years for me. If we go beyond the rich countries, many, many people have no access to radiology. You have seen a metric as that. It’s two-thirds of countries that doesn’t have access to radiologists. It’s in the world globally. It’s a big need. If we can contribute with our approach to more accessible health, we will be very happy.

[0:21:19] HC: This has been great, Paul. I appreciate your insights today. I think this will be valuable to many listeners. Where can people find out more about you online?

[0:21:26] PH: Yeah, sure. On LinkedIn, we are present. We have our website that describe what we do and the paper we publish, www.raidium.eu. Yeah, ping us if you want to reach. We are hiring on globally than we are as in France. We want to expand in the Europe and the US very soon.

[0:21:53] HC: Perfect. There’ll be a link to all of that in the show notes. Thanks for joining me today.

[0:21:58] PH: Thanks.

[0:21:59] HC: All right, everyone. Thanks for listening. I’m Heather Couture and I hope you join me again next time for Impact AI.

[END OF INTERVIEW]

[0:22:09] HC: Thank you for listening to Impact AI. If you enjoyed this episode, please subscribe and share with a friend. If you’d like to learn more about computer vision applications for people and planetary health, you can sign up for my newsletter at pixelscientia.com/newsletter.

[END]