Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.
Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.
Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.
Foundation Model Assessment – Foundation models are popping up everywhere – do you need one for your proprietary image dataset? Get a clear perspective on whether you can benefit from a domain-specific foundation model.

Transcript:

[INTRODUCTION]

[0:00:03] HC: Welcome to Impact AI. Brought to you by Pixel Scientia Labs. I’m your host, Heather Couture. On this podcast, I interview innovators and entrepreneurs about building a mission-driven, machine-learning-powered company. If you like what you hear, please subscribe to my newsletter to be notified about new episodes. Plus, follow the latest research in computer vision for people in planetary health. You can sign up at pixelscientia.com/newsletter.

[EPISODE]

[0:00:33] HC: Today, I’m joined by guest, Razik Yousfi, Senior Vice President of Technology at Paige to talk about foundation models for pathology. Razik, welcome to the show.

[0:00:43] RY: Thank you, Heather. Thank you for having me.

[0:00:45] HC: Razik, could you share a bit about your background on how that led you to Paige?

[0:00:48] RY: Yes, absolutely. So, I’m currently the Senior Vice President of Technology for Paige. I joined the company about five years ago with really, the objective of scaling the organization and building the team that today is in charge of developing all of the different products and doing the research we do at page. Prior to joining Paige, I was in Silicon Valley for roughly seven years, working at a company called HeartFlow that focuses on AI for cardiovascular disease. I joined that company as an engineer originally, and left it in 2019, running engineering.

My background is in computer science. I graduated from the field of AI, joined Siemens for a little bit of time focusing on GPU optimizations for medical image analysis. I spent a lot of my career in technology for medicine, had a couple of side industry jobs for a little while. But by and large, I ended up at Paige, because there was a fantastic opportunity to apply my knowledge and all those years of experience to a field of medicine that traumatically benefit from the application of events and cutting-edge AI and technology.

[0:01:57] HC: So, what does Paige do? Why is it important for cancer care?

[0:02:01] RY: So, Paige, is today focusing on digital and computational pathology. In other words, we really bring AI and novel AI solutions to the field of pathology to help pathologists make better-informed decisions, increasing their ability to find the correct diagnosis, helping them with some of the tedious tasks, and ultimately benefiting their patients. We really do two things. One of them is providing a platform that enables pathologists to move away from the microscope, which is the gold standard today.

So, the platform really acts as a software and infrastructure system that ingests images of glass slides after being digitized. We store them in the cloud. We enable the pathologist to access them and view them. We provide a set of tools that facilitate the navigation of those images, and ultimately, helps with moving away from the microscope.

On top of this platform, we’ve been focusing on building a diverse set of AI applications, which range from being able to identify and detect cancer, to being able to read cancer or even infer biomarkers. Those AI applications are built on top of the base platform. The field of pathology is critical in cancer care. You don’t have cancer until ultimately, a pathologist tells you that you have cancer. For every suspicion of cancer, you typically go to your physician, and then ultimately, you end up getting potentially a biopsy. That biopsy and piece of tissue is sent to a lab where it gets prepared. It’s eventually put into class slides. Those glass slides are stained. Those slides are given to a pathologist. And the pathologist is then basically in charge of looking at those glass slides and looking at all the different cells and the morphology of the tissue to identify cancer cells to eventually enable you to get the proper course of treatment.

Pathology is really the cornerstone of medicine. It’s one of those fields that has not really changed dramatically in terms of process for the last 100, 120 years. But it is a critical field of medicine today.

[0:04:12] HC: And what role does machine learning play in all of this?

[0:04:15] RY: Machine learning, ultimately, for us is really the central piece for all of our products. There’s been a lot of application of AI in the field of healthcare. You can build very sophisticated systems that really help you automate some tasks and ultimately act as an aid to the physicians. For us, specifically, because we are capable of training AI systems at a super large scale, we can build a model that can, in other words, see multiple images, far more images than the pathologist can see in their lifetime. And by using those models and giving those tools to the pathologist that can not only speed up the rate at which they can look through different things, but we can also help them identify cancer where they may maybe have missed it. They can ultimately review the results of the AI and make their own decision. But without advanced machine learning systems, it’s very difficult to build AI technology at a scale where you can actually trust its results, and I’ll really lean on it to be able to help you do a better job.

[0:05:21] HC: One thing that I’m particularly excited about because I’ve seen it in the news a lot lately surrounding Paige, is your new foundation model capabilities. I saw a recent preprint about this as well. Could you elaborate on for the audience who might not be familiar? What is the foundation model? And when is it useful?

[0:05:37] RY: So, that’s a great question. Indeed, I think the field of, I guess, AI and the discussions around foundation models have been very, very much top of mind for everyone over the last 12 to 18 months. A foundational model is a model trained on a very large set of data. The idea there is that you can, in turn, use that foundation model to build a wide range of downstream applications.

So, in our case, specifically, it enables us to train one model that ultimately have an extremely deep understanding of pathology and biology because it’s trained on an extremely large set of images. And once you have built that model, you can use it to build a wide set of derived applications, which can range from the ability of detecting cancer across multiple cancer types, or the ability of identifying biomarkers on a bunch of different organs, or automating the generation of reports.

Without a foundation model, you typically have to not only find a lot of data for each of those different tasks that I mentioned, but you also need to get access to extremely good data, and you would also need access to a lot of compute for each of those different acts. So, you can think of the foundation model as something that enables you to move the frontier of AI because you have access to something that’s extremely sophisticated. But it’s also something that enables you to develop those applications in a much faster way because you’re not starting from zero. Every time you’re starting from a baseline that has been trained on an extremely large set of data.

[0:07:07] HC: Like you said, the largest benefit is that you can kick start a new project because you already have a model that’s seen a lot of different data, and you likely don’t need as many labeled examples for a new model.

[0:07:18] RY: Yes, that’s exactly right. So, if you think, for example, about cancer, a large proportion of cancers are rare. For those particular cancers, you don’t have a lot of data. So, if you were to build a dedicated system that’s capable of finding cancer for that particular type, you may not be able to get a really high performance because you don’t have enough data. If you use a foundational model, you’re basically training a model that understands the morphology of cancer across a bunch of different tissue types. By doing so, being able to customize or fine-tune this foundation model to this rare cancer becomes a much simpler task because you’re not starting from nothing. You’re starting from a foundational model that intrinsically understands the morphology of cancer. And by fine-tuning it to a rare cancer, you can achieve a high level of accuracy in identifying cancer for that rare cancer types. So, without a foundation model, this is something that wouldn’t have been necessarily possible. It’s not possible thanks to it.

[0:08:18] HC: What does it take to train a foundation model for pathology?

[0:08:21] RY: I think what it takes to train a foundation model for pathology is similar to what it would take to train a foundation model in other fields. But in pathology, specifically, you need a lot of really good data. By data, I really mean pathology slides here. So digitized copy of the glass slides, scan at a really high resolution. Dataset at Paige are extremely diverse. It comes from more than 800 institutions, 45 countries. Our cases are really coming from all over the world. And getting access to that data is critical because you don’t want it to be learning something that’s biased. You want to build AI in a reliable and ethical way. And you need to make sure that your source of data is representative of as many populations as possible. So, data is definitely one of the big building blocks of that foundation model.

But with that amount of data, you need a lot of compute, right? So, you cannot train a foundation model with a tiny budget for compute and GPU resources. That is traditionally where a lot of your capital goes. We are thankful to the relationship we have with Microsoft, and specifically, Microsoft Research because it really enables you to have access to that larger amount of compute. We started building a GPU cluster here in the United States about five, six years ago. We’ve since then expanded to using the cloud for that. But even with all the resources we have at hand, we didn’t have enough resources to be able to train a large-scale foundation model like we’ve been able to with MSR.

On top of the data and the compute, obviously, need to have an extremely talented AI team that’s not only pushing the frontier of AI on the scientific side but also on the infrastructure and the engineering side. We’re processing gigantic amounts of data. Our datasets is petabyte scale. So, you need to really build the infrastructure that’s necessary to break down those images and process them at a large scale. And obviously, you need to not only use cutting-edge AI algorithms, but you may have to invent new ones. So, you need to stay at the forefront of AI new innovation, and ultimately, also bringing experts in the field of pathology and oncology to be able to make sense of the findings and being able to refine the model over time. It’s really the combination of data, compute, extreme expertise in AI on the scientific side, but also infrastructure side, as well as our medical knowledge that really enables you to build a system in pathology.

[0:10:49] HC: Are there any numbers you could share as far as how many images? How much compute? Different things like that it took to build a foundation model?

[0:10:56] RY: There are definitely some data points I can share. We, in fact, publish a lot of our work. Not too long ago, we pushed a paper to archive which we would be happy to share with the audience. Our current system has been trained on more than one and a half million whole slide images, and I think it’s important to know that images in pathology are extremely large. There are basically millions of pixels in each of those images. So, they represent billions of traditional images.

A one-and-a-half million slide set of pathology images will present an extremely large set of traditional images, the way you would traditionally think about them. We’ve built a system that’s in the hundreds of millions of parameters. We’re currently working on a new version that’s actually tripled the amount of images we’re currently using, and we’re using hundreds of GPUs for that development.

[0:11:48] HC: You mentioned that training a foundation model for pathology is quite similar to how you might train one for other imaging domains. But I’m curious to know whether there are any differences? Are there any slight modifications you might make to an algorithm, or ways to think about getting your dataset, or anything at all that stands out that’s different?

[0:12:05] RY: Yes. I think there are a couple of subtleties. The first one is what I just mentioned around the size of the images. There’s obviously the field of large vision models, that’s training foundation models in the field of imagery or even medical imaging. But pathology images are not like your radiology images, right? Their order of magnitude is bigger. So, because of that, you have to think about how to break them down and feed them to the GPUs at a speed and rate that you may not have to do so in other types of imaging modalities, or even medical imaging modalities.

So, that investment in infrastructure, in networking, in being able to connect the GPUs to one another is definitely unique. I think again, the other thing that’s a bit different about pathology is our ability to identify things that may not necessarily be visible directly in the images. For example, we do a lot of work in biomarkers, where we’re trying to identify the presence of a biomarker on top of an image. So, that is not something that’s necessarily visible to the naked eye or something that the pathologist can take a look at. But you can have access to additional types of modalities in the form of sequencing, for example, to be able to validate your findings.

There are a few subtleties associated to pathology. And ultimately, we take algorithms that may exist in other domains, but they need to be customized to enable us to achieve those tasks in a way that’s unique to our field.

[0:13:32] HC: So, I guess once you understand the nuance of digital pathology, and how these large images work, then you can address a standard approach that you might apply to natural imagery and do it slightly differently for pathology.

[0:13:46] RY: Yes. At a high level, yes. A lot of the algorithms used for training those models are more or less version of the same. But ultimately, they get applied to us in a way that’s unique to our ambitions and tasks, and ultimately domain.

[0:14:00] HC: How are you using foundation models at Paige? Are they already integrated into some of your products? Or is that still to come?

[0:14:06] RY: We are actually starting to use them. So, we use them in two ways. The first way is really for us to be able to speed up our existing product development pipeline. Again, you can think about it as in order to build a new clinical system, previously, we would have had to curate data for that particular task, use a lot of compute, reapply a lot of the same algorithms, or innovate. Today, we can basically build new systems by leveraging the foundation model in a faster way.

One of those applications is something that we have announced that we currently are soft launching, is an AI system that’s capable of doing identification of cancer, dozens of cancer types. So, in other words, we have one system that has not necessarily been trained on a particular type of cancer. But if you show it an image of tissue, the system can find cancer point at where the cancer is. That is something that was not possible without the foundation model. So, that is something that we’re super proud of launching now.

We are also using the foundation model as a mechanism for us to expand into new types of cancer types for the identification of biomarkers, or even improving our accuracy at identifying biomarkers for specific mutations on specific cancer types. Internally, the foundation model is already a tool that we use to speed up our development, but also something that is part of our products, as we’re launching this pan-cancer tumor detection system. We’re also working with partners and customers to license the foundation model in a way that enables other companies to build their own applications.

So, the embeddings created from the foundation model is something that ultimately characterizes the slides and the images in a way that is unique. By combining those embeddings in a very sophisticated way, you can discover and have new findings. Ton of companies out there focusing on computational pathology, and really are enabling them to do the work by giving them access to a model that has higher accuracy, and ultimately, a tool that can also enable them to speed up the development process.

[0:16:14] HC: What does the future foundation models for pathology look like?

[0:16:18] RY: That’s a great question. I mean, for us, specifically, we are already working on the next generation of the model that we’ve published on. So, we continue to work with Microsoft Research on that endeavor. Fundamentally, we’re expecting the model to not only be better at all of the tasks that we have already tested the model on. But we’re also looking at identifying emerging properties of the model, and discover things that were never discovered before.

For example, as I mentioned a bit earlier, a lot of the findings that you have in pathology can be directly validated by looking at them with the naked eye. So, if we identify cancer and show pathologists where the cancer was identified, and ultimately, they can agree or disagree with the finding, but that is something that you can’t find in the slides. When you start looking at the field of biomarkers, for example, and you’re looking at an expression of a biomarker or mutation on a slide, this is not necessarily something that you can see with the naked eye. We are looking at basically, expanding the ability of the financial model to really leverage morphology, and identify things on images that wouldn’t have been possible with the naked eye.

We are looking at expanding ultimately the use cases, to the extent where that foundation model truly understands, at a microscopic level, the biology of humans, such that you can build a ton of applications that you wouldn’t have been otherwise able to do just based on traditional computer vision or traditional medical image, medical image analysis. So, that can be in the form of looking at different types of stains, or looking at different types of structures on images, or looking at different types of patterns, and so on, so forth.

[0:17:57] HC: Foundation models seem to be popping up in a lot of different areas. We saw them with language models. Now, they’re in computer vision and within certain subsets of computer vision like pathology. Are there any lessons you’ve learned in developing these foundation models that could be applied to other types of imagery and might be useful for other companies who are thinking of going down this route?

[0:18:17] RY: That’s a great question. I think, maybe, it starts by acknowledging that building a foundation model is not easy. So, I wouldn’t necessarily recommend to every organization to build a foundation model, I think it’s important to realize that with a lot of the technology out there, you can build extremely novel applications without necessarily having to go build your own foundation model. I think building a foundation model needs to be done carefully. If you are really thinking about building one, it’s important to acknowledge that you need access to a lot of data. You need access to probably the largest data sets in your field coming from well-curated data sources, and you need to trust your data, something that you should take a look at, absolutely, before even engaging in that type of work.

You also need to guarantee that you have the necessary computer to be able to build those systems. Building a foundation model is extremely compute-intensive, extremely compute greedy. We see a lot of the funding lately, across the board, going to many Generative AI companies or cutting-edge AI companies. And ultimately, a lot of that funding goes back into buying GPUs or working with cloud providers to be able to get access to those images, and consume those images, and build those systems.

As you’re engaging into the work, the best dataset, make sure that you have access to compute, and make sure that you have access to the best team. Know-how knowledge, not only on the AI side, but also in the domain that you’re specifically looking at building a foundation model within.

[0:19:44] HC: Taking a step more broadly beyond foundation models, is there any advice you could refer to other leaders in machine learning teams?

[0:19:50] RY: Yes, absolutely. I think machine learning and AI is and has been at the center of various discussions lately. Every company today, every software company, every tech company probably has some type of AI initiative. I think AI, ultimately, really enables to increase productivity. They enable you to push the limits of what you’ve been able to do in the past. It is clearly something that’s going to change the world that is already changing the world for the greater good in most of its applications.

As you are thinking about machine learning, I think more broadly, AI, I think you need to think about how does AI affect my team today in terms of the tooling that I’m using? How do I build the right team if I’m starting to release kickoff development initiatives? You need to think about a diverse team. We think a lot about, again, the research, but you also need to think about how do you take that research and put it in the hands of users. And ultimately, that comes through a lot of engineering.

So, you need to build a diverse team that at the forefront of innovation on the AI side, that really remains on top of the literature. But you also need to hire extremely sophisticated profiles in infrastructure and MLOps and ml engineering to make sure that you can take that that innovation and put it into products that ultimately are going to be used in a seamless way.

[0:21:16] HC: Finally, where do you see the impact of Paige in three to five years?

[0:21:19] RY: That’s a great question. For us, ultimately, we really want to see the further adoption of digital pathology, and more importantly, we are really working towards thriving an increase in the in the use of AI in pathology that not only benefits the providers and the pathologists but ultimately benefits the patients. So, we truly believe that we can help pathologists do a better job, enable them to see things that they may have missed, enable them to remove some of the tedium associated to some of the tasks, and ultimately help them provide more info, more valuable information to oncologists, and ultimately help everyone give better treatment options to the patients.

So, we are driving a lot of the adoption of digital pathology by working with scanner providers, for example, to enable the digitization of those images. We are working to really speed up the development of AI solutions, not only on the Paige side, but as I mentioned, providing access to our foundation model to enable other companies to build their own solutions. I think, ultimately, that doesn’t work in isolation. We’ve been focusing a lot, for example, on compliance and regulatory initiatives. We worked with the FDA on getting the first FDA approval for a Paige prostate tech solution, for example, here in the US. And by working closely with all these governing entities, we believe that we can, in a safe and compliant way, start using the technology in a way that it makes a difference. So, that’s also something that we hope to drive over the next few years.

I guess in a nutshell, we’re seeing Paige having an impact. The AI is used on a daily basis to really enable patients to benefit from, ultimately, better care.

[0:23:07] HC: This has been great, Razik. I appreciate your insights today. I think this will be valuable to many listeners. Where can people find out more about you online?

[0:23:15] RY: Thank you so much, Heather. People can definitely reach out to me on LinkedIn. Happy to share my email address, [email protected]. I’d be more than happy to engage with anyone that has an interest in what we do and hopefully help to fill move forward in healthcare and the adoption of AI for healthcare.

[0:23:34] HC: Perfect. Thanks for joining me today.

[0:23:35] RY: Thank you.

[0:23:37] HC: All right, everyone. Thanks for listening. I’m Heather Couture, and I hope you join me again next time for Impact AI.

[OUTRO]

[0:23:47] HC: Thank you for listening to Impact AI. If you enjoyed this episode, please subscribe and share with a friend, and if you’d like to learn more about computer vision applications for people in planetary health, you can sign up for my newsletter at pixelscientia.com/newsletter.

[END]