LinkedIn – Connect with Heather.
Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.
Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.
Foundation Model Assessment – Foundation models are popping up everywhere – do you need one for your proprietary image dataset? Get a clear perspective on whether you can benefit from a domain-specific foundation model.

[INTRODUCTION]

[00:00:03] HC: Welcome to Impact AI, brought to you by Pixel Scientia Labs. I’m your host, Heather Couture. On this podcast, I interview innovators and entrepreneurs about building a mission-driven, machine-learning-powered company. If you like what you hear, please subscribe to my newsletter to be notified about new episodes. Plus, follow the latest research in computer vision for people in planetary health. You can sign up at pixelscientia.com/newsletter.

[EPISODE]

[0:00:33] HC: Today, I’m joined by guest Nathan Silberman, Vice President of Machine Learning and Engineering at Artera, to talk about enabling personalized treatment decisions for cancer. Nathan, welcome to the show.

[0:00:45] NS: So glad to be here.

[0:00:46] HC: Nathan, could you share a bit about your background and how that led you to Artera?

[0:00:50] NS: Sure. My role at Artera is leading the machine learning and engineering teams. My career has been a mix of leading engineering efforts, and following my Ph.D. in machine learning, leading AI, and engineering teams at several different startups, applying AI to healthcare. I’ve always been very motivated by the idea of being able to make a big impact in people’s lives using AI. When I first learned about Artera, I didn’t know very much about clinical decisions, and how they were made for cancer therapy. What I learned was that, clinicians often are in situations where they have very few tools on which to reliably make a decision about cancer therapy. Can you imagine that? You’re a patient who has just learned the worst. They’ve learned that they have cancer, and that the optimal way forward is unfortunately ambiguous. I couldn’t believe it. When I learned that Artera was trying to tackle this problem using AI, I felt like I had to join and be part of the solution.

[0:01:47] HC: What does Artera do, and why is it important for cancer care? [0:01:52] NS: Artera offers AI-enabled prognostic and predictive tests that help clinicians and patients make more informed decisions about cancer therapy. The problem clinicians face every day is that, for many patients, there can be multiple treatment options. Which therapy to choose is simply not an easy choice. Clinicians would ideally be able to accurately assess a patient’s risk of a cancer spreading, or adversely affecting the patient’s health in the short term. But often, that’s hard or impossible for a clinician to predict. Even more so, clinicians have been wanting and waiting for tools that can predict whether or not a therapy will work for that particular patient. This is ultimately where Artera steps in. A clinician puts in in order for an Artera test, and they get back a personalized risk assessment for that patient, as well as a prediction of benefit from a given therapy.

[0:02:49] HC: In these tests, how does machine learning come into play?

[0:02:52] NS: Machine learning effectively is the product that Artera offers. Once a clinician orders an Artera test, Artera’s machine learning algorithms interpret two pieces of information. One is the patient’s pathology slides. These are the same ones that a human pathologist used in order to perform the initial diagnosis. This second is a set of clinical variables such as age, and tumor grade. This may differ based on the cancer area that we’re operating in.

Artera’s machine learning algorithms are all multimodal, and that they interpret both these clinical variables as well as slide data, and produce a patient-specific risk assessment, and prediction of benefit from a given therapy.

[0:03:35] HC: Clinical data and pathology slide goes in, outputs – should this patient get this treatment or not, essentially?

[0:03:42] NS: That’s right. As well as, what is the risk of the patient’s, say, localized cancer spreading? Or, what is the risk of cancer specific mortality within five years or 10 years?

[0:03:56] HC: What kinds of challenges do you encounter in working with this pathology data, both the clinical data, and the pathology slides.

[0:04:04] NS: One of the biggest challenges in applying AI to pathology data is the nature in which slides are prepared by different laboratories, or scanned by different scanners. They often differ in surprising ways, and slides even degrade quite a bit over time. At Artera, we have invested a lot of time and effort to develop both training regimes that allow our algorithms to be insensitive to such changes. As well as, a very thorough validation protocol that we use to evaluate the performance of our algorithms before they get launched. In fact, I’m quite excited to share that we just had a paper accepted in the journal, AI in Precision Oncology that details our approach to validating these algorithms in a clinical laboratory.

[0:04:50] HC: Well, the validation part, we’ll get to in a minute because I am curious about that. Dealing with the variations in the slides themselves, how do you handle some of these challenges?

[0:05:00] NS: It’s a great question. We essentially use a combination of different training regimes that try to model those sources of variation. That ensure that our algorithms are ultimately insensitive to those changes when it comes to inference.

[0:05:15] HC: One of the things that comes up, not just in medical applications, but anything related to machine learning is bias. It might be with respect to race, but it can certainly come up in other areas in pathology. How might it come up with the pathology data that you’re working with? Are there some things your team is doing specifically to mitigate bias?

[0:05:34] NS: Great question. Yes, ensuring that our models aren’t biased is a major concern for all of us, and certainly a concern with any type of pathology data. I’m pretty proud of how Artera’s approached this, which is ultimately, to try to anticipate what is the patient’s ensemble distribution in the field. Then, to develop a really robust validation protocol to ensure that our models don’t exhibit the types of biases you want to avoid. Here’s some really concrete examples.

Firstly, Artera’s collected validation data which exhibits patient variation in geography, ethnicity, age, risk categories, treatment regimens, just a huge range of data across each of these categories. Secondly, we test to ensure that our models behave the same across the different digital scanners. You may already be familiar, but the digital scanner that one uses just scan a live slide makes an unfortunate impact in terms of the digital image that is ultimately produced.

Third, we collect data from a large number of clinical labs to ensure that our models aren’t biased towards a particular approach to preparing slides. For those who may not be as familiar, most slides are intended to be prepared the same way. That being said, in the real world, there’s just a lot of natural variation that occurs, and our models need to be prepared for it. Ultimately, the results of all of this is that, our models are able to behave really robustly across a wide range of real-world scenarios, because we prepared for it, by collecting the type of data that we need.

[0:07:09] HC: You mentioned a bunch of different categories that you want to be sure that you have validation data for. How did you go about identifying what these categories are, and identifying what your model might be bias with respect to?

[0:07:23] NS: A huge part of that is working with clinicians. We have a number of oncologists who work with Artera. We also partner with a lot of academic clinicians and large medical centers to be able to understand the areas that we want to operate in very well. Lastly, we keep an eye on the literature in terms of trying to understand the types of common failure cases that are seen across AI algorithms, not just in terms of pathology, but across all of healthcare.

[0:07:52] HC: This process to identify potential biases and make sure that you’re not biased towards any of these categories, that’s certainly part of your validation strategy. Is there more to it? You mentioned a paper related to this. I’m curious to know more about how you validate your models.

[0:08:07] NS: Yes, great question. We use a variety of approaches to make sure our models are safe and effective. Our clinical validation cohorts are typically constructed using a wide range of retrospective clinical trials. For example, our prostate biopsy product, we used five different phase three randomized clinical trials that accrued thousands of patients from across the country that have substantial diversity in terms of ethnic groups. We didn’t stop there, and we’ve continued to add additional validation cohorts that have validated on an additional five phase three trials, and a number of institutional real-world cohorts with different real-world distributions, risk categories, and treatment regimens.

I’m proud to say that Artera’s AI models have been proven incredibly robust across a pretty diverse range of patients and conditions. A great example of the success of this approach is actually a recent study which evaluated the algorithmic fairness of our algorithm, between African-American men and non-African-American men. The results demonstrated comparable performance between the two groups, exactly the results that you want to see.

[0:09:17] HC: How do you ensure that the technology you develop, and these tests are going to fit in with a clinical workflow, and provide the right kind of assistance to doctors, and ultimately, to patients?

[0:09:27] NS: Artera works very closely with clinicians, and their staff to ensure that our test is simple, and intuitive to order, and produces reports which are easy to interpret for both clinicians and their staff. We even have a test report guide posted on our website to help clinicians and patients interpret results. One thing that is important to know is that, traditional non-AI testing methods could easily take a month to get a result. Imagine this, imagine you’re a patient. You’ve just been diagnosed with prostate cancer, and the clinician asks you to sit and wait for another test result about whether you need toxic hormone therapy, or whether it’s safe to avoid it. That is going to be a terrible month. Rather than wait a month, Artera’s test provides the answer within two to three days after the lab receives the biopsy slide, and it is so rewarding to hear from clinicians, and especially patients about the relief we can provide by giving clarity sooner.

[0:10:30] HC: Really speeds the result here, makes a huge difference, and able to ease the mind of the patient, and help them progress towards the next step.

[0:10:40] NS: Absolutely.

[0:10:41] HC: Machine learning is advancing very rapidly now. There are new advancements hitting the headlines more frequently than ever. I’m sure you’ve seen this as we all do with ChatGPT, and all the other stuff over the last year. Are there any new developments in computer vision, or even AI more broadly that you’re particularly excited about? [0:11:01] NS: I think the most exciting new developments have been in a space of self-supervised learning and the development of foundation models. I think Artera has been extremely successful at developing a robust test in prostate cancer. We’re really excited about being able to help patients not just in prostate, but in other areas of cancer as well. Ultimately, the fastest way for us to make that impact is not to develop new models for each and every area that we move into. But instead, ideally, to instill a lot of the knowledge into a single-base model that works across cancers. I think there have been a number of really exciting developments in the literature that have illustrated [inaudible 0:11:40] which at least this might be true. What I can say empirically is that we have a few foundation models that we’ve seen broad reuse internally across areas. We’re not quite there in terms of one model for all areas, but we’re making really good progress and are continuing to invest a lot of research in this area.

[0:11:59] HC: Are these foundation models that you’re developing yourself, or are they some of the publicly available ones?

[0:12:04] NS: I’m going to say both. We have absolutely evaluated some of the publicly evaluated foundation models out there. The ones that we have developed ourselves take into account some of those sources of variation, and abilities to be unbiased in various ways that we haven’t seen from some of the publicly available ones. For us, internally, this is still very much a work in progress for one model that can operate across cancers.

[0:12:30] HC: That’s interesting. Taking into account the variations from the different labs and scanners, the different color variations there. That’s something you need to understand about the data, in order to either account for in your foundation model or to test for. Some of these foundation models are just trained, not necessarily blindly, but without a huge amount of thought about what data goes into it. It may not be optimal for every scenario. Is that what you’re seeing?

[0:12:57] NS: I think that’s right. I mean, firstly, I want to give credit to a number of the different efforts that are out there, that have attempted to produce foundation models that are just generally useful. I think, one of the very tricky things about healthcare is just how subdivided it is in terms of various cohorts. It’s very easy for, let’s say, a paper or an effort to be published, to be able to illustrate great results across a particular, and potentially very large cohort of patients, that just completely breaks down in another. This is, again, maybe a good rationale, a good motivator for how critical it is to have an extremely thorough validation protocol that doesn’t make any assumptions about the cohorts that you ultimately care about.

[0:13:44] HC: Yes. That’s a very good point about validation. Is there any advice you could offer to other leaders of AI-powered startups?

[0:13:51] NS: I think the biggest piece of advice I can give is really just making sure that you’re laser focused on the ultimate goal of patient impact. There are a lot of startups I’ve seen in healthcare that have just brilliant technical solutions, maybe any clever approach to overcoming problems with small datasets, or clever approaches to training on the edge. A lot of these are tackling really challenging AI research problems, and they are fascinating to read about, and definitely fascinating to work on. But even if you solve them, you haven’t necessarily solved an actual clinical problem or patient need.

One of the things that Artera has really gotten right from the beginning of the company is if there’s been an appreciation, and a habit of evaluating how we’re doing in the eyes of clinicians and patients. To that end, we have spent a lot of time understanding patient and provider needs in terms of user experience, workflow as we talked about, understanding the kind of evidence that’s important to generate, to wish ourselves in the community that we have a high impact and very accurate tests. This is something that we repeatedly talk about and share internally, so that everyone in the company has a patient-first mentality.

[0:15:05] HC: Finally, where do you see the impact of Artera in three to five years?

[0:15:09] NS: Yes. Let me answer that question with a story. Recently, we’ve heard from a patient diagnosed with prostate cancer. Normally, they would have been put on a common, but very aggressive therapy with a high risk of side effects. They ordered an Artera test, which helped them and their clinician decide on a less aggressive treatment, which had a great outcome for the patient. While also avoiding potential side effects from the more aggressive therapy choice. This is exactly the type of impact we just cannot get enough.

Artera has been very successful. In just a couple years, we’re now standard of care in prostate cancer. We’re the first and only predictive tests for therapy personalization in national cancer comprehensive cancer guidelines. For prostate, this is super exciting. But the reality is, we are just getting started. We are going to see new products from Artera across a wide range of cancer indications. I expect in three to five years, many, many thousands of patients across multiple cancer indications are going to be able to make more informed decisions about their cancer care that ultimately result in longer and healthier lifespans.

[0:16:20] HC: This has been great, Nathan. I appreciate your insights today. I think this will be valuable to many listeners. Where can people find out more about you online?

[0:16:28] NS: You can go to our artera.ai.

[0:16:30] HC: Perfect. Thanks for joining me today.

[0:16:34] NS: My pleasure.

[0:16:35] HC: All right, everyone. Thanks for listening. I’m Heather Couture, and I hope you join me again next time for Impact AI.

[OUTRO]

[0:16:44] HC: Thank you for listening to Impact AI. If you enjoyed this episode, please subscribe, and share with a friend. If you’d like to learn more about computer vision applications for people in planetary health, you can sign up for my newsletter at pixelscientia.com/newsletter.

[END]