Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.
Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.
Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.
Foundation Model Assessment – Foundation models are popping up everywhere – do you need one for your proprietary image dataset? Get a clear perspective on whether you can benefit from a domain-specific foundation model.

Transcript:

[INTRODUCTION]

[00:00:03] HC: Welcome to Impact AI, brought to you by Pixel Scientia Labs. I’m your host, Heather Couture. On this podcast, I interview innovators and entrepreneurs about building a mission-driven, machine-learning-powered company. If you like what you hear, please subscribe to my newsletter to be notified about new episodes. Plus, follow the latest research in computer vision for people in planetary health. You can sign up at pixelscientia.com/newsletter.

[INTERVIEW]

[0:00:33.2] HC: Today, I’m joined by guest, Indra den Bakker, CEO and Co-founder of Overstory, to talk about improving infrastructure management by monitoring vegetation. Indra, welcome to the show.

[0:00:45.7] IB: Thanks for having me.

[0:00:46.5] HC: Indra, could you share a bit about your background and how that led you to create Overstory?

[0:00:50.8] IB: Yeah, absolutely. So, I’m Indra den Bakker, CEO and Co-founder of Overstory. I have a background myself in computational intelligence, worked as a data scientist for several years for some large companies but also startup as first employee mainly in advertising technology. So, lots of data, I learned a lot, I think around probably five, or six years ago, I decided I wanted to do more with my skills than selling rats to people. So, I quit my job and [inaudible 0:01:15.7] was just very top of mind.

My mother’s from Indonesia, I travel a lot, saw a lot of beautiful places but also things that weren’t going well with our forests and trees. So, I really wanted to, I guess, defer my professional titles on helping fighting the climate and biodiversity crisis and initially file down machine learning computational title to detect deforestation with satellite images and machine learning. We did pretty well, that’s when me and my cofounders started to discuss what else can we do with satellite images and machine learning to look at our planet, specifically for installed forests and vegetation.

[0:01:49.9] HC: And so, what does Overstory do?

[0:01:51.6] IB: Yeah, so, at Overstory, we apply machine learning through satellite imagery to monitor and analyze vegetation near power lines and this is crucial for utility companies for several reasons, traditionally, utility companies. We started to fix training cycles so they would fit at the same area once every four to six years. We do the inspections by helicopters, drones, and most cases, actually by foot.

Whatever these trees as we all know are dynamic, right? They can have different growth rates, and types of failures such as we’ve seen in the past seasons and our technology really offers utilities of a holistic view of current risk across a large area of land and often, their food territory and this data allows them to optimize their vegetation management practices to operations and this is an important part of the equation because vegetation management is often the largest line item on the operational expense.

A huge course component, very manual and labor intensive and dangerous work. So, with our solution, they can actually optimize where to go. So, they relocate, for example, crews from low-risk areas to high-risk areas, based on the current conditions and risk and therefore, they’re enhancing the efficiency and safety of their operations.

[0:03:06.2] HC: And what role does machine learning play in your technology?

[0:03:08.9] IB: Well, with a machine learning background myself, I would say like the machine learning is really the backbone of our technology. It helps us to analyze vast amounts of data, satellite imagery in our case, detect patterns, and normally, sometimes that the human eye would probably miss if you just look at the satellite image, right?

So, for instance, one of the, I would say, more – most interesting applications is identifying tree species from satellite images. So, we use satellite imagery to distinguish different species of vegetation. For example, telling a pine tree apart from a spruce tree, and the model considers then various factors like spectral reflectors, what does the color look like, what’s the shape, and not only the color and RGB but multispectral, you know, spectral bands, looking at the shape and size of the tree.

I think to really solve this, you need to take a deep learning approach, right? You cannot just hand label the satellite images because you cannot see what the actual species are from the images itself but I also need good ground truth data, you need accurate labels, sometimes you need to loosen your labels and to combine it. So, I think it’s the perfect fit for the classic deep learning approach to get really high-quality output and it’s not only about doing it once, right?

I think, that [inaudible 0:04:26.7] really been applied AI, already specialized AI companies that you really try to solve a problem for our customers. It’s only about accuracy, it’s also about actionability. So, we also use [our burst 0:04:38.9] sometimes in the loop to make sure that actions derived from those insights can actually be used by our customers in the field.

[0:04:47.1] HC: So, your models are used to identify the tree species, perhaps detect individual trees as well or are there other types of models that you use?

[0:04:54.5] IB: Exactly. Declining trees model is what we have as well. So, understanding that the tree has had problems could be best or diseases. So bark beetle is a big problem, additional oak decline or other types of tree failures and that’s important for utility customers because if the tree is sick or declining in health, it has a risk of falling on the limelight. That’s actually a large part of the challenge utilities have.

It’s not the tree that’s standing next to the line, it’s the tree, it’s a little bit further away, gets sick, unhealthy, and then with the next storm falls on the line, causes an outage or in worse condition, it cause houses an inclination for wildfires. So, that declining tree part is also really important. Individual trees and growth. So, we also take images from different angles, stereo modeling to estimate the height of individual trees.

[0:05:44.3] HC: What type of satellite imagery do you use for this?

[0:05:46.0] IB: We mainly rely on very high-resolution satellite imagery. So, commercial data, 30 to 50 centimeters. Multi-spectral, at least four bands, it could be five or eight bands as well to really get to the species and treat it quite well. So many commercial satellite imagery.

[0:06:02.2] HC: And then, working with that satellite imagery, what kind of challenges do you encounter or a particular challenge related to training your models?

[0:06:09.9] IB: I mean, satellite imagery is really complex. So, I think it’s first starting working with satellite imagery, I was probably a little bit naive that we generalize as well to different territories, and different conditions but – and sometimes, good to be a bit naive, otherwise you wouldn’t get started but I mean, a lot of challenges with the satellite imagery, it’s varying resolutions sometimes, lighting conditions, terrain, cloud cover, and then trees look different in different geographies, right?

Like, a pine tree in California looks different from a pine tree in Spain for example. So, there’s a lot of differences to account for. That means that there’s not really like a one-size-fits-all model. I think what we’re developing is more of a very strong baseline model that will do well in different circumstances but then fine-tune our models.

For example, with whether humans are in the loop, or how I like it call it, barbers in loop, to refine, tune to specific regions and to make sure the accuracy is good for our customers and when I say accuracy, it’s not just about precision that we call of certain species. It’s really about how does it translate to a certain action for our customer, right?

Like, if a customer would create a pine tree and a spruce tree the same, then like mixing those species is less of a problem. So, we really try to take into account the end goal when developing these machine learning models, and the satellite imagery are definitely a variation can be taken from different angles as well.

That makes it complex but we like a nice challenge, so that is something that we sign up for, of course.

[0:07:38.3] HC: So, to handle different geographic regions, you’re fine-tuning your models but what about other variations like different weather conditions or lighting? Do you need to train new models in some of those scenarios? Are you able to train a single model for a particular geographic region that handles that diversity?

[0:07:55.0] IB: A mix of both. I think a good example, a recent example is, there are unfortunate wildfires that happened in Canada over the summer that led to a lot of haze across large part of North America. I would think, a lot of people see the images from the wildfire smoke in New York City but it was spread across a lot of different states in North America and that has impact on our image quality as well, right?

You can see the haze across large territories and there are some dehazing algorithms that were working well out of the box that we have to fine tune the algorithms to make it work for those type of short answers in that location. So, we do then sometimes fine tune the model to make sure we can, for example, segment the trees well in third person.

That’s why we always have – this is what we saw coming when I like, we knew there was smoke so we anticipated it a little bit but sometimes, it’s a surprise that the model isn’t performing well in a new region and we have automatic and QA happening across the paradigms to make sure we capture it early and then it needed to fine tune the models.

But of course, the general model or the general ideas that this scales to a lot of or large territories and I’m now talking about mostly about the outliers but those are important as well, of course, it’s important to tackle those as well.

[0:09:09.2] HC: Yeah, so, once you catch the outliers through your modern earned schemes, you can identify whether that type of outlier is something that’s important to tackle or whether it really is a fairly isolated incident and maybe not important for the overall grand scheme of things, is that right?

[0:09:25.4] IB: Exactly, exactly.

[0:09:26.5] HC: How does your team plan and develop a new machine learning product or feature? What are some of the first steps you take in that process?

[0:09:34.0] IB: Well, we work at a high pace, it’s at the startup so it’s highly iterative I would say. So, we start of course with the post statements, and conduct some research like what’s already been done, what’s working, what’s not working. I think that last part is also really important and then just really try to build an MVP as early as possible so that you get a good feeling about how things will turn out if you put more effort into it, right? Then I think then translating that to user feedback is important as I mentioned.

We can create a beautiful, very accurate map of all the species in the world but if that’s not actionable to our customers then there’s no use to me. So, really trying to translate those machine learning outputs into actions is really, really important to us. So, often it doesn’t start with as a machine learning problem. It starts with a problem statement from our customers that they were trying to tackle.

Sometimes machine learning has the answers, sometimes not but it is. I think we always want to address it with a very highly iterative process.

[0:10:29.7] HC: The research you mentioned in that process, how extensive is that for you? I know you’re a fast-paced startup but you know, I’m just curious how much your team might dive into the literature and look it up in source code basis to understand what’s been done before.

[0:10:42.3] IB: But that’s a continuous process because the field is moving so fast, right? So, it’s important to keep an eye on it. Of course, not get only distracted by new resources because there’s such a fast-moving field but we read a lot and we track a lot of the new resources and sometimes it’s applicable. We try it, it works but we will dive deeper and if it doesn’t work, we’ll pause it or park it for a while.

It doesn’t mean we won’t use it in the future, in the future but I think especially in the stage we are in, R&D is really happening on the spot while we are delivering for customers and I think that is a challenging dynamic, right? Because you want to push forward, push the boundaries of our machine learning models can do but they also have customers waiting for results to act upon.

So, I think now that we’re growing, the company will be R&D and deliveries will be split a little bit more clearly but yeah, I think R&D is part of our DNA and what we deliver to our customers and that will always be a large part of, at least from an engineering part, a large part of the work.

[0:11:42.5] HC: It can be a challenging dynamic for machine learning engineers as well to balance, getting something developed to meet a deadline versus taking the time to understand what research has been done and are they actually on the right path. Do you have any guidance in how you guide your team to handle those balances?

[0:12:00.0] IB: I mean, I think, first, I want is acknowledging that it’s difficult, right? And there’s no one solution fits all because sometimes there’s quick wins, sometimes it’s more long-term wins and it really depends on the doubt that you expect but I think fast iteration is important just to see if it’s useful or not and then I think prioritization, probably in every company is important but so is if you have a really how do you focus on product.

I think prioritizing quick and long-term wins over that delivery deadline that’s – I mean, the honest answer, it is a challenge but I think just always keeping an eye on what we’re trying to achieve, that’s a good forcing function I feel.

[0:12:38.2] HC: So, hiring for machine learning can be a challenge to the high demand for professionals in this field. You know, companies in all different spaces there are seeing these right now with, especially with AI being hyped the way it is. What approaches does the recruiting and onboarding have been most successful for your team?

[0:12:54.7] IB: I think hiring is always indeed a challenge, especially for highly specialized engineers with particular experience and I think what helps us is our mission, right? Our mission like being a climate tech company really helps to attract and from different types of industry to move into climate tech and of course, I leave it in a small bubble where everyone wants to join climate tech but I do see increase of people that move from ethic like myself or other industries that want to move in climate because this is just – probably the most important problem to tackle as a society right now, right?

So I think, that’s attracting a lot of people just the mission I think but I’ve heard a lot as well, it’s just the focus we have as a company. We’re not saying we’ll solve everything from day one. We have to take a little niche that we operate in, the electric utility industry and we’re building highly specialized machine learning models but also workflows to address a particular challenge they have.

And I think that’s attracting talent as well to be able to really solve the problem of the climate person, right? And I think that focus, I think that’s something that attracts people as well and then of course, company culture. In the end, it’s about people. So, they got interview process and onboarding process. It’s a lot about the people themselves, the people behind the company.

So, you want to enjoy working for the company, right? And the people around you. So, that’s an important aspect of hiring along. So, we do a lot of outreach as well to people that we think are good because sometimes, the most talented people are not actively looking for a job but they – or they don’t know yet. I think that’s why reaching out to people proactively has also been a positive strategy.

[0:14:29.8] HC: When you do bring on a new machine learning engineer or researcher, I imagine sometimes they haven’t been involved in the forestry applications before or perhaps haven’t even touched satellite imagery. How do you get them up to speed, what kind of training do you have for them?

[0:14:45.8] IB: Yeah, so remote-first company. So, a lot of people onboard remote as well so we have a – I’d say a very brute onboarding process. So, there’s one of documentation happening, there’s a lot of presentation. So documents people will go through videos as well to get up to speed. It’s a lot for people when they just joined but it’s a lot of good information to get up to speed.

But in parallel, we let people start working on particular challenges very early as well. So that’s not only learning new information or adapting to a new service but just getting their hands on a particular project. I think that combination of a lot of good documentation, good clear onboarding with some hands-on work, I think that worked well for us so far.

[0:15:28.7] HC: Thinking about broader about what you’re doing at Overstory, how do you measure the impact of your technology to be sure that you’re on track to accomplish what you set out to?

[0:15:38.3] IB: That’s, of course, a very important question that we also ask ourselves every time and I think the main impact of our technology is we measure it through, I would say, leading indicators because it’s hard to prove that you prevented the wildfire or prevented an outage, right? So, we’re looking for leading indicators like, how many high-risk power lines that we identified that the customer would otherwise have missed or how many hazard trees, or those are trees that are in decline, has fire potential that we flagged that could potentially cause an ignition and a large wildfires.

So, those are the leading indicators that we track. We also measure increased operational efficiency for our customers. Those are the indicators we’re looking for, where is our particular utility customers that we also look at larger metrics like how many acres of lands that we monitor so far and how many workers or utility assets have we monitored and protected over on a regular basis.

[0:16:34.6] HC: Is there any advice you could offer to other leaders of AI-powered startups?

[0:16:39.4] IB: Most important advice that I keep reminding myself as well, is like, stay focused on solving real-world problems. If you really want to achieve and I think that climate emission makes it easy because you know what the end goal is but you want to achieve something at scale.

I think focus is really important, it’s really better to solve one problem really well than trying to solve multiple problems at the same time, right? And I think with satellite images and machine learning there is endless opportunities and that’s like the goal. So the risk of AI as well, like you can tackle a lot of problems or you can tackle one problem really well and solve that problem as well and I think that’s what we are focused on is that more of that vertical or specialized AI that can do one thing very well but really solves a problem there as well.

[0:17:23.7] HC: And finally, where do you see the impact of Overstory and three to five years?

[0:17:27.3] IB: It’s hard to predict the future even with the data we have now but I would say, in three to five years, I see Overstory becoming an essential tool for utility companies globally with a real – this is important again because of our mission and a real measurable impact. So, I was thinking if you can number wildfires, a number of wildfires reduced and power outages prevented become standard in the industry for vegetation management. I think that’s what we’ll see in the next three to five years and we all – I would say, power infrastructure.

The data we have currently is super useful for other decision-makers as well. So, I think we’ll see early adoption of other industries using our data to improve their decision-making.

[0:18:09.3] HC: This has been great. Indra, your team at Overstory is doing some really interesting work. I expect that the insights you shared will be valuable to other AI companies. Where can people find out more about you online?

[0:18:20.0] IB: They can always go to our website, overstory.com or if there’s additional questions, feel free to email me directly at [email protected].

[0:18:28.6] HC: Perfect, thanks for joining me today,

[0:18:30.3] IB: Thanks for having me, I appreciate it.

[0:18:31.8] HC: All right everyone, thanks for listening, I’m Heather Couture and I hope you join me again next time for Impact AI

[END OF INTERVIEW]

[0:18:41.9] HC: Thank you for listening to Impact AI. If you enjoyed this episode, please subscribe and share with a friend, and if you’d like to learn more about computer vision applications for people in planetary health, you can sign up for my newsletter at pixelscientia.com/newsletter.

[END]