AI and machine learning are growing in significance as far as adapting to climate change is concerned. During today’s episode, I welcome Max Evans, founder and CTO at ClimateAi, to discuss the topic of climate resilience and how technology is driving progress in this arena.

Max begins our conversation with an overview of the important work he is doing at ClimateAi, before weighing in on the role of machine learning in the AI startup space. He describes in detail the chain of different machine learning models and the challenges associated with high dimensionality and quality in data of this nature. We touch on Max’s preferred methodologies, and he unpacks the role of literature searches, a lack of historical data, and the technological advancements he is able to leverage in his work at ClimateAi today.

“ClimateAi’s mission is to climate-proof our economic system. We want all businesses to make climate-informed decisions.” — Max Evans

“ML is a core mindset of AI startups in terms of how you solve problems, how you start with the exploratory data analysis, the hypothesis building, the baseline and investigation, the modeling, and the many loops.” — Max Evans

“Don't start with the technology or the product, but start with a customer need.” — Max Evans

“It wouldn't have been possible before, and it is definitely possible now to start building forecasts of the climate.” — Max Evans

“Newer methods are developing so rapidly that the edge of what's possible continuously shifts outward.” — Max Evans

“[The AI process] is really about building a data-driven, hypothesis-driven, need-solving culture in both your technological team and in your broader team at large.” — Max Evans

Transcript:

[INTRO]

[00:00:03] HC: Welcome to Impact AI, brought to you by Pixel Scientia Labs. I’m your host, Heather Couture. On this podcast, I interview innovators and entrepreneurs about building a mission-driven machine learning-powered company. If you like what you hear, please subscribe to my newsletter to be notified about new episodes. Plus, follow the latest research in computer vision for people in planetary health. You can sign up at pixelscientia.com/newsletter.

[INTERVIEW]

[00:00:33] HC: Today, I’m joined by guest Max Evans, CTO and Co-Founder of ClimateAi, to talk about climate resilience. Max, welcome to the show.

[00:00:42] ME: Thank you, Heather. I’m really glad to be here.

[00:00:45] HC: Max, could you share a bit about your background and how that led you to create ClimateAi?

[00:00:49] ME: Sure, of course. So I’m originally from Ecuador. I was actually born and raised there, and my family farms pineapples. So with an agricultural background, I came up to the States for undergrad. I studied applied math at Harvard. That’s where my data science journey started. I continued on to Stanford to deepen my ML skills, did an MBA and environmental engineering masters focused particularly in big data in the environmental space.

[00:01:21] HC: So what does ClimateAi do, and why is this important for adapting to climate change?

[00:01:26] ME: Yes. So ClimateAi, its mission is to climate-proof our economic system. I think we want all businesses to make climate-informed decisions. We find it important for the life, lives, and livelihoods. So we want to help protect and preserve all biodiversity, all life. We want to ensure that life like food security, energy security both, and livelihoods, how we live in our economic system. Just ensure that we can live in a good synergy with the environment and make sure that all of our decisions can take both the climate into account, both in how we affect it and how it affects us.

[00:02:13] HC: What role does machine learning play in your technology to accomplish this?

[00:02:18] ME: So I think in a traditional sense, we use ML algorithms in climate forecasting, in building impact functions. But I do think that ML is a core mindset of AI startups in terms of how you solve problems, how you start with the exploratory data analysis, the hypothesis building, the baseline and investigation, the modeling, and the many loops. So I’d like to think that it permeates the entire business in a data-driven ML process approach to a lot of problems.

[00:02:59] HC: What kind of models do you train then, and what kind of data goes into them and maybe some specific examples of the things you’re trying to predict?

[00:03:08] ME: Yes. So climate data is particularly complex. It’s very high-dimensional, and there’s a lot of it. So an example would be the long-term forecasts of the world. Generally, these are your CMIP6 models. You can think of the temperature, precipitation, humidity, wind speeds. All of these climatic variables get forecasted all the way out to 100 years. You want to then post-process the data, store it, increase the resolution. As you’re increasing the resolution, there’s an exponential increase there because it’s quadratic. Those are the variables. Those are the limitations of scale.

Then in terms of the models, you will have forecasting models directly. You have post-processing models in terms of increasing the resolution, increasing the accuracy, correcting extremes. Then you have what we would call ensembling models of how do you optimally mix different models together, both our own statistical models, the dynamical models. So the more physic-based differential equation-based models that the government centers put out.

Finally, which is probably the most important part, is the impact functions or how do you connect all of this climate and weather data to a particular business decision that needs to be made? You could think of it in terms of long-term strategic decisions like siting. Where should I put this farm? Or where should I put this processing plant or even shorter-term decisions? Should I harvest early or late? Or should I plant a long-season hybrid or a shorter-season hybrid?

Transforming the climate data into something that’s useful for decisions requires sometimes crop modeling or different impact functions that allows it to be used in this final way. So it’s a rather long chain of about four different types of ML models that come into making the weather and climate data usable for business.

[00:05:30] HC: It sounds like some of these models will become inputs to other models that you have in your process?

[00:05:37] ME: Exactly, exactly. They are sort of linked together.

[00:05:41] HC: Then working with climate data, you mentioned one of the challenges is it’s high-dimensional and that there’s lots of it. Could you elaborate on this, if there’s other challenges in working with climate data or in how you tackle the challenge of high dimensionality and quantity?

[00:05:58] ME: Yes. I think one of the problems is that most of the tools that are built are mostly for two-dimensional data, things that fit neatly into rows and columns. Generally, most data is broadcast or projected into these two dimensions. But when you have a lot of variables that’s one dimension, you will have space, which is latitude and longitude. That’s an additional two dimensions. You have time because it’s temporal as well which ends up being a third dimension.

Then you have lead times as well, which is when are you trying to forecast. So you have the now and the time that you’re trying to forecast. So even if you have a forecast today, you could have a forecast for next month, for next six months, for next year, for next decade. All in all, you’re looking at about four or five different dimensions and different use cases requiring the data in different forms. You might want a regional analysis that really focuses on the spatial aspect of the data. Or you might want an in-depth historical analysis of how climate is doing in this specific spot. So then you’re looking at a very temporal or time series query of it.

Or when you’re trying to evaluate how things have done at the six-month time to just get an idea of your historical skill, you’re looking at the lead time dimension. So being able to just query a large data that, as I mentioned, just keeps growing exponentially. If you’re doing just spatial or cubic, if you’re trying to also measure a height in space, so like wind speed or pressure, it ends up growing very quickly.

Most customers don’t want 100-kilometer or 200-kilometer grid spatially, which tends to be the default. But you want to then downscale it to 10 kilometers or sometimes even point-specific levels. So there’s a lot of ML work to be done in this space.

[00:08:15] HC: So the high-dimensional nature is on both the input side and on the output, but depending on the use case, you have to figure out how to summarize or downscale those outputs to meet the use case. Is that right?

[00:08:27] ME: Exactly, exactly. Because humans can consume data in high-dimensional formats, you can’t visualize it. You can easily consume it. So you do need to summarize it in some way towards the end and either a two or a maximum of three-dimensional space to make it ingestible, to make it usable for decision making. So there’s always that final challenge of both projecting and adding broader impact functions so that then it isn’t just a raw data feed but actually can tell you something useful for the decision that you’re trying to make.

[00:09:08] HC: How does your team go about planning and developing a new machine learning product or a new feature onto an existing product? What steps do they take in the very early stages?

[00:09:19] ME: So I think we tried to follow, and I think you could say a hybrid between a lean methodology and a Stanford design thinking methodology. So don’t start with the technology or the product, but start with a customer need. What is it that they want to do, and how can we then help them achieve, try to keep the loops tight and quick? So a need might be just knowing and forecasting your crop lifecycle.

So then we start there with the need and what sort of like the specifications be. What’s the resolution? How often do they need this information for like the refresh rate? How much of the information do they need? We try not to just overburden customers with data. Then a next step would be to plan a vision of what could be possible. I think this is the most exciting part where you look at the data sets available, the technologies available, where we are.

We like to focus specifically on this possible-impossible threshold. I think some of the nicer visions or nicer product divisions and features appear right in this threshold of things that have historically not been possible. But we are right now in a space where it might just be possible to do. So that sets us sort of like a vision for this product or this feature. Then we start building it in tight, simple iteration loops. You might start with a baseline, doing a literature review, figuring out what would a simple model be. Bring it to the client. Test it out.

Then go to the next level of simplicity or the next easier low-hanging fruit and continue developing an ML product sort of like leanly with quick customer feedback, understanding the data limitations, model limitations, product limitations, and trying to get that to cycle quickly into what’s needed or what really satisfies the final need of the product. So that’s our general framework for thinking in developing a new ML product.

[00:11:34] HC: You mentioned a literature search as part of that. To what extent do you look at the literature? Is this mostly focused on the application that you’re developing? Or is it more about what’s happening in the machine learning world that might solve this problem? How much time or how much effort do you put into that step?

[00:11:54] ME: Generally, you want to make sure that you’re using state-of-the-art, well, there’s always tradeoffs between just time and how state-of-the-art you can get. But, thankfully, right now, with the tools that exist, the tradeoff isn’t too big. But you do want to do a literature search. For example, we started using LSTMs looking to transition into transformer networks because they just seem better. So you do want to keep using the best models for the specific task that you’re doing. Say if it’s a time series forecast, you want to keep abreast of the latest developments technologically in modeling.

You also want to keep abreast of the latest developments in data to make sure that the papers that are using different data streams haven’t found something that you aren’t looking or aren’t considering new data streams. Also, there’s around tools, just making sure that you’re using the most up-to-date software tools or sometimes process-based tools in data observability, and you keep up to date with all.

It isn’t a single dimension for literature review but a general one. We do spend a lot of time writing our specs and writing our plans and doing our literature review. I’d probably say a good 10 to 20 percent of your time, if it’s spent in planning, is a really, really good investment. It just prevents going into false starts in the modeling, in the data, or in the processes and makes sure that you get buy-in and communicate things effectively, both within your team and to the broader organization. So do spend time planning. It is a very important step.

[00:13:48] HC: I definitely agree with that. So hiring for machine learning can be quite challenging due to the high demand for professionals in the field right now. What approaches to recruiting and onboarding have been most successful for your team?

[00:14:01] ME: Yes. I think, for us, we’ve been particularly lucky in that climate intelligence is a relatively new space. There is a big pool of extremely talented meteorologists, climate scientists PhD level. Some professionals as well, like who’ve already had a little bit of experience in the industry, generally in the government sector, just because the private climate science space is relatively new.

But from the public sector, both governments, PhDs, there’s a lot of talent. We’ve been in a unique position of the private sector, this new industry of climate science, of climate intelligence, just starting. So there’s been some wonderful, wonderful professionals who’ve pivoted their time at science skills into machine learning that we’ve been able to hire. It’s not a traditional hiring just from traditional computer science schools.

But we’ve actually had a lot of luck hiring from more the climate science schools, people who have published in machine learning journals, articles, and worked on the machine learning space for a long time but have come from, I would say, a non-traditional background of a more applied science, either in physics or in climate science. That has been hugely successful to us, as well as being able to grow the talent in-house and supplement the skills that they have with the skills that they need.

Generally, on the onboarding, for the second part of your question of how do we then set them up for success, we find that data scientists are software engineers as well. They need to have a very strong background of working with computer systems and being able to handle data, ingest it, process it, run through the full lifecycle.

So we don’t have any peer data scientists working in Fortran or MATLAB, just building their single models and then handing it off to a separate team that is more software-focused. We actually like to develop all of our machine learning engineers or data scientist into full stack, able to work in Python, able to connect directly with software engineers. I think that’s also been a big part of what we do, trying to develop these full-stack data scientists.

[00:16:52] HC: Yes. That way, the code they write is much more maintainable, I would imagine.

[00:16:57] ME: Yes. I think one of our head of engineering hires that we were interviewing mentioned that you could tell the background or the DNA of a startup by the tools that they use. The more data science-focused tend to have Python everywhere, your software engineers, your data scientists. Sometimes, even into the frontend ended up being very Python-based, whereas more software engineering background startups tended to be a lot more Java-based. It was a quick way of just gouging the origin story of an engineering team or of a technical team.

[00:17:35] HC: That’s interesting. So thinking about the machine learning capabilities you’ve built overall, are there any specific technological advancements that made it possible to do this now? Maybe it wouldn’t have been feasible a few years ago.

[00:17:49] ME: Yes, yes. I think we are leveraging heavily the advances in, basically, image-based processing and convolutional neural networks and NLP mostly around time series. So when you put both of these together, what they allow is spatial time series forecasting or being able to forecast a video, something that has space as well as time. Earth or the climate is this spatial time series. It is a video of what is happening to the world, going forward one frame at a time. We’re trying to forecast what will happen on this next frame.

It’s the same core technology that’s powering your self-driving cars. It’s the same core technology that’s powering your dolly on the image side or your transformer networks on your NLP side, like these newer ChatGPT. So it wouldn’t have been possible before, and it is definitely possible now to start building forecasts of the weather, of the climate.

It is going to be a hybrid approach. We will definitely stand on the shoulders of the physical models that have been built over this past 20, 30 years. But more and more, we see them integrated with these newer methods. These newer methods are developing so rapidly that the edge of what’s possible continuously shifts outward.

[00:19:27] HC: The newer methods are capable of ingesting a lot more data and making use of that data. So it’s not just – there’s hardware advancements, of course, to handle larger amounts of data and larger amounts of compute but larger models to make use of larger training sets and produce more effective outputs. I imagine that that’s part of what you’re getting at too.

[00:19:49] ME: Yes, yes, yes. I think one of the limitations in the climate space was that we only had a certain amount of historical data generally around when satellites were available. So around 1940s, we started keeping track. Right now, we have a good 80 years of high-resolution global data coverage to train ML models. But what we’ve found is that you can make use of synthetic data as well. This allows you to start going into this big data realm of allowing machine learning models to learn from your 1,000 years 100,000 simulations, and opens up the big data space for climate and weather forecasting.

[00:20:42] HC: So thinking, again, overall about what you’ve built and the impact it can have, not just the output from your impact models but the impact of your technology overall, how do you measure that, and how do you make sure that what you’re building has the impact that you want to have on the world?

[00:21:01] ME: Yes. I think so there’s the micro and the macro. On the micro side, we’ll have your traditional tools. Keep track of your RMSE or Root Mean Square Error. Keep track of how well you’re improving as a percentile from the baseline. So each of our sub-models gets evaluated. But as a macro measure, what we are really trying to push is your climate decision skill, which is, basically, are we allowing customers, people to make climate-focused decisions? How confident do they feel about making a decision? Do they feel like they are now able or have the information, the empowerment to make that decision?

It is a much softer metric but a more important one because it actually manages to capture all the interlinking models, all of our software, our usability aspects. It allows you to just keep track of are people using and making climate-smart decisions? Do they feel more confident when they’re doing so?

That’s how we measure the impact because every decision that can be made using more advanced climate information is a decision that can be made better, and that can be more efficient in its use of resources, more efficient in its outcomes. While we do also keep track of those final two, like the value that we’re providing, I do think that the decision, that climate decisions skill is our core measure of our technology.

[00:22:53] HC: Is there any advice you could offer to other leaders of AI-powered startups?

[00:22:58] ME: I think that it follows from what I mentioned before that I do not think that the AI process or the AI mentality is just about using transformers or LSTMs or Convolutional Neural Networks. It’s really about building a data-driven, hypothesis-driven, need-solving culture in both your technological team and in your broader team at large. So I think that there’s a lot of impact that the way of thinking can benefit the entire culture at large. Not to short-sell the potential of this scientific data-driven mentality.

[00:23:45] JY: Finally, where do you see the impact of ClimateAi in three to five years?

[00:23:50] ME: I think that our goal where we’d like to be in three to five years for every business, well, at least every weather-impacted or highly weather-impacted business, to be able to quickly and efficiently make a climate-aware decision. I think we can start building this habit, start building this tool. We can grow as an economy, as a climate-proof economy at the rate of technological progress, and we can start building this muscle. That would be wonderful.

Right now, we’re in agriculture. But as we grow into more and more industries, I would like every climate-impacted business, energy, mining, finance, insurance to be able to make these climate-aware or climate-proof decisions.

[00:24:43] HC: This has been great, Max. Your team at ClimateAi is doing some really important work for climate resilience. I expect that the insights you’ve shared will be valuable to other AI companies. Where can people find out more about you online?

[00:24:56] ME: I mean, feel free to go to climate.ai, our website where you can get an overview of all the people that are there and our core technologies and products. Or feel free to reach out to me at [email protected].

[00:25:09] HC: Perfect. Thanks for joining me today.

[00:25:11] ME: Wonderful. Thank you so much, Heather.

[00:25:13] HC: All right everyone. Thanks for listening. I’m Heather Couture, and I hope you join me again next time for Impact AI.

[OUTRO]

[00:25:24] HC: Thank you for listening to Impact AI. If you enjoyed this episode, please subscribe and share with a friend. If you’d like to learn more about computer vision applications for people and planetary health, you can sign up for my newsletter at pixelscientia.com/newsletter.

[END]

Resources for Computer Vision Teams:

LinkedIn – Connect with Heather.
Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.
Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.