LinkedIn – Connect with Heather.
Computer Vision Insights Newsletter – A biweekly newsletter to help bring the latest machine learning and computer vision research to applications in people and planetary health.
Computer Vision Strategy Session – Not sure how to advance your computer vision project? Get unstuck with a clear set of next steps. Schedule a 1 hour strategy session now to advance your project.

[INTRODUCTION]

[00:00:03] HC: Welcome to Impact AI, brought to you by Pixel Scientia Labs. I’m your host, Heather Couture. On this podcast, I interview innovators and entrepreneurs about building a mission-driven, machine-learning-powered company. This episode is part of a miniseries about foundation models.

Really, I should say, the main specific foundation models. Following the trends of language processing, the main specific foundation models are enabling new possibilities for a variety of applications with different types of data, not just text or images. In this series, I hope to shed light on this paradigm shift, including why it’s important, what the challenges are, how it impacts your business, and where this trend is heading. Enjoy.

[INTERVIEW]

[0:00:48.9] HC: Today, I’m joined by guest, Max Mergenthaler Canseco, CEO and co-founder of Nixtla, to talk about a foundation model for time series. Max, welcome to the show.

[0:00:59.5] MMC: Thank you so much, Heather, I’m very excited to be here and excited to talk to you and to your audience.

[0:00:59.5] HC: Max, could you share a bit about your background and how that led you to create Nixtla?

[0:01:09.0] MMC: Absolutely. I have a pretty, what you could say, eclectic background. I started my formal academic training in Philosophy. I did my undergrad and grad studies in philosophy specializing on topics such as logic, philosophy of science, and philosophy of mind but I was very curious about the formal aspect of philosophy. So, I also studied, doing a lot of logic and with the time, I grew a big interest towards programming and machine learning.

So, I started to learn online and by myself, Python an R, and started developing some initial machine learning algorithms, and then I worked as a researcher at the Max Planck, doing payroll economics and experimental psychology, and after that, I led some research and data themes at the Mexican Government but I always had this drive to try to create something new. So, that’s when I started my journey as a founder.

The first startup that I founded was a natural language processing startup that was monitoring Twitter to try to gain certain insights of the political sphere in our region in Latin America, and after that, I worked at a human forecasting startup where we started thinking about how to create large-scale time series pipelines that we’re not only reliable and very accurate but also very efficient and computationally scalable.

And those, a set of ideas and specific chapters in my life was what led me to cofound Nixtla, to try to continue to solutions for times used for casting in the world.

[0:02:51.5] HC: So, tell me more about Nixtla and how you do that with time series and why it’s important.

[0:02:56.3] MMC: Absolutely. So, time series are in one aspect, the DNA of the world. With the event and the new explosion in LLMs and video models, we sometimes forget that systems, institutions, businesses, they don’t speak really in natural language or text, they speak in data, and most of that data is time series data. Time is an essential component to understand a change of course, but also to understand our reality.

So, time series is maybe a somewhat technical term for a very familiar aspect of our reality, namely, trying to measure and track important metrics across a time, and what Nixtla is trying to do is to build the technology that enables us better forecasting algorithms. So, to put it in other words, from the very beginning of our organization, we have been trying to cope with the uncertainty of the future.

We have been trying to develop methods to forecast and understand what will happen next, and this goes back to the very essence of our species trying to know, for example, how high the rivers will be next year in order to plant accordingly, to the very state-of-the-art questions that we’re asking ourselves in terms of financial forecasting, IOT, anomaly detection, and of course, also trying to predict the weather, trying to predict an understand energy markets. So, in one sense, what we’re trying to do is what OpenAI, DeepSeek, Anthropic, Mistral, all of those companies are doing for text and video, we are trying to do that for time series data.

[0:04:46.3] HC: Am I understanding is the least part of the solution is the foundation model? Why is that your approach for tackling time series?

[0:04:53.6] MMC: Yeah, that’s a very interesting question. So, classically speaking, time series was approached and had a highlight approximately during the 60s with econometric-based models. Very famous models that most of you or your audience probably studied that at school, like ARIMA and other autoregressive models were developed in that time, and were used mainly in macroeconomic settings, and some of them are still used.

For example, central banks, to predict inflation, ETC. However, classical time series approaches have a certain problems and limitations with them. One of the most famous problems of classical approaches is that they are local models. They only understand and take into account the autoregressive values of one particular series. Of course, that problem can be overcome with newer approaches such as creating boosting trees or other global models that try to understand the whole context of the series.

And even some multivariant models, that try to understand the relationship between the different series. However, the main aspect of all of the above and the techniques, statistical machine learning narrow forecasting methodologies, imply that you have to train your own models. That means that you have to be able to, first of all, develop enough expertise in the field to do an accurate construction of the architecture and the models that you want to use.

And second of all, that you are able to train and deploy those models, and that approach worked well for a couple of years the same way that natural language processing worked well in the classical sense, for a couple of years but now, we are entering a new era where a massive amount of data can be used to train very big transformer or neural net architectures, and the advantage of those foundation models.

And that’s why we’re pushing the agenda of foundation models in time series is that they are the same as LLMs, train on a very vast amount of data, so therefore, they have a lot of knowledge about different structural patterns in time series, such as seasonality’s trends. The other big advantage is that they are a lot faster to use than training your own models, enabling a whole new set of applications.

And the third very important aspect is same with LLMs is that they are a lot easier to use, given that you don’t have to deploy your own pipeline. You simply call a foundation model the same way as you call an API and get predictions out of the box with a few lines of code. So, to put it more concisely, instead of spending hundreds of thousands of dollars, months, and a lot of time building your state-of-the-art pipeline, now, companies can deploy very accurate forecasting and anomaly detection pipelines in hours, instead of months.

[0:07:41.7] HC: What types of time series data go into training this model?

[0:07:45.7] MMC: TimeGPT was trained on what at the moment was the largest collection of open data. We have data from very different sectors such as healthcare, telemetry, IOT, manufacturing, finance, e-commerce, work data, ETC. We are now also seeing that other research labs are expanding on possible available datasets by using synthetic data but also some very interesting research lab have proposed the idea of introducing even time series data that was not originally time series.

So, for example, one lab used images and translated those images into time series data, and now they train the foundation model that was trained entirely on time series data that was a transformation from images, and they proved that that foundation model was able to accurately predict other domains such as e-commerce or retail or electricity. That’s the answer to the question.

[0:08:48.2] HC: How do you go about training this foundation model, in particular, is it similar or different to how you might train an LLM?

[0:08:57.0] MMC: If they’re, of course, vast similarities, and there are, of course, also some specific differences. So, in the case of TimeGPT, we used transformer-based architecture, however, some of the details of how we are encoding or decoding, how we are configuring the loss functions, how the attention mechanism is specifically working is different than LLMs. So, in some of the more technical details, we’re still not revealing.

But the main intuition is that both problems are trying to forecast in a, let’s say, sec to sec, weight, volt, models are trying to forecast the next, if you want, token. In the case of LLMs token is a subset of strings or letters. In our case, the next token is a set of numbers, and that seems like a maybe small difference but it is a huge difference in the sense that of course, language has a very specific structure in itself and time series, normally have another structure.

What’s very interesting is and that’s one of the main contributions that we show to the field is we show it when we published TimeGPT one that time series data does have indeed a structure, and someone could assume that there is no shared pattern between, let’s say, retail data and IoT data or electricity data and healthcare data. However, they’re – by proving that our general model does transfer across domains, even domains that were not seen by the model before, proving that we show that there is something like an underlying structure of time.

And after we published TimeGPT one, that intuition was confirmed by many other research labs that publish other implementations or foundation models for time series, such as the Amazon team with a very interesting model called Chronos. By the way, one of the things that they mentioned in the subtitle is something like the grammar of time or the structure of time, hinting precisely with this intuition that timeshares in itself also have some sort of, if you want, grammar of their students, something that can be learned and has a structure in itself.

But other research lab has, such a Salesforce, Google, ETC, of course, CMU, have also validated this intuition by showing that you can train a foundation model on time series data, and then forecast on unseen data, either in a zero-shot fashion or by finetuning your model.

[0:11:21.7] HC: What are some of the challenges you’ve encountered in building a foundation model?

[0:11:26.6] MMC: So, at the beginning, one of the biggest challenges was getting the data, because, in our field, there wasn’t such a thing as Imaginate, which was particularly relevant to develop, let’s say, computer vision models, and we also didn’t have such clear think as Reddit or Google Books or all the collection of text that was available for training LLMs. So, trying to find, curate, and scrape that first data set was a challenge also because in our particular case, a team was rather focused on the algorithmic part of things.

So, we didn’t have a lot of expertise in scraping, organizing, curating, storing, building huge datasets. So, that was an interesting challenge at the beginning, we overcome that challenge, and I think now the biggest challenges, the field, and ourselves are trying to explore how are the scaling loss of training and of course, and inference, are going to behave in our field. So, one very interesting difference between time series data and text data is that it is commonly known now.

We might be approaching a certain limit in the amount of data that the new amount of data that we can use to train new LLMs given that we have exhausted most of the, like classical sources of textual formation, we have probably scraped 90% of all the books that have been written by human history. We have scraped a very high percentage of the entire Internet. We have scraped a lot of what is already there.

There is, of course, new things that we can transcribe like YouTube videos, interviews, I don’t know, TV programs but the idea, and this is known between LLM researchers is that in the case of LLMs, we are nearing a certain limit into at least, the amount of data that can be used to improve these models. With time series data, it’s different, given that there is a lot more time series data than there is text, given the time series data is machine-produced.

So, every single day, millions of sensors across the world are producing billions and billions of time series data points. So, I think there’s an interesting question and an interesting challenge and understanding how precisely do the scaling loss of time series data behave, how much better are these models going to be. So, that’s an interesting challenge, an interesting question that I think the field is thinking about and you can see this in the sense that people are starting to a new and new and new variety of data sources.

That’s challenge one, understanding scaling loss. Challenge two is I think also trying to understand a very basic intuition about forecasting, namely how does the rest of the world or exogenous covariants or exogenous variables influence the forecast. So, to put it more group-friendly, one question that we constantly ask ourselves is what is the optimal way to encoding additional information besides the mere outer aggressive values.

Or, to put it in a more completely or practical, how does, for example, price affect demand, how does weather affect attendance to a football game, and so on. So, that’s another interesting challenge, those were all things that we are working on and things that I’m sure all the other research labs are also exploring.

[0:14:44.9] HC: It sounds like a fairly fundamental question but how do you know whether your foundation model is good?

[0:14:50.9] MMC: Yeah, that’s a very important question. I mean, the easy answer is of course that you can always create a benchmark dataset and you can very objectively evaluate if that foundation model is good in the sense that you have very standard error metrics that are used in the field. You have min absolute error, you have standard minutes, and maybe you have classical error metrics, some of them have certain problems and are not recommended in production.

But you can easily construct the dataset and then test the different foundation models and with those benchmarks, you can see which models are best. There is a huge props to the team at Salesforce that created an open benchmark platform that is hosted in hogging phase, which is open source. So, now different researchers can submit their open-source models and see how they rank.

So, that’s the first dimension of evaluating our foundation model, accuracy, how much better is this than the other foundation model or my old build. The second element of course is how much faster and more efficient is the model. Sometimes, same as with LLMs, being accurate is not enough. You want an accurate but also fast answer and sometimes in specific cases, you even might compromise certain accuracy points if the model is significantly faster and your applications are near real-time.

So, that’s the easy answer. The hard answer is that given that we are all training on massive amounts of data and some of us are not disclosing which datasets we’re using, it’s always a problem for academics to try to benchmark foundation models because there might be always leakage. So, let’s say you want now to forecast and compare TimeGPT against other foundation models and you want to use one public data source because obviously, you don’t have private data sources.

And if you do, you can’t use them for forecast, for benchmarking necessarily. So, you would encounter the problem of, “Okay, how sure am I that the other teams didn’t use this data set to train their model?” And that’s a big problem in foundation models. In the industry, that’s not really a problem because when we go out and sell to companies, they are of course sure that we didn’t use their data for training because their data is strictly private and confidential.

That’s an interesting aspect of foundation models in time series, that benchmarking is not as straightforward as one might think.

[0:17:14.5] HC: Do you ever find a model that performs well across the benchmarks is not the top performer for a new task or are those benchmarks generally a good place to look on which model is going to be the best fit for a new task?

[0:17:28.1] MMC: Historically speaking, I think the field, the time sphere field has suffered from precisely what you are hinting at. The benchmarks are not necessarily representative of the real world and a couple of – last year still, like the biggest benchmarks, the classical benchmarks that we use in the industry, which are either the M competitions, with the latest was the M5 or M4 competitions.

Some new datasets like electricity and energy they are actually very small. So, even the benchmark, the foundation benchmark paper that I was referring to that is implemented now in cognitive phase, I think it has something like 150,000 time series, which sounds like a lot but to give you a comparison, I think a match net had 10 or two or three orders of magnitude more data. So, I think right now in our field probably benchmarks are not necessarily indicative of how well a model is going to perform in real-world data.

At least, not for all cases, which of course poses a challenge, and I think there’s a huge opportunity in our field to create a very robust and very big benchmark that would try to help us understand which models are performing good in which cases and which tasks.

[0:18:46.6] HC: How are you currently using your foundation model and how do you plan to use it in the future?

[0:18:51.3] MMC: TimeGPT was created as a commercial product. Nixtla has been around for more than three years and we became known in the open source, in the academic, and in the industry for developing what are now by mainly considered the industry standards in open source forecasting. So, we have a set of libraries completely open source, MIT licensed that have been downloaded more than 22 million times and more than 12,000 GitHub stars are used by thousands of companies across the world and all of that was free.

And we developed that by raising some VC money and at some point, we started asking ourselves, “Okay, what could be the next thing that we could do to make Nixtla a commercially viable enterprise?” And that’s when also when we started researching and thinking about TimeGPT. So, right now, our foundation model is offered as a commercial product. You can access it through an IPI because you need to create an account at dashboard.nixtla.io, and then you get a token, and with that token, you can start making API calls and get out-of-the-box predictions.

We’re also part of the Azure model catalog, so the same place where you could find the enterprise versions of Open AI, DeepSeek, and Cohere. You can also find our models and use them, and we also offer enterprise solutions for big enterprises where we have themselves host the model for compliance and security reasons. That’s currently the way that you can use TimeGPT. Of course, this is a very important question right now is, “Okay, why is it still closed source?”

DeepSeek I think has made a very strong argument that the future of Gen AI might lie in the open source. We are also of course thinking of open-sourcing this, our ethos has always been open-source end and we decided not to do it at the moment because we were competing with the biggest companies in the world such as Asana and Google, Salesforce, IBM, and we were a little startup. So, we wanted to keep them out given that we were the first to publish but in the future, you probably can expect that you can also use TimeGPT by yourself in an open-source manner.

[0:20:59.8] HC: So, a combination of commercialization and open source, it sounds like it’s probably in your future then.

[0:21:05.9] MMC: Exactly.

[0:21:06.8] HC: What does the future of foundation models for time series look like?

[0:21:10.1] MMC: That’s a very interesting question. I think I was hinting at some of the points before, namely, we’re saying we’re going to – I think we’re going to see a lot of the things that we are seeing in LLMs. One thing that’s coming soon is trying to identify and push the limits of the scaling loss of time series, so bigger models with more data, I think that’s something that’s coming in the future.

The other is going to be better and more accurate support for exogenous variables. I think another big aspect of time series foundation models is multimodality, trying to understand how these models could interact with other models such as text models. I think that’s going to be huge, trying to also interact with the time series model with either text by providing context or text by trying to understand the output.

So, I think that’s coming and then I think another interesting, more fundamental aspect that might arise in the next couple of months is, “Okay, how does the intuition of, for example, a mixture of experts’ chain of thought reinforcement learning work in our field? Is there something similar like a set of different models that can specialize in a specific time series tasks and then used?"

So, I think that we’re also going to see some of those intuitions that come from the LLM field translated into the time series field soon.

[0:22:33.1] HC: Is there any advice you could offer to other leaders of AI-powered startups?

[0:22:37.5] MMC: No, not at the moment. Right now, given the whole revolution that you want that happen with DeepSeek, I am not sure what concrete advice I would give to other AI startups. What I normally answer to when I get out of this question in other forums or other podcasts is probably the only thing that I would advise is given that everything is so uncertain and everything is so complicated, probably if you try to optimize for doing something that you really like and that you really believe in.

Then if you’re somewhat more philosophical make the journey itself a destination, then I think no matter what the end outcome is then you’re going to have a certain upside. So, I think that would be my advice, try to do something that you like, try to contribute positively to the world, and then hope for the best because everything else is completely uncertain.

[0:23:29.7] HC: Well, to throw another somewhat futuristic question at you, where do you see the impact of Nixtla in three to five years?

[0:23:36.3] MMC: For us, the vision is very clear. We want to become the go-to solution for time series and our dream is that we get embedded in all possible systems and make the world better in all different aspects. So, to be very specific, right now we are seeing a very interesting adoption of LLMs that is being used from the smallest companies and often smallest companies in the world to the biggest enterprises.

From the new interns to the more most veteran developers and CTOs and CEOs using this technology and we think that that will happen with foundation models in time series too. We do think that from the smallest hotdog stand or coffee shop wherever in the world, they would be interested in trying to understand better how many products they are going to be selling next month, how the weather might impact their sales, and try to optimize for that in order to throw less food away or to some, more sophisticated, the inventory optimization.

So, we do see times used being completely horizontal from those very small organizations to the largest banks and companies in the world adopting foundation models, and our vision, our dream is to be the company that enables that massive adoption for time series foundation models because we are right with the set of hypothesis that I have outlined. Then by making that a reality, we would be helping to make the world a lot more efficient. And that would have direct positive impacts into consumers, into enterprises, into business, into institutions, and that’s if you want the dreamy aspect of Nixtla days. Everyone, that’s the right American expression for the pipe dream of creating Nixtla and the big future that we’re envisioning.

[0:25:22.8] HC: This has been great. Max, I appreciate your insights today. I think this will be valuable to many listeners. Where can people find out more about you online?

[0:25:31.2] MMC: You can find more about us in LinkedIn, you can search for Nixtla at LinkedIn, and you can follow me. We also have a Twitter account, Nixtla Inc. @Nixtlainc, or I’m afraid it is no longer Twitter, an X account. You can also of course follow our website, Nixtla.io, and I think those would be the main sources of information. On the other side, we’re always happy to have more open-source users and open-source contributors.

So, follow us on GitHub, github.com/nixtla, you will find our different libraries. You can join our Slack channel, we’re always happy to support your used case, and very important for the research side of your audience or students, we offer free credits and extended free trials for people using TimeGPT for research and/or academic purposes. So, that’s how people can keep communicating with us, Heather.

[0:26:22.2] HC: Perfect, and I will link to all of those resources in the show notes. Thanks for joining me today.

[0:26:27.0] MMC: Thanks so much for the invitation, thanks so much for the insightful questions, and I hope this is interesting for you and your audience.

[0:26:33.9] HC: All right everyone, thanks for listening. I’m Heather Couture and I hope you join me again next time for Impact AI.

[END OF INTERVIEW]

[0:26:44.4] HC: Thank you for listening to Impact AI. If you enjoyed this episode, please subscribe and share with a friend, and if you’d like to learn more about computer vision applications for people in planetary health, you can sign up for my newsletter at pixelscientia.com/newsletter.

[END]