← back

The ROI of AI

Created on: 2026-10-27

Introduction

Building AI is hard. If you work for a company and are thinking about adding AI to some of your processes (literally everyone's concern in 2023), then there are two primary questions that you should be asking:

This post is about the second question. If you have questions about the first, ping me! Artificial Intelligence is the only thing that most people can discuss in 2023. OpenAI introduced ChatGPT at the end of 2022, changing everything forever. Rivers of digital ink have been spilled in writing about AI this year, and I don't need to add a lot of noise on top of what has been said. But I will do so for a bit because I can't start talking about AI without mentioning what is happening. It's unbelievable out there. What ChatGPT and OpenAI did was not a technological breakthrough; although it is hard to argue against their technology, it works very well. We have had foundational language models for quite a while now. Their stroke of genius was to enable the masses access to this technology. They transformed an odd, word-generating model into something that anyone could play a bit with. That sparked an interest in creativity, and the rest is history. The funny thing is that one of the intrinsic problems of AI remains: is it worth it? As a company, should I invest money in, say, adding AI to my photo editing process, my inbound email forecast, or a chat interface to my website so that users can converse with my data instead of browsing around? Today, I want to discuss how you can measure if an investment in AI is sound. And for that, I want to use three different examples with their own complexities:

Automating a repetitive process

This is the easiest of the three examples described above and has been the subject of extensive debate in the programming community: we even have jokes/memes about it.

If the task that you are performing is repetitive and remains almost the same every time it is completed, you can measure how often you do it and how long it takes. In the case of typing into a computer the information in a personal card, depending on the card, more or less information is available (name, LinkedIn, phone number, email, etc). That might lead to varying times. In any case, it should be pretty easy to estimate the average time that the task takes. Let's say that for the personal card situation, it takes about 10 seconds. The next question is how often you perform the task: do you process ten business cards per hour or one every week? Once you calculate that number, you can calculate the total amount of time spent on the task:

If the employee or the person performing the task earns 20 dollars per hour, you should not invest more than $8.3 monthly on a business card processing system. The situation is never THAT SIMPLE. What if the processing system is not always accurate? What if I still need to pay for the time, even if it’s not doing anything? What if the delay negatively affects other parts of the business, even if indirectly? We will not answer all those questions here, but if you are interested, let me know, and I'll consider writing about those topics in the future.

Acting on inaccurate information

The first time I learned about the Value of a Stochastic solution in the context of Stochastic Programming, I was fascinated. It’s one of those ideas that is very simple yet beautiful and makes a lot of sense when you see it. I won't go into many details here, but let's say that a simple variation on it can be used to estimate the ROI of our uncertain projects. Imagine that you run a manufacturing business and need to decide how much to produce for the upcoming week. The variable X denotes how much you choose to produce for the upcoming week, c is how much it costs to produce each unit, p is the price you sell your products, and D is the demand for your product. If your goods are perishable, that means that whatever is not sold within a week needs to be discarded. Under such a model, the profit in a given week is given by:

profit

=

−

�

×

�

+

min

{

�

,

�

}

×

�

(\textit{profit} = - X \times c + \min{D, c} \times p)

The table below shows some examples of this for different values:

It’s easy to see that in the setting above, the optimal is to produce X = D. Unfortunately, we don't know D in advance, so that we will need a forecast for it. Depending on the business setting, predicting demand can be very easy or almost impossible. Even without knowing how accurate our demand forecasting models will be, we can measure how their performance will impact the overall profitability of the company. The following table shows how much the forecasted D (Df) differs from X and how that affects the price.

With this information, we can go back to our current demand prediction model and check for how much we have missed our estimates in the past. We can now quantify the Value of having more accurate demand forecasts. It might be that the result of this analysis is that you can't get more profits by implementing a new AI system. It might also be the case that you are leaving millions of dollars on the table, and having access to a forecasting model that is 10% more accurate will be precisely what your company needs. For completeness, let's assume that we have access to past forecasts (we can see how much product was manufactured and use that as a proxy for expected demand) and actual demand (as sales records). Imagine that the table looks as follows:

In total, after five weeks, 17 units of profit were lost for having an inaccurate forecasting system. The total profit for those five weeks amounted to 91 units. That means it could have been 18% higher with a better forecasting system, not minor.

Will a chatbot make my site better?

With the rise of ChatGTP, most companies want a chatbot that can answer information about their data. For some companies, adding a chatbot will be a transformative experience that will revolutionize their businesses. For others, it might barely have an impact on their bottom line. Is there a way to know what to expect before starting? Well, the answer is yes and no. If what a chatbot will enable is a massive, unprecedented scale, then you are out of lack; it will be tough to measure that before getting started. On the other side, if what you think will be the driving factor for your chatbot's success is the "personalization" and "human touch" that it offers, then you might have a cheaper way to get started. My proposal for this last section is simple: fake it until you make it. Hire a human or two, and have them answer as fast as possible, as robot-like as possible, for a week, and see the results. This works as follows:

Wrapping UP

Implementing an AI project is a complex decision that most organizations will face in the upcoming years. Within the vast realms of what AI can do, there are several projects to choose from: some that will have a long-lasting impact on your business and others that will only drain your bank account. I recommend always starting small, going for the lowest-hanging fruit, and getting a taste of what AI can do for you. After that, you will have time to go big and implement more complex projects. When deciding which project should go first, consider the three approaches mentioned here to estimate the ROI of your investment. Happy number crunching, and until the next time!