AI is becoming more accessible, so why is enterprise adoption still so expensive?

By Sabya Sen, Vice President, IMEA & APAC, Alteryx

As companies move generative AI from experimental phases into everyday operations, a new question is emerging: how much is AI actually costing the business? But as more and more companies introduce the use of generative AI and LLMs in their operations, companies are faced with the new challenge of the unpredictability of AI expenditure.

It is not only about how much AI a company uses, but how effectively it does it. Each prompt, response, and piece of information used by LLMs (Large language models) requires tokens, and this has an impact on costs. Therefore, as AI is implemented across business processes, uncontrolled token consumption can reduce the expected ROI from these investments.

According to research conducted by Alteryx and its partner NextWave, the use of appropriate modelling, business logic along with reusable workflows can reduce token spending by up to 95%, processing time by up to 85%, and improve accuracy by up to 100%. These results indicate that AI needs to go beyond adoption and move towards optimisation.

The hidden cost of AI adoption

Unlike traditional software, where costs are often tied to licensing or end-user counts, generative AI has a more varied cost model. The extent of data processed, the nature of requests, and the employed machine learning model can all have an impact on the bill a company pays.

This cost structure becomes more complex when artificial intelligence is deployed across multiple teams and departments. One inefficient workflow can look minor, but if the messy workflow is duplicated many thousands of times or millions of times, the issue becomes serious. Poorly prepared data is the number one reason for that. Sending too large a volume of unnecessary data to the large language model raises the number of tokens used and does not necessarily improve the answer.

The second thing to think about is the chosen model. One should not forget that not every problem should be solved with the most sophisticated reasoning model. An inconvenient choice of an expensive model for a simple task can become unnecessary expenditures; repeating the same request to the machine increases consumption.

This is how the “token trap” appeared – companies start using artificial intelligence more, but receive less and less effect with every additional request.

The AI bill is bigger than the token bill

However, consuming tokens is only one factor in the calculation. The true expense of deploying enterprise AI involves added expenditures such as infrastructures, licensing, integration, supervision, and human monitoring. AI outputs may incur greater expense if they need to be reviewed, edited, or recreated due to lack of reliability of the underlying data and logic.

This means that the efficiency of AI is more of a business matter than only a technology issue. A cheaper AI response does not necessarily mean a more efficient outcome if the results are unreliable. For companies, the cost of achieving a trustworthy and useful business outcome is a much more relevant measure.

Why business logic still matters

With the arrival of AI assistants and autonomous agents, there’s a belief that AI can take on more of the workload. However, AI is not necessary for all tasks.

For example, straightforward calculations, data transformations or rule-based applications can normally be done better with deterministic processes. AI could be utilised where reasoning and interpretation can really make a difference. This would thus allow organisations not to apply an unnecessarily costly AI model to perform tasks that do not require it.

For an effective enterprise AI strategy, organisations need more than just access to powerful models. They must have a managed layer that allocates data to the model, determines the logic to be used before the model is prompted, and selects the most appropriate model for the task.

From AI adoption to AI economics

Companies will increasingly rely on economics to help them assess how far along they are on the path of AI maturity. Businesses will require to have greater visibility into AI usage, its costs and whether its outcomes can be measured. In order to do that, companies would need to handle the use of AI as carefully as they are used to dealing with other technologies.

Fortunately, there is something companies can do to ensure effective use of AI. For example, large data sets could be filtered before being sent to an LLM. Duplicate data can be eliminated. Large documents can be split into appropriate sections, and inefficient AI requests can be reduced.

Above all, it is very important to match an AI model to the complexity of a problem that needs to be solved using this model.

Thus, the future of AI in the enterprise will be determined not by the amount of AI used, but by its effectiveness. When businesses switch from AI experimentation to production, monitoring token consumption, improving accuracy, and establishing governance will be crucial for the protection of investment in AI.