By Atharva Pawde, Programme Lead, Centre for Accelerating India’s Growth, NFPRC Foundation
We instinctively understand why an airline ticket costs money. Behind a two-hour flight lie fuel, crew, airport charges, maintenance and insurance and an enormous physical infrastructure invisible to the passenger.
Artificial intelligence feels different. A question is typed into a small box; An answer appears instantly. Nothing seems to move and no human seems involved. The experience feels less like using industrial infrastructure than like drawing on a more unlimited digital resource.
But AI is not weightless. Between the prompt and the response, specialised processors perform vast numbers of calculations, model parameters move repeatedly through high-bandwidth memory, electricity powers the computation, and cooling systems remove the heat. What looks like an exchange of words is an industrial process.
The user may see only a subscription fee or a charge of a few fractions of a dollar. That modest invoice sits on top of one of the most capital-intensive supply chains in computing
– and the economics of that supply chain, not benchmark scores, will decide which AI companies survive the decade.
The first clue is the unusual way AI is priced: by tokens. Tokens are fragments into which language is divided before a model can process it. No customers think in these units, but they give providers a rough meter for computational work, much as a taxi charges for distance travelled rather than the cost of the vehicle. The user does not buy the model or the GPU, but the processing required to interpret at instruction and answer it.
This explains why a longer prompt is not merely “more text”. It requires the model to read, interpret and retain more information. Reading the prompt and generating the answer are two distinct operations – prefill and decode – and they consume the machine differently. The response then has to be generated sequentially, one token at a time, with each new token influenced by everything that came before it. Output typically costs several times more than input for this reason.
The difference is economically significant. Correcting a sentence is one workload and examining a 200-page document, comparing arguments across chapters and preparing a report is another. Both look like prompts in the same interface, but they impose very different demands on the underlying infrastructure.
What does that meter look like in rupees? At an exchange rate of approximately ₹95 to the dollar, OpenAI’s current standard price for GPT-5.6 Terra—$2 per million input tokens and
$12 per million output tokens—translates to roughly ₹190 and ₹1,140 respectively.
One million requests, each containing 2,000 input tokens and producing 500 output tokens, would therefore generate a model-usage bill of about ₹9.5 lakh. Further down the supply chain, AWS lists an eight-H100 GPU instance in Mumbai at $37.76 an hour: around ₹3,600 an hour, or more than ₹3.1 crore if reserved continuously for a year. These are illustrative public prices rather than the internal costs of every provider, but they reveal the structure of the business: interactions priced in paise must collectively recover infrastructure costing crores.
At the centre of that infrastructure is the GPU and the sticker price tells us surprisingly little. A powerful accelerator is economical if it serves large numbers of users over several years. The real cost emerges when it does not.
A GPU processing a well-packed batch is producing value. A GPU waiting for traffic, constrained by memory bandwidth, or held in reserve for unpredictable demand is expensive machinery sitting idle. The challenge is not acquiring more chips but keeping them occupied without users noticing delays. This shifts the discussion from hardware economics to utilisation economics.
Batching, caching, quantisation, speculative decoding and scheduling determine how much useful work can be extracted from each processor. Even small gains matter. If a system serves more requests on the same hardware, the effective cost of every response falls. The commercial value of the GPU depends not only on how powerful it is, but on how efficiently that power is converted into usable intelligence.
This is also why inference deserves more attention than they receive. Training a frontier model is expensive and dramatic. It requires enormous clusters, vast datasets and months of engineering effort and attracts headlines because they resemble the cost of building an industrial facility. But training happens only periodically. Inference is continuous. A model may be trained once but called billions of times. At scale, the economics of each individual response can determine whether the business is sustainable.
Cheaper access, meanwhile, does not mean cheaper operations. As price falls, users move from short questions to uploading documents, generating video, analysing databases and deploying autonomous agents. A single request may activate several models, search external sources, call software tools and repeatedly evaluate its own output. What appears to the user as one interaction may actually be a chain of hidden computational events and subscription prices cannot always rise as quickly as usage intensity.
Even verbosity has a cost. Additional words consume processing time, energy and infrastructure capacity across millions of daily requests, unnecessary output becomes a measurable operating expense. The better question is therefore not whether a model answers correctly but whether it reaches that answer efficiently – fewer reasoning steps, less retrieval, and the discipline to stop once the task is complete?
Latency compounds this: a response that finishes faster frees the processor for another request and milliseconds saved expand the capacity of an entire computing fleet.
For Indian enterprises, this is not an abstraction. Under the IndiaAI Mission, more than 38,000 GPUs have already been onboarded for the common compute facilityfar exceeding the original target of 10,000 with access offered at subsidised rates starting at about ₹65 per hour. This expansion can lower the entry barrier to experimentation, but it does not remove the need to manage utilisation and recurring inference expenditure. It is the difference between an AI pilot that clears procurement and one that dies in the second budget cycle. CIOs should therefore interrogate cost per resolved task, not merely cost per token, and ask vendors how efficiently their available compute capacity is actually being utilised.
For several years, the industry has celebrated larger models, bigger clusters and higher benchmark scores. That phase is unlikely to disappear, but it may no longer be sufficient. The winners may not be the companies that build the most powerful model at any cost but those that deliver the greatest amount of useful intelligence from every unit of silicon, energy and capital.
Behind every seemingly effortless answer lies a meter that is already running.