Why AI that works in the lab fails on the factory floor: A conversation with Prof. Roop Mahajan, Virginia Tech
Every CIO has seen the same pattern: an AI model posts impressive accuracy in the pilot, wins internal buy-in, gets funded for scale-up — and then quietly underperforms once it hits real production data. The instinct is to blame the vendor, the data team, or the model itself. Prof. Roop Mahajan, Director, Institute for Critical Technology and Applied Science, Virginia Tech says the real problem is more fundamental: most industrial AI is built to win in the lab, not to survive on the floor.
Mahajan has spent three decades on both sides of that gap — running AI-driven process control at Bell Labs, where a fractional drift in temperature or contamination can wipe out an entire production run, and later building his own neural network software, CU-ANN, because the commercial tools of the time couldn’t hold up under real manufacturing conditions. His diagnosis is blunt: academic and vendor benchmarks reward peak accuracy on clean, curated datasets. Factory floors don’t offer that luxury. Sensors drift. Operators vary. Raw materials fluctuate. Equipment ages in ways no training set anticipated. A model optimized for a demo is, by construction, unprepared for the environment it’s about to be deployed into.
This matters more for CIOs than it might first appear, because the fix isn’t a better model — it’s a different deployment philosophy. Mahajan’s approach embeds physical and process constraints directly into the AI system, rather than treating it as a black box statistical layer sitting on top of the plant. The result is software that may sacrifice a few points of benchmark accuracy in exchange for something far more valuable in production: predictable behavior, interpretable failures, and reliability that holds up under exactly the kind of imperfect, shifting conditions that kill most industrial AI deployments within a year of go-live.
Some edited excerpts from a highly intellectual conversation:
Every policy conversation about AI in manufacturing in India starts with infrastructure — fabs, corridors, PLI. You have spent thirty years working on the intelligence layer that sits on top of that infrastructure. Is India sequencing this correctly, or are we building the highway before we know how to drive?
Infrastructure is necessary, but not sufficient. A highway only adds value if you know what vehicles will run on it, who will drive them, and how traffic flows. While fabs and PLI schemes are vital foundations, India’s ultimate manufacturing competitiveness will be decided by the intelligence layer: sensing, process understanding, control, materials knowledge, and people who can connect physics with data.
My main concern is that AI is sometimes treated as a software layer that can be added after the fact—may be like an app you download after paying for the phone. In industrial manufacturing, it is difficult and uneconomical to paste intelligence on top of a legacy process. We must embed sensing, process chemistry, materials science, and process science from day one. We must build infrastructure, yes, but build the physical highway and the driving capability simultaneously.
India has an enormous installed base of legacy manufacturing — textiles, auto components, chemicals, pharma — that the West has largely automated away or offshored. Does that base represent a liability in an AI-driven world, or a training ground that countries building greenfield factories simply do not have?
I see India’s legacy manufacturing base as a tremendous training ground, not a liability. AI models require messy, real-world problems. The variability, maintenance anomalies, and efficiency gaps found in India’s chemical, pharmaceutical, and automotive clusters provide exactly the kind of data that industrial AI needs to learn from. Greenfield factories may look pristine, but they often lack the decades of operational data, maintenance history, and process variability that make industrial AI truly valuable.
India should not apologize for its legacy base.
It possesses something many emerging manufacturing ecosystems lack: decades of accumulated process knowledge. The opportunity is to instrument and digitize these factories, codify that knowledge through AI, and embed those lessons into the next generation of manufacturing systems. Properly leveraged, India’s legacy industries can become a powerful competitive advantage in the age of AI.
India is investing heavily to build fabs. In your experience running AI-driven process control at Bell Labs and CU Boulder, what does it actually take to run a fab well — and how far is India’s current policy from addressing that?
Building a fab is an infrastructure challenge; running a fab is a talent, discipline, and ecosystem challenge. At Bell Labs, I learned that semiconductor manufacturing is unforgiving—a small drift in temperature, chemistry, or particle contamination can destroy yield. Fabs are highly sensitive systems that depend on highly experienced process engineers, maintenance specialists, metrology experts, suppliers, and real-time data.
This is exactly where AI-driven process control becomes a competitive advantage. It can continuously monitor process variation, detect subtle deviations, and enable predictive intervention before yield is compromised. India’s current policy has done an excellent job laying the groundwork through investments in capital and physical infrastructure. The next challenge is developing the specialized workforce, supplier ecosystem, and manufacturing culture required to operate these facilities at world-class levels.
You built and copyrighted your own ANN software, CU-ANN, because the available tools were not good enough for production conditions. What does that tell us about the gap between AI that works in a paper and AI that works on a factory floor?
Developing CU-ANN taught me that there is a profound difference between AI that performs well in a research paper and AI that survives on a factory floor. Manufacturing data are noisy, incomplete, and constantly changing as sensors age, equipment drifts, operators vary, and raw-material properties fluctuate. A model that achieves impressive accuracy on a curated dataset can fail under real production conditions. This reality led us to develop CU-ANN.
By grounding the software in process physics and equipment behavior, we embedded physical constraints directly into the neural network. This hybrid approach reduced overfitting and prevented unphysical predictions. Academic research often rewards peak validation accuracy, but industrial deployment demands robustness, interpretability, long-term stability, and predictable failure modes. On the factory floor, the most valuable AI system is not the one that is occasionally brilliant; it is the one that remains reliable every day under imperfect and changing conditions.
Your team is running a research programme that goes from Indian coal to graphene to thermal management in AI chips. Walk us through that chain — why does it matter for India’s position in AI hardware?
The chain is simple but powerful: India has coal; coal contains carbon with inherent structural disorder, which we chemically convert it into graphene oxide, reduced graphene oxide, graphene-like materials, quantum dots, composites, and related carbon nanostructures to address one of the biggest constraints in AI hardware: heat. As computing power increases, thermal management increasingly limits chip performance, reliability, and energy efficiency. Coal-derived graphene materials can enable advanced heat spreaders, thermal interface materials, packaging solutions, and other cooling technologies.
Furthermore, this unique materials opportunity extends far beyond silicon to data center infrastructure. AI facilities consume enormous amounts of energy just for cooling. By reinforcing recycled plastics with coal-derived graphene oxide nanofillers, we can develop advanced insulating materials that drastically slash parasitic thermal loads and boost energy efficiency. If India wants a meaningful position in AI hardware, it must look beyond semiconductors to the broader materials ecosystem. Coal-to-graphene is not about extending the life of a legacy resource; it is about transforming an abundant domestic feedstock into a strategic materials platform for the AI era.
Your students are now at Microsoft, Intel, Qualcomm, and IITs. When you look at what they are working on today versus the problems you gave them as doctoral scholars, where do you see the clearest continuity — and where do you see a break?
The clearest continuity lies in first-principles thinking. My doctoral scholars were expected to cross-validate governing equations through experimentation, multi-physics simulations, and real manufacturing constraints. Whether they are designing architectures at Microsoft, Intel, Qualcomm, or serving on the faculties of the IITs, the core lesson remains unchanged: understand the underlying physical phenomenon rather than become enamored with the computational framework. In the end, it is physics, physics, physics.
The clearest break is the extraordinary growth in computational power, scalability, and algorithmic sophistication. Today’s researchers have access to advanced simulation platforms and AI tools that remove many of the computational barriers we faced. However, this progress carries a risk: producing visually compelling or statistically impressive results that are disconnected from physical reality. AI has transformed our tools, but it has not eliminated the need for mechanistic understanding, critical thinking, and experimental verification.
You have been inside both the research system and the industrial system — Bell Labs, Boeing, IBM, Texas Instruments as sponsors, now Virginia Tech India and Thapar. What is the conversation India’s manufacturing and technology policymakers are not having that they urgently need to?
India’s critical policy gap is not technological; it is structural. We continue to treat advanced manufacturing, artificial intelligence, energy infrastructure, materials science, workforce development, and supply chains as isolated verticals. In reality, industrial competitiveness emerges from how these pieces work together.
A semiconductor fab is not merely a building; it is an ecosystem of materials suppliers, process engineers, maintenance expertise, real-time data systems, reliable power, water management, and logistics. Likewise, we cannot discuss data centers without addressing thermal management, grid stability, water consumption, and environmental impact. Policymakers must shift from technology acquisition to ecosystem building.
India has no shortage of talent. What it needs is a systems-level strategy that connects science, engineering, manufacturing, and policy, while creating space for high-risk, high-reward “Black Swan” innovations that can redefine entire industries.
If you had to identify the single engineering insight from your 1990s work that is still not adequately understood or applied in how people deploy AI in manufacturing today, what would it be?
The single insight is that domain knowledge and operational data must be integrated. Even in the 1990s, when we were using neural networks, we rejected blind curve fitting in favor of physics-based, data-assisted artificial neural networks. Manufacturing processes are governed by physics, chemistry, thermodynamics, and equipment history. AI must operate within those constraints. Many practitioners still treat AI as a purely statistical exercise, assuming that more data automatically produces more intelligence. It does not.
Bad data can produce confident but incorrect conclusions. Sensors drift. Correlations disappear as processes evolve. That is why the future lies in physics-informed machine learning, where domain knowledge guides model development and validates predictions. AI should complement physical understanding, not replace it. The most reliable systems emerge when data science and engineering judgment work together.
If the Prime Minister’s Office called you tomorrow and said they had one serious intervention available — one policy, one institutional investment, one structural change — to close the gap between India and the leading AI-in-manufacturing nations, what would you tell them to do?
I would recommend launching a national network of AI-enabled manufacturing testbeds embedded within real factories, not confined to academic laboratories. Each testbed should integrate real-time sensing, process control, materials characterization, digital twins, energy analytics, and workforce training.
The Government of India should anchor these testbeds in sectors where the country already possesses scale and competitive advantage: textiles, pharmaceuticals, chemicals, auto components, electronics packaging, and advanced materials. The initiative should bring together industry, IITs, state and private universities, and international partners to solve real production problems with measurable economic and societal impact. Success should be measured through operational metrics—yield improvement, energy efficiency, defect reduction, equipment uptime, and new product development—not by academic publications alone.
India does not need more isolated pilot projects. It needs a disciplined national mechanism that systematically converts scientific knowledge into manufacturing capability and establishes global leadership in a select set of strategically important sectors.