Open Source Knowledge Platforms: The Key to Accurate AI Bots
By Rahul Kulkarni, CTO, Samagra
An interesting facet of the debate around Artificial Intelligence (AI), is that it continues to be a discussion amongst the most privileged about what AI means for their privileges. Will it take away our jobs? What does it mean for the economy? There is not much of a conversation on what AI means for the masses. Or what does AI mean for those citizens of India, who are not on Twitter currently debating the threat of AI? There is reason to believe that AI can positively impact millions of lives, provided it is leveraged correctly and made
accessible. A pioneering example of this was successfully demonstrated recently in Odisha.
During the Krushi Odisha event held in Bhubaneswar in February, the Government of Odisha showcased Ama KrushAI, an AI-powered chatbot designed specifically for farmers. The annual farmers’ event, which this year focused on emerging technologies in agriculture drew over 20,000 farmers and agri-entrepreneurs from across the state. Ama KrushAI was developed by combining ChatGPT’s intelligence, the Department of Agricultures advisory content on Agri, and Bhashini’s (India’s nationally recognised language interpretation platform) translation engine. The chatbot provides personalised advisory services in Odia, Hindi, and English to the state’s farmers, offering guidance on best agronomic practices, government schemes, and loan products from over 40 commercial and cooperative banks. Here’s an example of Ama KrushiAI at work. One curious farmer, Dipa Sahu in the Mayurbhanj district of Odisha, asked the chatbot about pest management for sucking pests on the mesta plant. The chatbot quickly responded in Odia: “One approach is to sow the Mesta seeds early, which can help prevent the pests from attacking.
Another option is to use a light trap to catch the pests before they can do damage. Seed treatment can also be effective for which, Carbofuran 3G can be used at a rate of 30 gm per kg of seed, and it can help prevent pest attacks for up to 30 days. If you have a standing crop of Mesta and you notice an infestation, insecticides such as Dimethoate 0.05% or Oxydemiton methyl 0.05% can be applied at a rate of 400 ml per acre, mixed with 200 liters of water” The level of detail and accuracy in the response to Dipa’s question, is impressive. The answer tells Dipa not just about various pest management techniques, but also the insecticide dosages that she needs to apply. What made this possible? How did Ama KrushAI manage to furnish a farmer with an accurate and detailed response to a very
specific question?
Deconstructing this technological wizardry, boils down to a single inquiry: What constitutes a high-quality response? Comprehensiveness, structure, and accuracy. ChatGPT has a remarkable performance in the first two metrics, i.e., comprehensiveness and structure. Its extensive training on web knowledge provides a comprehensive response, and its training on large language models (LLM) provides inherent structure to the response. However, in terms of accuracy, ChatGPT falls short. These inaccuracies, often referred to as hallucinations, result from the limited scope of the knowledge database that the bot depends on. In the case of Ama KrushAI, if the chatbot relied solely on web-based information, it wouldn’t have been able to answer Dipa’s specific question.
The answer would instead have been comprehensive but generic: “There is a combination of techniques such as cultural practices, biological control, and chemical control that can be adopted. Cultural practices such as crop rotation, intercropping, and planting resistant varieties can help reduce the infestation. Biological control of sucking pests can be done by introducing natural enemies such as predatory insects and parasites. Insecticides can be used for chemical control sucking pests on Mesta plants. It is important to use the insecticides according to label instructions, and only when the infestation level is beyond the threshold limit”. Therefore, in cases where the answers need to be grounded in both context and domain, the strength of the knowledge database directly impacts the accuracy of the response. To address this, Ama KrushAI leveraged a knowledge database created over the past 5 years using advisory content from the Department of Agriculture and Odisha University of Agricultural Technology.
AI’s accuracy and ultimately its success relies heavily on localised domain knowledge. Which in turn requires that the creation of knowledge systems is democratised while keeping a check on accurate sourcing of information. Traditional knowledge management systems were based on static documents and FAQ pages that could be accessed through search engines, while modern knowledge management systems are expected to be more advanced, with features such as multilingual and conversational interfaces, as well as logical reasoning, conditionality, causality, and correlation built in.
Rather than being solely about AI and mathematics, the game is equally about knowledge bases, which requires collaboration from everyone. A stellar example of this is Bhashini which is India’s AI-led language translation platform aimed at breaking the language barrier by offering digital services in local languages. It enables building multilingual datasets through a crowd-sourcing initiative called Bhasha Daan. Its universal language contribution API (ULCA) has enabled creation of the largest repository of datasets of Indian languages.
To draw a parallel, in order to improve the accuracy of knowledge databases, a universal knowledge contribution API (UKCA) is needed to enable trusted sources to contribute to a federated platform for knowledge sharing. Like Bhashini, which was used to translate text for the Ama KrushAI chatbot, the UKCA would allow contributions on a variety of topics, such as sustainable practices, food processing and biotechnology. To ensure sustainability, it would be crucial to adopt an open-source model that prevents any single company from monopolising the knowledge bases. In this realm, data is king, and improving the accuracy of these databases can significantly contribute to developing use cases that cater to the localised content and knowledge needs of the masses.
