The Nepali Language, Nepali Data, and the Dream of Homegrown AI: The Story of ‘Himalaya AI Labs’

 The Nepali Language, Nepali Data, and the Dream of Homegrown AI: The Story of ‘Himalaya AI Labs’

Are the artificial intelligence (AI) systems we use in our daily lives truly our own? Most of the time, we rely on AI developed by companies such as Google, OpenAI, NVIDIA, and DeepMind. However, these systems often fail to provide accurate responses in the Nepali language—simply because they were not trained with a focus on Nepali.

While the rest of the world is advancing rapidly in the field of AI, Nepal currently functions merely as a consumer rather than a creator. Seeking to challenge this stark reality and make Nepal self-reliant in AI, a team of young Nepalis has launched an initiative called ‘Himalaya AI Labs.’ Working remotely from both within Nepal and abroad, this group of over a dozen young professionals in the IT sector is working tirelessly to build world-class AI infrastructure for the country.

It has not been very long since these young individuals first connected and began their collaborative work. Many still vividly recall the ‘Gen Z movement’ that took place on Bhadra 23 and 24 (September 8–9). Sparked by the government’s decision to shut down social media platforms—a move that fueled widespread outrage among the youth—this movement, which gained traction under the hashtag ‘Nepo Baby,’ did more than just drive political change in Nepal; it also introduced many people to platforms like Discord. As the country moved toward political stability following the 'Gen-Z' movement, a group of young people—passionate about the Nepali language, AI, and software development—gathered on Discord to devise plans for Nepal’s AI future.

Some of these individuals had first met while participating in various technology-related events. Seeking to channel these connections into creative endeavors, they launched a Discord server called 'HimalayaGPT.' Over time, the group expanded to include AI engineers, data scientists, and linguists.

The server soon evolved beyond casual conversation. It became a hub for in-depth discussions on critical issues: Nepal’s sovereignty in the age of AI, national data security, the concept of "AI for Nepal, built by Nepalis," and the underrepresentation of Nepali cultural contexts within global AI systems. Concluding that developing a homegrown AI was essential, they envisioned building a robust "sovereign AI ecosystem" to strengthen Nepal’s development and sovereignty. To turn this vision and mission into reality, they formally registered an organization named 'Himalaya AI Labs.'

Notably, OpenAI—the developer of the generative AI tool ChatGPT—was itself established as a non-profit foundation, with the stated goal of creating AI that benefits everyone and ensuring its safety. In this spirit and framework, Himalaya AI Labs has registered itself in Nepal as a non-profit organization, with the objective of utilizing AI for the public good.

Himalaya AI is led by Dinesh Karki. Preliminary research on "tool calling"—published by the "Hermes Agent" project from the US-based AI research lab "Nous Research"—is considered to have laid the foundation for the current development of "agentic" AI. Having contributed to such significant AI research, Karki is now leveraging his experience to build Himalaya AI, serving as the organization’s Chief AI Officer (CAIO).

The organization’s leadership team also includes Rohan Pradhan as Chief Technology Officer (CTO), Milan Shrestha as Chief Information Officer (CIO), and Maulik Shakya as Chief Operating Officer (COO). Additionally, Pawan Dhakal serves as the Research and Implementation Lead; Anil Tamang and Kiran Silwal work as Voice R&D Engineers; and Siddhanta Bhattarai and Ishan Maharjan serve as Vision R&D Engineers. Furthermore, Sandhya Karki acts as the Dev and Community Lead, Deepa Amatya serves as the Growth Strategist and Government Relations Lead, and Rahul Pradhan works as a Safety Researcher. Supported by a team of interns, this group is conducting the research and development essential for Himalaya AI.

A photograph taken at an event jointly organized by PyTorch Community Nepal and Himalaya AI Labs towards the end of June. A photograph taken at an event jointly organized by PyTorch Community Nepal and Himalaya AI Lab towards the end of June.

What is Himalaya AI Lab doing?

The organization aims to make Nepal self-reliant in AI technology by developing AI models based on Nepal's own data, ensuring that the country does not have to rely solely on foreign models. As part of this initiative, the company has unveiled an AI model called 'Himalaya GPT.' It is a pre-trained model with a capacity of 0.5 billion (500 million) parameters, capable of processing the Nepali Devanagari script accurately and efficiently.

The organization has made this model openly available to everyone as an 'open research' resource via the 'Hugging Face' platform. Researchers and developers are free to use it and can even further refine or modify it.

Nep Tokenizer

AI or computers cannot read words directly the way humans do. Before processing any text, they break it down into smaller segments known as 'tokens.' The technology used to create these segments is called a 'tokenizer.'

Globally, most development in tokenizer technology has focused on the English language, while very little work has been done for the Nepali Devanagari script. Many foreign AI models tend to break a single Nepali word into numerous tiny, meaningless fragments. This makes the AI's output process slower and more costly.

Himalaya AI, for the Nepali language... A technology called 'Nep Tokenizer' has been developed. It breaks down Nepali words into the fewest possible segments. This allows for more cost-effective and faster operation and training of AI. By using fewer tokens, the AI ​​can process, retain, and respond to larger amounts of Nepali text simultaneously. It is claimed that this technology, researched by Himalaya AI Labs, processes Nepali text twice as effectively as other foreign models.

DeepGyan

This is an AI-based 'learning platform' currently being developed by Himalaya AI Labs. It is designed to help Nepali students read and understand their textbooks more easily. On this platform, students can upload PDF files of their textbooks. They can then ask the AI ​​questions or seek clarification on concepts they do not understand while reading. The AI ​​provides answers based exclusively on the content of the PDF file uploaded by the student. This ensures that students can acquire knowledge at the appropriate level with the AI's assistance.

The platform also incorporates OCR (Optical Character Recognition) technology, which can identify text even in PDF files that are not clear or easily readable, making the content accessible. Furthermore, it can generate short animations or videos to help explain complex topics or concepts in a simple, easy-to-understand manner. Students have the flexibility to search for answers and gain understanding either from the specific page they are currently reading or from the entire textbook. It is reported that this platform is being developed for use not only in Nepal but also in other countries as an AI-assisted learning platform.

Beyond the Devanagari script, the Himalaya AI team is working on a "multilingual voice AI" capable of accurately interpreting and processing Nepali accents and phonetics—the study of how individuals articulate sounds using their mouth, tongue, teeth, lips, and throat. This technology will enable AI-generated speech to sound natural, as if spoken by a native Nepali speaker. The team suggests that this could open up opportunities for using AI-generated voices in various applications, such as in call centers or for character dialogue in films.

What other initiatives are underway?

The team is also developing technology to recognize and interpret Devanagari script—whether printed or handwritten—found in documents. This is expected to facilitate the digitization of government and corporate records.

Furthermore, they are curating and releasing large, diverse datasets for training AI models. These include a GPT pre-training corpus, datasets for Devanagari and Indic-Latin scripts, as well as resources for OCR (Optical Character Recognition), SFT (Supervised Fine-Tuning), TTS (Text-to-Speech), tokenization, model evaluation, and other Nepali language-related applications. Furthermore, they have fine-tuned Google’s ‘Gemma 4’ model using a Nepali dataset, which has resulted in better and more accurate outputs in the Nepali language. It is claimed that by using this refined model, tasks that previously took 10–12 hours can now be completed in just 5–6 hours.

They are also developing a ‘Nepali AI repository’ to ensure that various Nepali companies can access the data necessary for AI research and development. To this end, they have initiated collaborations with Nepali universities and the government.

Additionally, they are working on technology that converts text written in Nepali into images. As part of this service, they are developing OCR (Optical Character Recognition) and Vision AI models capable of automatically scanning documents—such as citizenship certificates—submitted during account opening, and extracting the information contained within them. They are also developing multilingual voice and speech AI systems for bank call centers—capable of listening to customer inquiries and providing accurate responses in Nepali—and offering custom foundational model services for large-scale industries.

The organization aims to develop models that can understand languages ​​written in the Devanagari script (including Hindi), as well as languages ​​like Newari and Maithili.

What are the future plans?

While the current model has a capacity of 500 million (0.5 billion) parameters, the goal is to develop a model with a capacity of 8 billion parameters by the end of 2026. They aim to scale this AI model to a capacity of 100 billion parameters, aspiring to make it one of the best in South Asia.

However, enhancing an AI model's capabilities requires a corresponding level of computing power for training. Currently, the necessary GPU (Graphics Processing Unit) infrastructure is not yet available in Nepal. COO Maulik Shakya notes, "Some Nepali companies have begun offering GPU services. The government has also expressed an intention to facilitate the introduction of such infrastructure into Nepal. It would certainly make our work easier if we could source GPUs from Nepali companies." Nepali firms are not in a position to spend hundreds of thousands of rupees per hour to rent GPUs from foreign providers over the course of several months. Consequently, he explained that a company within the 'Himalaya AI Ecosystem' has already begun constructing a data center with the goal of providing GPU access to Nepali developers and startups—payable in local currency.

He stated that the organization is working across all necessary sectors to lay the groundwork for AI infrastructure in Nepal. Highlighting ongoing collaborations with other companies within the AI ​​ecosystem, he described the vision of Himalaya AI: "We are building a foundational AI ecosystem in Nepal—serving as an 'AI diffusion blueprint'—to establish the country as an AI hub for the 'Global South' over the next decade." We are making Tom.