Our Research

DeepRAG: Building a Custom Hindi Embedding Model for Retrieval Augmented Generation from Scratch

Nandakishor M

arXiv:2503.08213, 2025

This paper presents our work on DeepRAG, a specialized embedding model built specifically for Hindi language in RAG systems. While LLMs have gotten really good at generating text, their performance in retrieval tasks still depends heavily on having quality embeddings - something that's been lacking for Hindi despite being one of the world's most spoken languages. The author tackled this by creating embeddings from the ground up rather than just fine-tuning existing models. The process involved collecting diverse Hindi texts (over 2.7M samples), training a custom SentencePiece tokenizer that actually understands Hindi morphology, designing transformer architecture with Hindi-specific attention mechanisms, and optimizing with contrastive learning. Results showed a 23% improvement in retrieval precision compared to the multilingual models commonly used. The paper details a methodology that could help others working with low-resource languages where the one-size-fits-all multilingual models fall short. The embeddings have also been integrated with LangChain to build complete Hindi RAG systems. While there's still more to explore, this work addresses a critical gap for Hindi NLP and demonstrates why language-specific approaches matter.