Medical information is only useful when people can understand it. While English has a large ecosystem of medical resources, high-quality medical translation for Nepali remains limited. General-purpose language models can translate everyday text reasonably well, but they often struggle with clinical terminology, dosage instructions, and domain-specific language.
This project explores whether a large language model can be adapted for English–Nepali medical translation using parameter-efficient fine-tuning instead of retraining the entire model. The goal was to build a reproducible baseline that can serve as a foundation for future research in low-resource medical NLP.
Why I Built This
Many multilingual language models support Nepali, but most are trained for general translation rather than medical language. Clinical text introduces challenges that aren’t common in everyday conversations, including specialised terminology, formal writing style, and safety-critical information.
Instead of relying on prompt engineering alone, I wanted to evaluate whether domain-specific fine-tuning could significantly improve translation quality while remaining feasible on consumer-grade hardware.
Building the Dataset
One of the biggest challenges was the lack of publicly available English–Nepali medical parallel data. To address this, I compiled a parallel corpus consisting of 58,682 aligned sentence pairs collected from publicly available medical resources, health education materials, question–answer datasets, and health-related articles. The dataset contains only publicly available information and excludes personally identifiable or protected health information.
Model and Training
The project uses Meta’s LLaMA 3.1 8B Instruct as the base model. Rather than fine-tuning all 8 billion parameters, I adopted a parameter-efficient approach using LoRA together with 4-bit NF4 quantization (QLoRA). This reduced GPU memory requirements while training only about 0.52% of the model’s parameters.
Training was performed on a Tesla T4 (16 GB VRAM) provided by Google Colab over approximately 25–30 hours, demonstrating that domain adaptation can be achieved without access to expensive multi-GPU infrastructure.
Evaluation
To evaluate the model, I used two standard machine translation metrics:
- BLEU for n-gram overlap
- ChrF++ for character-level similarity, which is particularly useful for morphologically rich languages such as Nepali.
The fine-tuned model achieved:
| Metric | Score |
|---|---|
| BLEU | 11.63 |
| ChrF++ | 34.65 |
Compared with the zero-shot LLaMA 3.1 baseline, this represents an 8.9× improvement in BLEU and a 2.1× improvement in ChrF++, showing that parameter-efficient fine-tuning substantially improves domain-specific translation performance.
What I Learned
The project reinforced that fine-tuning is only part of building a reliable translation system. Dataset quality, domain coverage, evaluation methodology, and error analysis all have a significant impact on the final results.
Although the model preserves many medical terms and produces grammatically correct Nepali translations, it can still omit safety-critical information in some cases. For that reason, it should be treated as a first-draft translation assistant rather than a replacement for qualified bilingual medical professionals.
Future Work
There are several directions for extending this research:
- Expand the dataset to specialised medical domains such as cardiology and oncology.
- Compare directly with multilingual encoder–decoder translation models on the same evaluation set.
- Support bidirectional Nepali↔English translation.
- Incorporate human evaluation alongside automatic metrics.
- Explore hybrid approaches that combine language models with medical terminology databases.