Translate Server

https://github.com/dwani-ai/indic-translate-server.git

git clone https://github.com/dwani-ai/indic-translate-server.git
cd indic-translate-server

python -m venv --system-site-packages venv

source venv/bin/activate
pip install --upgrade pip setuptools wheel packaging cython

pip install torch==2.7.1 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

#pip install -r requirements.txt

pip install -e git+https://github.com/VarunGumma/IndicTransToolkit.git@main#egg=IndicTransToolkit

pip install fastapi uvicorn  "numpy<2.0"
python src/server/translate_api.py --host 0.0.0.0 --port 7862 --device cpu

Indic Translate Server

Overview

This project sets up an Indic translation server, allowing translation between various languages including English, Kannada, Hindi, and others. It utilizes models from AI4Bharat to perform translations.

We suggest to use non-distilled models for better translation.

Languages Supported

Here is the list of languages supported by the IndicTrans2 models:

Assamese (asm_Beng)	Kashmiri (Arabic) (kas_Arab)	Punjabi (pan_Guru)
Bengali (ben_Beng)	Kashmiri (Devanagari) (kas_Deva)	Sanskrit (san_Deva)
Bodo (brx_Deva)	Maithili (mai_Deva)	Santali (sat_Olck)
Dogri (doi_Deva)	Malayalam (mal_Mlym)	Sindhi (Arabic) (snd_Arab)
English (eng_Latn)	Marathi (mar_Deva)	Sindhi (Devanagari) (snd_Deva)
Konkani (gom_Deva)	Manipuri (Bengali) (mni_Beng)	Tamil (tam_Taml)
Gujarati (guj_Gujr)	Manipuri (Meitei) (mni_Mtei)	Telugu (tel_Telu)
Hindi (hin_Deva)	Nepali (npi_Deva)	Urdu (urd_Arab)
Kannada (kan_Knda)	Odia (ory_Orya)

Live Server

We have hosted an Translation service for Indian languages.

https://demo.dwani.ai

Prerequisites

Python 3.10 + VsCode
Ubuntu 22.04
Internet access to download translation models.

Setting Up the Development Environment

Create a virtual environment:
```
python -m venv venv
```
Activate the virtual environment:
For Mac/Linux
```
source venv/bin/activate
```
On Windows, use:
```
venv\Scripts\activate
```
Install dependencies: ```

pip install torch==2.7.1 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 pip install -r requirements.txt ```

Model Downloads for Translation

Collection Models on HuggingFace - IndicTrans2

Below is a table summarizing the available models for different translation tasks:

Task	Variant	Model Name	VRAM Size	Download Command
Indic to English	200M (distilled)	indictrans2-indic-en-dist-200M	950 MB	`huggingface-cli download ai4bharat/indictrans2-indic-en-dist-200M`
	1B (base)	indictrans2-indic-en-1B	4.5 GB	`huggingface-cli download ai4bharat/indictrans2-indic-en-1B`
English to Indic	200M (distilled)	indictrans2-en-indic-dist-200M	950 MB	`huggingface-cli download ai4bharat/indictrans2-en-indic-dist-200M`
	1B (base)	indictrans2-en-indic-1B	4.5 GB	`huggingface-cli download ai4bharat/indictrans2-en-indic-1B`
Indic to Indic	320M (distilled)	indictrans2-indic-indic-dist-320M	950 MB	`huggingface-cli download ai4bharat/indictrans2-indic-indic-dist-320M`
	1B (base)	indictrans2-indic-indic-1B	4.5 GB	`huggingface-cli download ai4bharat/indictrans2-indic-indic-1B`

Sample Code

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from IndicTransToolkit import IndicProcessor

DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

src_lang, tgt_lang = "hin_Deva", "eng_Latn"
model_name = "ai4bharat/indictrans2-indic-en-dist-200M"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)

model = AutoModelForSeq2SeqLM.from_pretrained(
    model_name, 
    trust_remote_code=True, 
    torch_dtype=torch.float16, # performance might slightly vary for bfloat16
    attn_implementation="flash_attention_2"
).to(DEVICE)

ip = IndicProcessor(inference=True)

input_sentences = [
    "जब मैं छोटा था, मैं हर रोज़ पार्क जाता था।",
    "हमने पिछले सप्ताह एक नई फिल्म देखी जो कि बहुत प्रेरणादायक थी।",
]

batch = ip.preprocess_batch(
    input_sentences,
    src_lang=src_lang,
    tgt_lang=tgt_lang,
)

DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

# Tokenize the sentences and generate input encodings
inputs = tokenizer(
    batch,
    truncation=True,
    padding="longest",
    return_tensors="pt",
    return_attention_mask=True,
).to(DEVICE)

# Generate translations using the model
with torch.no_grad():
    generated_tokens = model.generate(
        **inputs,
        use_cache=True,
        min_length=0,
        max_length=256,
        num_beams=5,
        num_return_sequences=1,
    )

# Decode the generated tokens into text
with tokenizer.as_target_tokenizer():
    generated_tokens = tokenizer.batch_decode(
        generated_tokens.detach().cpu().tolist(),
        skip_special_tokens=True,
        clean_up_tokenization_spaces=True,
    )

# Postprocess the translations, including entity replacement
translations = ip.postprocess_batch(generated_tokens, lang=tgt_lang)

for input_sentence, translation in zip(input_sentences, translations):
    print(f"{src_lang}: {input_sentence}")
    print(f"{tgt_lang}: {translation}")

Run the sample code

python translate_code.py

Alternate forms of Development

Running with FastAPI Server

Install dependencies:

pip install -r server-requirements.txt

You can run the server using FastAPI: 1. with GPU

python src/server/translate_api.py --port 7860 --host 0.0.0.0 --device cuda --use_distilled False

with CPU only

python src/server/translate_api.py --port 7860 --host 0.0.0.0 --device cpu --use_distilled False

Evaluating Results for FastAPI Server

You can evaluate the translation results using curl commands. Here are some examples:

English to Kannada

curl -X 'POST' \
  'http://localhost:7860/translate?tgt_lang=kan_Knda&src_lang=eng_Latn&device_type=cuda' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "sentences": [
    "Hello, how are you?", "Good morning!"
  ],
  "src_lang": "eng_Latn",
  "tgt_lang": "kan_Knda"
}'

Response:

{
  "translations": [
    "ಹಲೋ, ಹೇಗಿದ್ದೀರಿ? ",
    "ಶುಭೋದಯ! "
  ]
}

Kannada to English

curl -X 'POST' \
  'http://localhost:7860/translate?src_lang=kan_Knda&tgt_lang=eng_Latn&device_type=cuda' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "sentences": [
    "ನಮಸ್ಕಾರ, ಹೇಗಿದ್ದೀರಾ?", "ಶುಭೋದಯ!"
  ],
  "src_lang": "kan_Knda",
  "tgt_lang": "eng_Latn"
}'

Response:

{
  "translations": ["Hello, how are you?", "Good morning!"]
}

Kannada to Hindi

curl -X 'POST' \
  'http://localhost:7860/translate?src_lang=kan_Knda&tgt_lang=hin_Deva' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "sentences": [
    "ನಮಸ್ಕಾರ, ಹೇಗಿದ್ದೀರಾ?", "ಶುಭೋದಯ!"
  ],
  "src_lang": "kan_Knda",
  "tgt_lang": "hin_Deva"
}'

Response

{ "translations": [ "हैलो, कैसा लग रहा है? ", "गुड मॉर्निंग! " ] }

CPU

curl -X 'POST' \
  'http://localhost:7860/translate?src_lang=kan_Knda&tgt_lang=eng_Latn&device_type=cpu' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "sentences": [
    "ನಮಸ್ಕಾರ, ಹೇಗಿದ್ದೀರಾ?", "ಶುಭೋದಯ!"
  ],
  "src_lang": "kan_Knda",
  "tgt_lang": "eng_Latn"
}'

Response

{
  "translations": [
    "Hello, how are you?",
    "Good morning!"
  ]
}

References

IndicTrans2 Paper
AI4Bharat IndicTrans2 Model
AI4Bharat IndicTrans2 GitHub Repository
IndicTransToolkit
Extra - pip install git+https://github.com/VarunGumma/IndicTransToolkit.git

Contributing

We welcome contributions! Please read the CONTRIBUTING.md file for guidelines on how to contribute to this project.

Also you can join the discord group to collaborate

License

This project is licensed under the MIT License - see the LICENSE file for details.

FAQ

Q: How do I change the source and target languages?

A: Modify the compose.yaml file to set the SRC_LANG and TGT_LANG variables as needed.

Q: How do I download the translation models?

A: Use the huggingface-cli commands provided in the Downloading Translation Models section.

Q: How do I run the server locally?

A: Follow the instructions in the Running with FastAPI Server section.

License

This README provides a comprehensive guide to setting up and running the Indic Translate Server. For more details, refer to the linked resources.

Citation

@article{gala2023indictrans,
title={IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages},
author={Jay Gala and Pranjal A Chitale and A K Raghavan and Varun Gumma and Sumanth Doddapaneni and Aswanth Kumar M and Janki Atul Nawale and Anupama Sujatha and Ratish Puduppully and Vivek Raghavan and Pratyush Kumar and Mitesh M Khapra and Raj Dabre and Anoop Kunchukuttan},
journal={Transactions on Machine Learning Research},
issn={2835-8856},
year={2023},
url={https://openreview.net/forum?id=vfT4YuzAYA},
note={}
}

Indic Translate Server

Table of Contents

Overview

Languages Supported

Live Server

Prerequisites

Setting Up the Development Environment

Model Downloads for Translation

Sample Code

Run the sample code

Alternate forms of Development

Running with FastAPI Server

Evaluating Results for FastAPI Server

English to Kannada

Kannada to English

Kannada to Hindi

Response

CPU

Response

References

Contributing

License

FAQ

License

Citation