Open Armenian LLM project releases data, model and full training recipe

A team from COPA has published an open language-model ecosystem built specifically for Armenian. The project is detailed in the Hugging Face Blog, alongside a research paper, datasets, model checkpoint and code repository.

The release includes ArmWeb, a curated collection of 4.37 million Armenian news documents containing 3.3 billion Gemma-4 tokens, and ArmSTEM, a verified English–Armenian corpus of 373,000 mathematics and science problems. The latter includes 324,000 problems with step-by-step solutions. Both resources are intended to address a long-standing gap: Armenian training material has largely appeared only as part of broad multilingual web crawls.

The team also released arm-gemma-e4b, a Gemma-4-E4B model continued-pretrained on 10 billion tokens from the project’s data. Its training recipe, evaluation configurations and corpus pipeline are available under an MIT-licensed codebase, while the model uses Gemma terms.

Benchmark results

On a six-task Armenian likelihood evaluation, arm-gemma-e4b achieved a mean accuracy of 0.500, ahead of the unadapted Gemma-4-E4B base at 0.477, ArmenianGPT-1.0-3B at 0.471, HyGPT-10b at 0.436 and tweety-7b at 0.353. It recorded the strongest result among the listed open models on the Belebele-hye, INCLUDE-Armenian and m-MMLU-hy knowledge tasks, although other models led on some individual benchmarks.

For developers, the important distinction is reproducibility. The project publishes the training data sources, token counts, hyperparameters and evaluation setup, making it possible to audit the model or adapt the workflow for other underrepresented languages. It also gives Armenian-language applications a directly usable foundation instead of requiring teams to start with opaque multilingual data.

Source: Hugging Face Blog