AraManuscript-VL: Arabic Manuscript Recognition

This model is a fine-tuned version of Qwen2.5-VL-3B-Instruct for recognizing and transcribing Arabic historical manuscripts. It takes an image of a manuscript line as input and generates its corresponding Arabic transcription.

The model was adapted using LoRA fine-tuning on a multi-source Arabic manuscript dataset containing diverse examples from different collections. The training data includes various handwriting styles, historical scripts, and challenging manuscript conditions, including degraded images, faded text, image noise, and variations in document quality.

The objective of this model is to handle the difficulties commonly found in historical Arabic document recognition, where variations in handwriting, old writing conventions, damaged pages, and complex layouts make automatic transcription challenging.

---

Model Summary

Base model	Qwen2.5-VL-3B-Instruct
Model parameters	~3B parameters
Fine-tuning method	LoRA fine-tuning using PEFT and Transformers
Trainable parameters	~297M parameters (LoRA adapters)
Task	Image-Text-to-text OCR for Arabic historical manuscripts
Model type	Vision-Language Model (VLM)
Languages	Arabic
Domain	Historical manuscripts / Handwritten Text Recognition
License	Qwen Research License Agreement (inherited from the base model)

---

Dataset

The model was fine-tuned on a merged dataset combining real Arabic handwriting and historical manuscript samples with synthetic Arabic text generated using AI. The dataset includes line-level manuscript images, isolated handwritten words, and synthetic text lines, covering diverse writing styles, historical scripts, and image conditions.

The dataset contains three types of data:

Data type	Included datasets	Description
Historical manuscript line images	Historical Arabic Handwritten Text Recognition Dataset, Muharaf, KHATT, HCIMA, Nakba, Rasm, Rasam-1, Rasam-2, Baybars, Iskandar, Tarima	Handwritten manuscript line images paired with Arabic transcriptions. These samples cover different historical scripts, handwriting styles, layouts, and challenging document conditions.
Word-level images	Kalima words, Egyptian words	Cropped Arabic word images paired with their corresponding text labels.
synthetic Arabic text lines	Synthetic samples generated using AI	Additional synthetic training samples used in the initial training stage to help the model learn Arabic text recognition before moving to complex historical manuscript data.

Dataset split

Split	Number of samples
Training	102,071
Validation	8,042
Test	9,322

---

Preprocessing and Data Preparation

Before fine-tuning, all datasets were converted into a unified image-text format suitable for vision-language OCR training.

The preprocessing pipeline included:

• Image resizing and normalization: Images were resized while preserving their aspect ratio. Small images were upscaled when required according to dataset characteristics using Lanczos interpolation. Large images were constrained to a maximum resolution of 1024×1024 pixels. Final image dimensions were adjusted to multiples of 32 to ensure compatibility with the vision encoder.

• Data validation: Images were verified before training. Corrupted or unreadable files were identified and excluded from the training pipeline.

• Text preprocessing: Arabic transcriptions from different sources were normalized by removing unnecessary special characters, eliminating excessive spaces, and correcting formatting inconsistencies while preserving the original Arabic content. Different transcription formats were prepared according to the curriculum training stage:

• • Stage 1: Synthetic data transcriptions.
  • Stage 2: Transcriptions without Arabic letter dots (dots removed)
  • Stage 3: Original transcriptions with all letter dots preserved.

• Dataset-specific augmentation: Augmentation was applied only to the training split. Different transformations were selected for each manuscript collection, including brightness and contrast variations, blur simulation, slight geometric transformations, and morphological operations to simulate variations in scan quality, ink thickness, and writing appearance.

• Dataset splitting: The data was divided into training and validation subsets using a stratified split to preserve the distribution of manuscript types. Word-level datasets were used only during training.

---

Training Strategy (3-stage curriculum)

This model was not trained in a single step. A three-stage curriculum training strategy was used to progressively transform a general-purpose vision-language model into an Arabic manuscript OCR system.

Stage 1 — Arabic Text Recognition Initialization

The first stage focused on learning general Arabic text recognition using a synthetic Arabic text lines corpus. This stage helped the model learn basic image-text alignment, Arabic character shapes, and word-level recognition before introducing the complexity of historical manuscripts.

Stage 2 — Historical Manuscript Recognition without Letter Dots

The second stage started from the merged model obtained after Stage 1 and further fine-tuned it on historical manuscript transcriptions with Arabic letter dots removed. During this stage, the model learned to recognize historical handwriting while focusing on character shapes without the additional visual information provided by letter dots, such as:

• ٮ → ب، ت، ث
• ح → ج، ح، خ

Removing these fine-grained details reduced the complexity of the recognition task and allowed the model to first focus on learning manuscript structures, handwriting styles, and character shapes.

Stage 3 — Historical Manuscript Recognition with Letter Dots Preserved

The final stage started from the merged model obtained after Stage 2 and was fine-tuned on the original historical manuscript transcriptions with all Arabic letter dots preserved.

This stage improved the model's ability to distinguish visually similar Arabic characters and produce detailed transcriptions closer to the original manuscript text. The checkpoint released in this repository corresponds to this final stage.

---

Training Details

Fine-tuning approach

The model was adapted using LoRA instead of full parameter fine-tuning.

The base model was initialized from the previous curriculum stage checkpoint, and each stage continued fine-tuning from the merged weights of the previous stage.

Model and LoRA configuration

Parameter	Value
Base model	Qwen2.5-VL-3B-Instruct
Fine-tuning method	LoRA
Rank (r)	128
LoRA alpha	256
Dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Bias	None

Training configuration

Parameter	Value
Training framework	Transformers + PEFT
Trainer	Seq2SeqTrainer
Precision	bfloat16
Optimizer	Adafactor
Learning rate	1e-5
Scheduler	Cosine decay
Warmup steps	500
Epochs	3
Batch size	1
Gradient accumulation	16
Effective batch size	16
Weight decay	0.01
Maximum gradient norm	1.0

Input formatting and loss computation

Each training sample was formatted as an image-text instruction pair:

• The user input contains the manuscript image and the OCR instruction:

  "انسخ النص العربي الموجود في هذه الصورة حرفاً بحرف بدون أي إضافة أو تعديل:"

• The assistant output contains the corresponding Arabic transcription.

During training, the loss was computed only on the assistant transcription tokens. The instruction/prompt tokens were masked with -100 to prevent them from contributing to the training objective.

Hardware and Technical Environment

Training was performed using GPU acceleration with automatic device placement.

Component	Configuration
Python	3.11.15
PyTorch	2.11.0
PyTorch CUDA build	12.8
NVIDIA Driver CUDA support	13.2
Precision	bfloat16
GPU	NVIDIA RTX 4000 Ada Generation (20 GB)

Evaluation Results

Curriculum Learning Results

The model was trained progressively using a three-stage curriculum, where each stage increased the difficulty of the OCR task.

Training Stage	Training Data	CER (%) ↓	WER (%) ↓	CSER (%) ↓	WSER (%) ↓	WCER (%) ↓
Stage 1	Synthetic Arabic text recognition	0.09	0.44	0.08	0.39	0.06
Stage 2	Historical manuscripts (without dots)	6.55	23.50	6.40	23.02	4.53
Stage 3	Historical manuscripts (with dots)	6.63	22.37	5.79	20.14	4.30

The results show how gradually increasing the difficulty of the task helps the model adapt better to historical Arabic manuscripts.

---

Training and Validation Analysis

The model was trained progressively through three fine-tuning stages. The training and validation losses were monitored at each stage to analyze the learning behavior and convergence of the model.

Stage 1 — Synthetic Arabic Text Recognition

The first stage focused on learning fundamental Arabic text recognition using synthetic data.

The training and validation losses decrease during training, indicating that the model progressively learns to recognize Arabic text from the synthetic dataset.

Result	Value
Final Training Loss	0.012352
Final Validation Loss	0.016346
Generalization Gap	0.003994

---

Stage 2 — Historical Manuscripts Without Dots

In the second stage, the model was fine-tuned on historical Arabic manuscripts without dots.

The loss curves show the model's adaptation to historical manuscript writing. The decrease in validation loss indicates improved performance on unseen manuscript samples.

Result	Value
Final Training Loss	0.126825
Final Validation Loss	0.259728
Generalization Gap	0.132903

---

Stage 3 — Historical Manuscripts With Dots

The final stage introduced dotted historical manuscripts, requiring the model to distinguish fine-grained visual details such as dots and their positions, which are particularly important for Arabic character recognition.

The training and validation curves demonstrate the model's final adaptation to the complete historical manuscript recognition task. The relatively small gap between training and validation loss suggests good generalization to unseen samples.

Result	Value
Final Training Loss	0.175909
Final Validation Loss	0.468054
Generalization Gap	0.292145

---

Final Performance on Arabic Manuscript Datasets

The final Stage 3 model was evaluated on multiple Arabic manuscript datasets representing different historical writing styles.

Dataset	CER (%) ↓	WER (%) ↓	CSER (%) ↓	WSER (%) ↓	WCER (%) ↓
Kalima	6.44	22.83	5.15	19.05	3.99
KHATT	8.77	28.98	7.83	26.72	5.76
Nakba	3.65	14.39	3.18	12.96	2.37
HICMA	6.04	20.05	5.22	17.41	3.90
Muharaf	11.35	31.36	10.20	29.08	7.48
Tarima	8.45	30.00	7.22	26.37	5.42
RASM	4.61	18.03	4.05	16.38	3.00
RASAM-1	5.09	18.60	4.42	16.78	3.30
RASAM-2	12.14	37.21	11.08	35.38	8.08
Iskandar	6.90	22.35	6.13	20.12	4.52
Baybars	6.40	21.97	5.37	18.92	4.07
Overall	6.63	22.37	5.79	20.14	4.30

The differences in performance across datasets are related to the diversity of Arabic manuscript styles, including variations in handwriting, historical period, image quality, and writing complexity.

Comparison with Existing Arabic OCR Models

The proposed model was compared with representative Arabic OCR approaches.

Dataset	Proposed Method		Best Baseline		Reference
	CER (%)	WER (%)	CER (%)	WER (%)
RASM	4.61	18.03	5.72	29.74	Paper
KHATT	8.77	28.98	7.76	29.98	Paper
Kalima	6.44	22.83	4.10	20.43	Paper
Nakba	3.65	14.39	7.90	24.40	Baseline study
HICMA	6.04	20.05	58.47	94.51	Paper
Muharaf	11.35	31.36	11.67	36.47	Paper
TariMa	8.45	30.00	24.20	—	AMIDDA 1.0 Line 4B
RASAM 1	5.09	18.60	25.00	—	AMIDDA 1.0 Line 4B
RASAM 2	12.14	37.21	27.60	—	AMIDDA 1.0 Line 4B
Iskandar	6.90	22.35	15.90	—	AMIDDA 1.0 Line 4B
Baybars	6.40	21.97	13.40	—	AMIDDA 1.0 Line 4B

Usage

The publicly available AraManuscript-VL checkpoint uses 3-bit HQQ quantization, with the vision encoder and output layer remaining unquantized. The example below loads the model directly from main and applies black-border detection and image resizing before recognizing Arabic text.

Requirements

The inference example was tested with Python 3.11 and the following packages:

pip install "torch==2.14.1" "torchvision" "transformers==4.57.6" "tokenizers==0.22.1" "huggingface-hub==0.36.0" accelerate hqq qwen-vl-utils opencv-python-headless pillow

A compatible PyTorch and Torchvision installation is required. For CUDA-enabled inference, install PyTorch and Torchvision builds compatible with your system before installing the remaining dependencies.

Inference

import cv2
import numpy as np
import torch

from PIL import Image
from transformers import (
    AutoConfig,
    AutoProcessor,
    Qwen2_5_VLForConditionalGeneration,
)
from hqq.core.quantize import HQQLinear

MODEL_ID = "dsaru-cerist/AraManuscript-VL"
REVISION = "main"

PROMPT = (
    "انسخ النص العربي الموجود في هذه الصورة حرفاً بحرف "
    "بدون أي إضافة أو تعديل:"
)


def patch_rope(config):
    section = [16, 24, 24]

    for c in (config, getattr(config, "text_config", None)):
        if c is None:
            continue

        rp = (
            getattr(c, "rope_parameters", None)
            or c.to_dict().get("rope_parameters")
            or {}
        )

        section = rp.get("mrope_section", section)
        c.rope_scaling = {
            "type": "mrope",
            "rope_type": "default",
            "mrope_section": section,
        }
        c.rope_theta = rp.get("rope_theta", 1000000.0)

    return config


def fix_model(model):
    # Match the input embedding and output head weights
    emb = model.get_input_embeddings().weight
    head = model.get_output_embeddings()

    head.weight = torch.nn.Parameter(
        emb.data.to(
            device=head.weight.device,
            dtype=torch.float16,
        ),
        requires_grad=False,
    )

    # Convert remaining FP32 parameters to FP16
    for p in model.parameters():
        if p.dtype == torch.float32:
            p.data = p.data.half()

    # Set HQQ computation to FP16
    for module in model.modules():
        if isinstance(module, HQQLinear):
            module.compute_dtype = torch.float16

            if getattr(module, "meta", None) is not None:
                module.meta["compute_dtype"] = torch.float16

                for key, value in list(module.meta.items()):
                    if (
                        torch.is_tensor(value)
                        and value.dtype == torch.float32
                    ):
                        module.meta[key] = value.half()

            if (
                getattr(module, "bias", None) is not None
                and module.bias.dtype == torch.float32
            ):
                module.bias = module.bias.half()

    return model


# Load the model configuration
config = patch_rope(
    AutoConfig.from_pretrained(
        MODEL_ID,
        revision=REVISION,
    )
)

# Load the quantized model
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    MODEL_ID,
    revision=REVISION,
    config=config,
    device_map="auto",
    dtype=torch.float16,
).eval()

model = fix_model(model)

# Verify that the HQQ layers were loaded
n_hqq = sum(
    isinstance(module, HQQLinear)
    for module in model.modules()
)
print(f"HQQ layers loaded: {n_hqq}")

if n_hqq != 252:
    raise RuntimeError(
        f"Expected 252 HQQ layers, found {n_hqq}"
    )

# Load the processor
processor = AutoProcessor.from_pretrained(
    MODEL_ID,
    revision=REVISION,
    trust_remote_code=True,
)
processor.tokenizer.padding_side = "left"


def preprocess_image(image):
    image = image.convert("RGB")

    image_bgr = cv2.cvtColor(
        np.array(image), cv2.COLOR_RGB2BGR
    )
    gray = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2GRAY)

    # Detect the manuscript content
    mask = np.zeros_like(gray, dtype=np.uint8)
    mask[gray > 40] = 255

    kernel = cv2.getStructuringElement(
        cv2.MORPH_RECT, (11, 11)
    )
    mask = cv2.morphologyEx(
        mask,
        cv2.MORPH_CLOSE,
        kernel,
        iterations=2,
    )

    contours, _ = cv2.findContours(
        mask,
        cv2.RETR_EXTERNAL,
        cv2.CHAIN_APPROX_SIMPLE,
    )

    valid = [
        contour
        for contour in contours
        if cv2.contourArea(contour) >= 300
    ]

    if valid:
        contour = max(valid, key=cv2.contourArea)
        x, y, w, h = cv2.boundingRect(contour)

        if w > 0 and h > 0:
            image = image.crop((
                max(0, x - 2),
                max(0, y - 2),
                min(image.width, x + w + 2),
                min(image.height, y + h + 2),
            ))

    # Limit image dimensions
    image.thumbnail(
        (1024, 1024),
        Image.Resampling.LANCZOS,
    )

    width, height = image.size
    width = max(32, ((width + 31) // 32) * 32)
    height = max(96, ((height + 31) // 32) * 32)

    return image.resize(
        (width, height),
        Image.Resampling.LANCZOS,
    )


def transcribe(image_path, max_new_tokens=128):
    image = preprocess_image(Image.open(image_path))

    messages = [{
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": PROMPT},
        ],
    }]

    text = processor.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )

    inputs = processor(
        text=[text],
        images=[image],
        padding=True,
        return_tensors="pt",
    )

    input_device = model.get_input_embeddings().weight.device
    inputs = inputs.to(input_device)

    if "pixel_values" in inputs:
        inputs["pixel_values"] = inputs["pixel_values"].to(
            dtype=torch.float16
        )

    with torch.inference_mode():
        output_ids = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=False,
            num_beams=1,
            repetition_penalty=1.0,
            eos_token_id=processor.tokenizer.eos_token_id,
        )

    # Remove the prompt tokens
    output_ids = output_ids[:, inputs.input_ids.shape[1]:]

    return processor.batch_decode(
        output_ids,
        skip_special_tokens=True,
        clean_up_tokenization_spaces=False,
    )[0]

# Example
print(transcribe("path/to/manuscript_line.jpg"))

Input and Output

• Preferred inputs: Cropped Arabic manuscript text-line images.
• Preprocessing: Black-border detection and resizing are applied before inference. Use the same preprocessing settings as the evaluation pipeline to reproduce reported results.
• Decoding: Greedy decoding with do_sample=False and num_beams=1.
• Output: The model returns the recognized Arabic text.

Intended use

• Automatic transcription of historical Arabic manuscript images into digital text.
• Supporting researchers, paleographers, and archivists in the digitization and analysis of Arabic manuscript collections.
• Creating searchable archives and facilitating access to historical documents.
• Providing a foundation for further adaptation to new manuscript collections, writing styles, and historical periods.

Out of scope

• General OCR for modern printed Arabic documents such as forms, invoices, or books.
• Recognition of non-Arabic scripts or languages not represented in the training data.
• Fully automatic publication-ready transcription without human verification.

---

Limitations

• Performance varies significantly by manuscript type, muharaf style documents (highly diverse handwriting, historical degradation) show the highest error rates (10.8% CER) versus more homogeneous sources like NAKBA (3.5% CER).
• The model can hallucinate plausible-looking but incorrect text on heavily degraded or ambiguous passages, always have a human review output intended for scholarly use.

Datasets

The model was trained and evaluated using the following publicly available datasets:

• Kalima Dataset (Bouchal et al., 2025 [1])
• KHATT Dataset (Paper)
• Hcima Dataset (official website)
• Muharaf Dataset (Hugging Face)
• Nakba Dataset (Hugging Face)
• Baybars Dataset (Hugging Face)
• Iskandar Dataset (Hugging Face)
• Tarima Dataset (Hugging Face)
• RASM Dataset (Keinan-Schoonbaert and British Library)
• RASAM-1 Dataset (Hugging Face)
• RASAM-2 Dataset (Hugging Face)
• Egypt word Dataset (Hugging Face)

Kalima OCR

The fine-tuned model is integrated into Kalima OCR, a comprehensive AI platform for processing and digitizing historical Arabic manuscripts and heritage documents.

Kalima OCR provides a wide range of AI-powered capabilities, including automatic handwriting recognition, manuscript analysis and comparison, intelligent manuscript interaction, heritage content summarization, Quranic text verification and diacritization, Hadith referencing, poetry verification, biographical text processing, and audio transcription.

Acknowledgements

This work builds upon Qwen2.5-VL-3B-Instruct developed by Alibaba Cloud.

We sincerely thank the creators and contributors of the publicly available Arabic handwriting and historical manuscript datasets used in this project.

We also acknowledge the support of the Research Center on Scientific and Technical Information (CERIST) providing the research environment, computational resources, and GPU infrastructure required for model development, training, and experimentation.

Citation

If you use this model, its fine-tuning code, or the results reported in this work in your research, please cite the following references:

@software{dsaru-ceristAraManuscript-VL,
  author  = {Belgacem, Khawla and Belaid, Ahror and Bouchal, Hakim},
  title   = {AraManuscript-VL: Qwen2.5-VL for Arabic Manuscript Recognition},
  year    = {2026},
  url     = {https://huggingface.co/dsaru-cerist/AraManuscript-VL}
}

@article{bouchal2025towards,
  author    = {Bouchal, Hakim and Belaid, Ahror and Meziane, Farid},
  title     = {Towards Accurate Recognition of Historical Arabic Manuscripts: A Novel Dataset and a Generalizable Pipeline},
  journal   = {ACM Transactions on Asian and Low-Resource Language Information Processing},
  volume    = {24},
  number    = {10},
  articleno = {104},
  year      = {2025},
  publisher = {Association for Computing Machinery},
  doi       = {10.1145/3744243},
  url       = {https://doi.org/10.1145/3744243}
}

References

[1] H. Bouchal, A. Belaid, and F. Meziane, "Towards accurate recognition of historical Arabic manuscripts: A novel dataset and a generalizable pipeline," ACM Transactions on Asian and Low-Resource Language Information Processing, vol. 24, no. 10, pp. 1–30, 2025.