Azul OCR - Tifinagh Handwritten Character Recognition

March 2024 Computer vision Academic Project Ibn Tofail University, Kenitra February 2024 - June 2024
Azul OCR - Tifinagh Handwritten Character Recognition
OCR Tifinagh CNN PaddleOCR Tesseract Cultural Heritage Deep Learning Computer Vision
View on GitHub Watch Demo

🎯 Project Overview

Azul OCR (ⴰⵣⵓⵍ - “Hello” in Amazigh) is an innovative OCR system specialized in Tifinagh handwritten character recognition. The project aims to preserve and digitize Amazigh cultural heritage by developing an artificial intelligence solution capable of accurately recognizing Tifinagh handwritten characters and sentences.

Team:

  • Rima Daqch - Project Manager, Tesseract OCR Developer
  • Adil Eddarif - Tesseract OCR Developer, CNN Model Developer
  • Brahim El Aboudi - PaddleOCR Developer, CNN Model Developer
  • El Mehdi Hicham - PaddleOCR Developer, CNN Model Developer
  • Zineb Lahraoui - PaddleOCR Developer, CNN Model Developer

Supervised by: Prof. Khadija Lekdioui

Mission: “Preserving and digitizing Amazigh cultural heritage”

🔍 Problem Statement

Challenges

“Tifinagh script is an essential element of Amazigh cultural heritage, but it remains underrepresented in modern technological tools.”

  • 📚 Lack of OCR tools: No dedicated OCR solution for Tifinagh manuscripts
  • 🎨 Character complexity: Significant variability in handwriting
  • ⚠️ Cultural risk: Potential loss of valuable content for future generations
  • 🔬 Limited accuracy: Existing solutions not adapted to Tifinagh specificities

Solution

Development of Azul OCR, hybrid system combining:

  • Tesseract OCR fine-tuned for Tifinagh
  • PaddleOCR adapted for Amazigh characters
  • Custom CNN for character recognition
  • Intuitive interface for capture and recognition

🏗️ Technical Architecture

🔧 Key Components

Module 1: Image Preprocessing

Applied techniques:

  • 🖼️ Otsu Thresholding: Automatic binarization
  • 📐 Resizing: 64×64 → 128×128 pixels
  • 🎨 Filtering: Noise reduction
  • 🔄 Augmentation: Rotation, translation, zoom

Module 2: Recognition Engines

Three approaches:

  • 📝 Tesseract OCR: Fine-tuned on AMHCD dataset
  • 🎯 PaddleOCR: Adapted lightweight architecture
  • 🧠 Custom CNN: Model trained from scratch
  • 🔀 Hybrid model: Prediction fusion

📊 System Workflow

Azul OCR - Complete System Workflow ⴰⵣⵓⵍ - Tifinagh Manuscript Recognition System 📸 Input Image Preprocessing Otsu + Resize + Normalize Detection Type? Characters CNN Model Custom Architecture 33 Tifinagh classes 6 Conv layers + Dropout 94.2% Accuracy ⭐ Precision: 93.8% | Recall: 94.1% Best for: Isolated characters Words/Phrases Tesseract OCR Fine-tuned on AMHCD LSTM + Language Model Custom training data 87.5% Accuracy ⭐ CER: 12.5% | WER: 18.3% Best for: Simple layouts Text Lines PaddleOCR DB++ + CRNN End-to-end detection Fine-tuned on Tifinagh 89.6% Accuracy ⭐ CER: 10.8% | WER: 15.4% Best for: Complex layouts 🏆 Hybrid Model (Weighted Voting): 91.2% Accuracy | Best Overall Performance

🔬 Technical Deep Dive

1️⃣ CNN Architecture Details

🧠 Custom CNN Architecture for Tifinagh Recognition 6 Convolutional Layers + Dropout Regularization + 33-class Classification Input Layer 128×128×1 Grayscale Normalized 📸 Conv Block 1 Conv2D (32, 3×3) ReLU MaxPool (2×2) → 64×64×32 Conv Block 2 Conv2D (64, 3×3) ReLU MaxPool (2×2) → 32×32×64 Conv Block 3 Conv2D (128, 3×3) ReLU MaxPool (2×2) → 16×16×128 Conv Block 4 Conv2D (256, 3×3) ReLU MaxPool (2×2) → 8×8×256 Conv Block 5 Conv2D (512, 3×3) ReLU MaxPool (2×2) → 4×4×512 Conv Block 6 Conv2D (512, 3×3) ReLU MaxPool (2×2) → 2×2×512 Flatten Layer 2048 neurons (2×2×512) Dense Layer 1 512 neurons + ReLU Dropout (50%) Output Layer 33 neurons (classes) Softmax activation 📊 Model Statistics • Total Parameters: ~2.5M • Training Time: ~3 hours • Inference Speed: ~15ms/image • Accuracy: 94.2% 🎯 Training Details • Optimizer: Adam (lr=0.001) • Loss Function: Categorical Crossentropy • Batch Size: 32 • Epochs: 50 (Early Stopping) 🏆 Best performance on isolated character recognition | Trained on AMHCD dataset

2️⃣ Tesseract OCR Fine-tuning

📝 Tesseract OCR Fine-tuning Pipeline Adapted for Tifinagh Script Recognition | LSTM-based Architecture 1️⃣ Data Preparation • AMHCD Dataset • Ground Truth (GT) Files • Box Files Generation 2️⃣ LSTM Training • 100+ iterations • Language Model • Character Error Rate ↓ 3️⃣ Evaluation • Test on validation set • CER: 12.5% • WER: 18.3% 4️⃣ Deployment • Export .traineddata • Integration in App • Real-time Recognition 🏗️ Tesseract LSTM Architecture Input Layer: Variable-width image strips (H=48px) Convolutional Layers: 3 layers for feature extraction LSTM Layers: Bidirectional LSTM (2 layers, 256 units each) Output Layer: CTC (Connectionist Temporal Classification) 📊 Training Metrics • Training Images: 15,000+ • Iterations: 120 epochs • Training Time: ~12 hours • Final CER: 12.5% • Word Accuracy: 87.5% 🔧 Key Optimizations ✅ Custom Unicode mapping ✅ Tifinagh-specific wordlists ✅ Context-aware correction ✅ Augmented training data ✅ Multi-font training 🎯 Best Use Cases ✓ Documents with simple layouts ✓ Printed text recognition ✓ Well-separated lines ✓ High contrast images ✓ Historical manuscripts 🏆 87.5% accuracy on word/phrase recognition | Ideal for structured documents

3️⃣ PaddleOCR Integration Pipeline

🎯 PaddleOCR Complete Pipeline DB++ Text Detection + CRNN Recognition | End-to-End Tifinagh OCR 📄 Input Image Tifinagh Text Text Detection DB++ Algorithm Differentiable Binarization ResNet50 Backbone 📦 Bounding Boxes Polygons (x,y coords) Text Recognition CRNN Architecture CNN + BiLSTM + CTC Fine-tuned on Tifinagh ✅ Final Output 🔍 DB++ Text Detection Backbone: ResNet50 (Pretrained) Feature Pyramid: FPN for multi-scale Head: Probability + Threshold maps Output: Polygon coordinates (x,y) Performance: 95.3% detection rate 📝 CRNN Text Recognition CNN: 7 convolutional layers RNN: 2 BiLSTM layers (256 units) CTC: Connectionist Temporal Classification Vocabulary: 33 Tifinagh characters Accuracy: 89.6% on test set 🎓 Fine-tuning Process Detection Model: • Pretrained: ICDAR2015 dataset • Fine-tuned: 5000 Tifinagh images • Epochs: 300 (early stopping) • Learning rate: 0.001 → 0.0001 Recognition Model: • Pretrained: Multi-language CRNN • Fine-tuned: AMHCD dataset • Epochs: 200 with augmentation • Batch size: 256 📊 Performance Metrics Detection Precision: 95.3% Detection Recall: 92.7% Recognition Accuracy: 89.6% Character Error Rate: 10.8% Inference Speed: ~45ms/image 🏆 Best performance on complex layouts and multi-line text | End-to-end pipeline

💻 User Interface

Features

✅ Upload Image: Support PNG, JPG, JPEG
✅ Take Picture: Direct capture via webcam
✅ Real-time Recognition: Instant results
✅ Copy Output: Copy recognized text
✅ Multi-format Export: TXT, JSON, CSV

Interface Screenshots

Azul OCR intuitive user interface

Azul Interface

Gate Shift Modules

🛠️ Challenges & Solutions

Challenge 1: Handwriting Variability

Problem: Great diversity of writing styles
Solution:

  • Intensive augmentation (×5)
  • Hybrid model (fusion of 3 approaches)
  • Fine-tuning on real data

Challenge 2: Limited Dataset

Problem: Few annotated data available
Solution:

  • Collaboration with IRCAM
  • Aggressive data augmentation
  • Transfer learning from pre-trained models

Challenge 3: Sentence Recognition

Problem: Difficult word segmentation
Solution:

  • Use of PaddleOCR for detection
  • Post-processing with linguistic rules
  • Contextual correction

💼 Impact & Value

For Cultural Heritage

✅ Digital preservation of ancient manuscripts
✅ Accessibility of Amazigh texts to all
✅ Facilitated education for new generations
✅ Academic research on Amazigh culture

For Institutions

✅ Rapid digitization of archives
✅ Automatic indexing of documents
✅ Text search in digital corpus
✅ Reduced costs vs manual annotation

🔮 Future Improvements

Short-Term:

  • Improve accuracy on complex sentences (>95%)
  • Support for regional Tifinagh variants
  • REST API for external integration

Long-Term:

  • Mobile application (iOS/Android)
  • Recognition of degraded historical texts
  • Extension to other African alphabets
  • Multilingual OCR (Tifinagh + Latin + Arabic)

🛠️ Technical Stack

Deep Learning: TensorFlow, Keras, PyTorch
OCR Engines: Tesseract 4.0, PaddleOCR
Computer Vision: OpenCV, scikit-image
Frontend: React.js, HTML5, CSS3
Backend: Flask, FastAPI
Deployment: Docker, Heroku

🎓 Skills Developed

  • OCR Development: Fine-tuning Tesseract/PaddleOCR
  • CNN Architecture: Custom model design
  • Image Processing: Otsu, filtering, augmentation
  • Team Collaboration: 5-person agile team
  • Cultural Awareness: Amazigh heritage preservation

📚 Key References

  1. Tesseract OCR - Google Open Source
  2. PaddleOCR - PaddlePaddle Team
  3. AMHCD Dataset - IRCAM & Ibn Zohr University
  4. Otsu Thresholding - IEEE Trans. SMC (1979)

📧 Contact

Team Lead: Rima Daqch
Technical Lead: El Mehdi Hicham
Email: mehdihicham736@gmail.com
LinkedIn: linkedin.com/in/elmehdihicham
GitHub: github.com/MehdiHCH

Supervisor: Prof. Khadija Lekdioui
Institution: Faculty of Sciences, Ibn Tofail University

🏆 Achievements

✅ 94.2% Accuracy on character recognition
✅ 91.2% Accuracy on sentence recognition (hybrid)
✅ First Tifinagh OCR open-source in Morocco
✅ Institutional collaboration with IRCAM
✅ Intuitive interface tested by 50+ users

“Every forgotten manuscript is a voice of humanity silenced.”
ⵛⴰⵇⴰ ⵎⴰⵏⵓⵙⵖⵔⵉⵜ ⵓⴱⵍⵉⵎ ⴻⵙⵜ ⵓⵏⴻ ⵓⵡⵏⴰⵙ ⴷⴻ ⵍⵓⵎⵍⵉⵏⵏⵉⵏ ⵔⴻⵙⵓⵉⵜⴰ ⵓⵢ ⵙⵉⵍⵓⵏⵉⵛ

Azul OCR - AI in service of Amazigh cultural heritage