RAG for Arabic Knowledge Bases: Architecture and Pitfalls

How to build retrieval-augmented generation systems that actually work on Arabic enterprise content — embeddings, chunking, hybrid search, and evaluation.

Key takeaways

• Arabic morphology breaks naive chunking — use morphology-aware tokenization.
• Multilingual embedding models need Arabic-specific fine-tuning for enterprise domains.
• Hybrid search (dense + sparse) handles Arabic's rich morphology better than dense-only retrieval.
• Evaluate retrieval quality separately from generation quality — they fail for different reasons.

Why RAG on Arabic content is harder than English

Embedding models for Arabic enterprise content

General-purpose multilingual embeddings (e.g., from OpenAI or Cohere) handle Arabic at a basic level but underperform on domain-specific terminology — legal Arabic, medical Arabic, or financial reporting in KSA. SkyStack fine-tunes embedding models on client corpora using contrastive learning, which significantly improves retrieval relevance for enterprise queries.

Arabic-aware chunking strategies

Naive fixed-size chunking (e.g., 512 tokens) breaks Arabic sentences mid-word and splits semantic units. Use sentence-boundary detection tuned for Arabic punctuation patterns, then merge sentences into chunks that respect paragraph and section boundaries. For structured documents (policies, regulations), use heading-based chunking that preserves hierarchy.

Also handle mixed-content documents common in Saudi enterprises: Arabic body text with English headers, tables with bilingual data, and embedded dates in both Hijri and Gregorian calendars. SkyStack's preprocessing pipeline normalizes these before chunking.

• Use Arabic sentence tokenizers (not generic regex splitting).
• Preserve document metadata (section, page, source) in chunk records.
• Normalize Arabic text: remove tatweel, standardize hamza forms, handle kashida.

Hybrid search: combining dense and sparse retrieval

Dense vector search captures semantic similarity but struggles with exact Arabic terminology — a specific regulation number, a product code, or a person's name. Sparse retrieval (BM25 with Arabic stemming) excels at exact matches but misses paraphrases. Combining both with reciprocal rank fusion consistently outperforms either approach alone on Arabic enterprise benchmarks.

SkyStack deploys hybrid search using a vector database (Qdrant or Pinecone) for dense retrieval alongside Elasticsearch with Arabic analyzers for sparse retrieval. The fusion layer weights results based on query type: keyword-heavy queries lean sparse, conceptual queries lean dense.

Evaluation: measuring what matters

Separate retrieval evaluation (are the right chunks returned?) from generation evaluation (is the answer accurate and grounded?). For retrieval, measure recall@k and mean reciprocal rank on a labeled test set of Arabic queries. For generation, evaluate faithfulness (does the answer contradict the source?), relevance, and completeness.

Build a golden test set of 50–100 query-answer pairs covering your domain. Include edge cases: mixed-language queries, ambiguous terms, and questions that span multiple documents. Update this test set quarterly as content evolves. Without rigorous evaluation, RAG quality degrades silently.

Frequently asked questions

Why is Arabic RAG harder than English?

Tokenization, morphology, and chunk boundaries differ. Naive English chunking breaks Arabic meaning.

What data should we index first?

High-value internal docs with clear owners: policies, SOPs, product FAQs, and approved knowledge articles.

How do we prevent hallucinations?

Cite retrieved passages, refuse when retrieval is weak, evaluate on Arabic golden sets, and keep humans in the loop for high risk.

SkyStack — Riyadh, Saudi Arabia. AI, ERP, CRM, custom software, and mobile apps for Vision 2030 and the GCC. https://www.skystack.sa/blog/rag-arabic-knowledge-bases-guide