Dike logoDocs

Search in legal corpora cannot rely on standard single-vector semantic search. A robust legal search engine must satisfy two opposing requirements simultaneously:

  1. Conceptual Generalization: Understanding legal semantics and synonyms across languages and phrasing (e.g. إنهاء العقد, فسخ العقد, انفساخ العقد).
  2. Lexical Precision: Exact-match retrieval for specific statutory identifiers, decree numbers, and legal keywords (e.g. المادة 157, القانون رقم 131).

Dike combines both modalities into a unified Multi-Stage Hybrid Pipeline.

Rendering diagram...

1. Dual-Vector Retrieval

Every incoming query is mapped into complementary dense and sparse representations:

  • Dense Semantic Representation: Captures high-level legal concepts, contextual intent, and cross-lingual matches (Arabic to English and back).
  • Sparse Lexical Representation: Captures exact statutory numbers, legal terms, and court citations without loss of precision.

2. Reciprocal Rank Fusion (RRF)

Dense and sparse candidate lists are fused server-side using Reciprocal Rank Fusion (RRF):

RRF_Score(d) = 1 / (60 + rank_dense(d)) + 1 / (60 + rank_sparse(d))

Benefits of Hybrid Fusion

  • Zero Client Overhead: Combining candidate sets server-side reduces network payloads and guarantees optimal candidate diversity.
  • Automatic Balancing: If a query is purely conceptual with no sparse overlap, the dense component dominates; if a query contains exact statute numbers, the sparse component prioritizes exact matches.
  • Payload Pre-filtering: Filters for jurisdiction, doc_type, year, and status are evaluated directly during index lookup before ranking.

3. Cross-Encoder Reranking

While bi-encoders score queries and documents independently for fast index retrieval, the cross-encoder processes the query and top candidate chunks with full cross-attention:

Score(query, doc) = CrossEncoder([CLS] + query + [SEP] + doc)

Performance Characteristics

StageCandidates EvaluatedLatency AddedAccuracy Metric (NDCG@10)
Dense OnlyEntire Corpus~25ms0.72
Hybrid RRFEntire Corpus~45ms0.84
Hybrid + Cross-Encoder RerankTop 4 × top_k~110ms0.94

4. Response Field: fusion

Every search response includes a diagnostic fusion property:

  • "rrf": Both dense and sparse index passes contributed to the final score ranking.
  • "dense": Lexical sparse vectors produced zero candidate matches, so ranking fell back to dense semantic embeddings alone.

Next Steps

On this page