Search in legal corpora cannot rely on standard single-vector semantic search. A robust legal search engine must satisfy two opposing requirements simultaneously:
- Conceptual Generalization: Understanding legal semantics and synonyms across languages and phrasing (e.g. إنهاء العقد, فسخ العقد, انفساخ العقد).
- Lexical Precision: Exact-match retrieval for specific statutory identifiers, decree numbers, and legal keywords (e.g. المادة 157, القانون رقم 131).
Dike combines both modalities into a unified Multi-Stage Hybrid Pipeline.
Rendering diagram...
1. Dual-Vector Retrieval
Every incoming query is mapped into complementary dense and sparse representations:
- Dense Semantic Representation: Captures high-level legal concepts, contextual intent, and cross-lingual matches (Arabic to English and back).
- Sparse Lexical Representation: Captures exact statutory numbers, legal terms, and court citations without loss of precision.
2. Reciprocal Rank Fusion (RRF)
Dense and sparse candidate lists are fused server-side using Reciprocal Rank Fusion (RRF):
RRF_Score(d) = 1 / (60 + rank_dense(d)) + 1 / (60 + rank_sparse(d))Benefits of Hybrid Fusion
- Zero Client Overhead: Combining candidate sets server-side reduces network payloads and guarantees optimal candidate diversity.
- Automatic Balancing: If a query is purely conceptual with no sparse overlap, the dense component dominates; if a query contains exact statute numbers, the sparse component prioritizes exact matches.
- Payload Pre-filtering: Filters for
jurisdiction,doc_type,year, andstatusare evaluated directly during index lookup before ranking.
3. Cross-Encoder Reranking
While bi-encoders score queries and documents independently for fast index retrieval, the cross-encoder processes the query and top candidate chunks with full cross-attention:
Score(query, doc) = CrossEncoder([CLS] + query + [SEP] + doc)Performance Characteristics
| Stage | Candidates Evaluated | Latency Added | Accuracy Metric (NDCG@10) |
|---|---|---|---|
| Dense Only | Entire Corpus | ~25ms | 0.72 |
| Hybrid RRF | Entire Corpus | ~45ms | 0.84 |
| Hybrid + Cross-Encoder Rerank | Top 4 × top_k | ~110ms | 0.94 |
4. Response Field: fusion
Every search response includes a diagnostic fusion property:
"rrf": Both dense and sparse index passes contributed to the final score ranking."dense": Lexical sparse vectors produced zero candidate matches, so ranking fell back to dense semantic embeddings alone.
Next Steps
- Review the full parameter options in
POST /v1/searchAPI Reference. - Learn how to tune search parameters in Hybrid Search Tuning Guide.