Ingestion errors and the collapse of retrieval accuracy
85% of enterprise data is unstructured in 2026. Text-only RAG pipelines fail on scanned pages, where recall@5 drops to 0.03 from a 0.91 baseline on text-native pages. Ingestion errors in the document pipeline destroy retrieval signals before a query ever hits the vector database. When a text-only RAG pipeline ingests a document containing a complex table or a schematic, the parser either skips the visual content or converts it to garbled character strings that the embedding model cannot work with.
Multi-column layouts in PDFs cause the parser to interleave sentences from adjacent columns. This creates semantically incoherent chunks that embed poorly and retrieve inconsistently. Tables also lose relational context when a naive extractor produces a flat string of cell values. A query about quarterly revenue fails if the chunk lacks a coherent statement of the data. You already know that a vector database stores embeddings in high-dimensional space.
An audit of a production corpus reveals the scale of this problem. If more than 15% of the corpus falls outside the born-digital text category, a text-only pipeline has a coverage problem that tuning prompts will not fix. The industry-wide RAG observability blind spot is a significant failure for most organizations. Engineers who only test retrieval on text-native samples during development miss this gap until a production question hits a document the parser cannot read.
Modality weighting failures in multimodal indexing
Most teams move through document processing approaches in sequence and discover failures only after a production error occurs. They often start with caption-and-index methods after realizing text parsers miss visual data. This method fails when a figure is a 40-row table or a chart requiring specific data point reading. Captions describe the document but do not encode the retrievable content. Unified vision embeddings or page-as-image approaches solve this but require infrastructure decisions many teams skip.
The assumption that an embedding model handles all retrieval heavy lifting is incorrect. Published benchmarks show that optimal retrieval uses a specific blend: 30% text, 25% caption, 30% OCR, and 15% image content. This combination produces a 57.3% improvement over text-only baselines. Using equal weights across all modalities produces significantly worse results.
| Gemini 1.5 Pro Capability | Metric |
|---|---|
| Maximum Context Window | 10 million tokens |
| Recall at 1 million tokens | Over 99.7% |
| Recall at 10 million tokens | 99.2% |
| Inference Cost per 1M tokens | $1.50 |
Teams must calibrate modality weights per document class rather than per pipeline. Financial statements need higher OCR and structured extraction weights to capture table data. Technical specifications require higher image embedding weights for labeled diagrams. Most teams skip this calibration because they lack labeled evaluation data. A practical starting point involves running 100 to 200 production queries through both text-only and multimodal pipelines to measure answer quality.
The limits of long context and the cost of scale
Gemini 1.5 Pro provides a 10 million token context window, but this does not negate the need for RAG. Large context windows present challenges in velocity, value, volume, and variety. Retrieval in massive contexts faces a 30-second delay for 360,000 tokens. High inference costs also undermine the value proposition of using massive contexts for every query.
LLMs exhibit a U-shaped performance curve regarding context usage. Accuracy is highest for information at the beginning and end of the window. Accuracy is lowest for information in the middle of the context window. This effect is strongest when inputs occupy up to 50% of the model’s context window.
Retrieving 50,000 tokens to provide context for an LLM that can only handle 4,000 tokens leads to errors. Summarization layers used to bridge this gap introduce new errors. In one healthcare case study, a RAG system retrieved 50,000 tokens for an ML model that only processed 4,000. The resulting summarization layer introduced errors that led to 34% of predictions being incorrect. How can a pipeline maintain accuracy when the document distribution shifts in production?
