Amazon Web Services Introduces Query-Aware Compression to Reduce RAG Costs
AWS has detailed a post-retrieval customization pattern for Amazon Bedrock that uses a smaller model to filter retrieved context, potentially lowering RAG costs and reducing hallucination risks.
Original source published: August 21, 2026
Amazon Web Services (AWS) has published a guide on using query-aware compression to optimize Retrieval Augmented Generation (RAG) workflows on Amazon Bedrock. By implementing a post-retrieval step, developers can use a smaller, lower-cost model to filter retrieved chunks for relevance before sending them to a primary model for final answer generation. This process aims to reduce the number of input tokens processed by the primary model, which can lead to cost savings while maintaining answer quality.
The architecture involves an AWS Lambda function that orchestrates two model calls via the Amazon Bedrock Converse API. The smaller model extracts verbatim spans relevant to the user's query, which are then provided as context to the primary model. According to AWS, this approach may also decrease the surface area for hallucinations. The pattern is compatible with existing Amazon Bedrock features, including Knowledge Bases, and can be combined with prompt caching or reranking to further optimize performance and costs.