“RAG performance can scale almost linearly w.r.t. log inference FLOPs” Demonstration Based RAG (DRAG) Method Adding demonstrations as k in-context examples. Prompt: documents, input query, final answer. Parameters: number of documents, number of in context samples, number of iterations upper bound. Iterative Demonstration Based RAG (IterDRAG) Method DRAG above, and then the model can generate a new sub-query. The model decides Parameters: number of documents, number of in context samples, number of iterations upper bound.