Agentic RAG
By Ai Flowi · Published
Agentic RAG is retrieval-augmented generation in which an agent decides when to retrieve evidence and how to use it to answer a task. A fixed RAG workflow retrieves on a predefined path; an agentic design makes retrieval available as a tool. Start with fixed retrieval, then consider an agent when questions require different searches or clarification.
This guide proposes a restricted retrieval tool in AI Flowi Workflow and a comparison procedure. The component contracts are documented in the catalogue; not production-tested by Ai Flowi. The combined retrieval-tool and answer path has no recorded runtime result here.
1. Define what the agent may decide
Give Agent a narrow task: answer questions from one approved knowledge base, identify supporting sources and stop when the evidence is missing. Keep document ingestion, permissions and updates outside that task. A retrieved document is evidence to examine, not permission to run another tool or change the workflow. Put this distinction in system_prompt and test it with a document that contains instruction-like text.
Use the existing knowledge-base setup guide for creating and inspecting the base. This design begins after your corpus is ready. Write down which document versions are eligible and which questions require clarification. If every request needs the same single search, the knowledge-base chatbot design provides a simpler comparison path.
2. Prepare accounts and a retrieval-only flow
You need access to the builder, a populated knowledge base, the credentials required by its embedding provider, and a configured language-model provider. Use your own provider account and API key where required; the provider bills usage separately. Follow model configuration for credentials instead of placing secrets in the tool description or prompt.
Build the search portion from the documented Knowledge Base retrieval path, then verify the loaded flow’s query and return mapping before making it a tool.
The distinct design choice is how much query reformulation the agent may perform. Consider a synthetic question about keeping equipment during an extended placement. A useful search would carry both equipment and extension; replacing it with a broad question about returns may omit the condition that motivated the request. Save the original request beside the proposed search wording and check that the subject and condition survive. Reformulation should remove conversational ambiguity without deciding the policy answer in advance.
3. Describe the tool and its boundaries
Restricted retrieval settings: example configuration, not a recorded test
| Component | Catalogue inputs to set | Inspection before continuing |
|---|---|---|
| Knowledge Base | knowledge_base: your approved corpus; top_k: 4; include_metadata: true. | Inspect Results for passage text and source identifiers; four is an illustrative candidate count. |
| Flow as Tool | flow_name: select the saved retrieval flow; tool_name: policy_lookup; tool_description: search approved policy passages for one question; return_direct: false. | Check the query argument mapping and actual return shape; neither is documented by the exported inputs. |
| Agent | model: configured LanguageModel connection; tools: retrieval Tool; max_iterations: 3; system_prompt: evidence policy below. | The iteration value is a trial setting, not a measured call budget or verified termination guarantee. |
Use the tool-type boundary documented in the Agent input reference when connecting the saved retrieval tool.
Compare the two possible mistakes at that boundary: a tool description can be too broad to guide query choice, or so narrow that the agent never selects it for a paraphrase. Keep the same saved flow while changing only its description. Inspect whether the selected query becomes more specific, less specific or unchanged; a new description that merely produces a longer answer has not demonstrated a useful retrieval decision.
4. Separate retrieval instructions from answer instructions
Use the answer-grounding contract in knowledge-base chatbot after retrieval; add the following query-choice instructions for this agentic trial.
Before search, identify the requested subject and condition.
Ask the user to resolve an unstated subject before building a query.
Carry the subject and condition into the retrieval argument.
Do not rewrite a question into an assumed policy answer.
If another search is proposed, state which missing fact it seeks.Set that text in Agent.system_prompt. Make tool_description describe the search scope and returned evidence, not a broad promise to solve any question. A useful description distinguishes policy lookup from unrelated actions. Avoid connecting writing or external messaging tools to this experiment: they add permissions that the retrieval question does not require.
Use AI agent evaluation for capturing tool observations and marking unobserved behavior.
For this task, the decisive observation is the actual search query received by the retrieval flow. A well-written final answer does not reveal whether the extension condition reached that query. Keep the loaded flow’s received argument beside the original request; this also separates an agent’s poor reformulation from a correct reformulation lost during tool binding.
5. Compare with fixed retrieval on identical questions
Use the same corpus, model, evidence instructions and question set for both designs. In the fixed path, submit the question directly to Knowledge Base.search_query. In the agentic path, submit it to Agent.input_value. Save the actual retrieved passages and final answer for each attempt so a change in retrieval is distinguishable from a change in generation.
| Query trial | Agent decision to inspect | Fixed-path comparison |
|---|---|---|
| Equipment question with an extension condition | Does reformulation retain both the item and the condition? | Compare the passages with those returned for the original wording. |
| Pronoun without a stated subject | Does clarification happen before a search argument is constructed? | Do not compare an unresolved pronoun with a fully specified agent query. |
| Long question containing incidental details | Which details are removed, and do any change applicability? | Keep the original full query as a baseline. |
| Search produces no usable passage | Does any additional query change its meaning rather than repeat it? | Record duplicate searches separately from new evidence. |
Record whether retrieval happened, whether its query addressed the task, and whether the agent stopped appropriately. A polished response can hide an irrelevant search. Use RAG evaluation to label passage coverage and answer grounding separately; do not declare the agentic variant preferable merely because it performed more searches.
6. When is agentic retrieval unnecessary?
Keep general tool-permission and stopping checks in AI agent evaluation alongside this retrieval comparison.
Choose the fixed path when the agent repeatedly constructs the same query you already have. Choose clarification before either path when the user’s subject is unresolved. Consider tool-selected retrieval only when the saved observations show a useful difference in query choice for the same task. This decision is specific to your questions; extra searches alone do not justify another model decision.
The catalogue does not define Flow as Tool argument binding, Agent iteration-limit behavior, automatic retries or failure recovery. Inspect the Agent reference for those field boundaries. If you cannot observe tool selection and returned evidence in your environment, keep the fixed retrieval path until that uncertainty is resolved. Repeated model and retrieval calls also add work; measure their usage before selecting a design.
Usage planning
Plans: Go from USD8/month billed yearly (USD10 monthly), Plus from USD16 (USD20 monthly), Pro 100 from USD80 (USD100 monthly), Pro 200 from USD160 (USD200 monthly) and Pro 500 from USD400 (USD500 monthly), with 5,000 to 250,000 Flowi Tokens/month and top-ups; 1 token per node run; third-party API and model usage is not included.
Count the nodes actually executed during each search and answer attempt. A configured iteration setting is not a billing estimate, and model tokens are a separate unit from workflow node runs.