RAG Chunking Strategy
By Ai Flowi · Published
A RAG chunking strategy decides how source text is divided into retrievable passages while retaining the context needed to answer questions. Choose boundaries around meaningful sections, then test whether retrieval returns the rule and its necessary conditions together. There is no universal chunk size or overlap that suits every document and question.
This guide proposes a comparison of chunk previews and retrieved evidence in AI Flowi Workflow. Splitter components are documented in the catalogue; not production-tested by Ai Flowi. Knowledge Base retrieval is documented; not production-tested by Ai Flowi for this comparison. The example settings are trial inputs, with no measured retrieval improvement or optimal size claimed.
1. Prepare text with deliberate boundary challenges
Use synthetic documents containing a section heading, a main rule, an exception, and a table whose cells depend on row and column labels. Add a second section with similar vocabulary but a different subject. These features reveal whether splitting separates facts that need to be interpreted together or creates misleading fragments.
Synthetic document outline, not an ingested result:
Section: Equipment returns
Rule: Return borrowed equipment at the end of a placement.
Exception: Keep it when an approved extension remains active.
Table: Item category | inspection requirement
Section: Visitor passes
Rule: Return the pass at the end of a visit.Write questions that require the rule plus the extension exception, and questions that require a table value plus its category. Label the source sections required for each answer. A preview can look readable while still splitting an exception away from the only passage a short query retrieves.
Related: RAG Reranking.
2. Choose the splitter contract you can inspect
Consult the Split Text settings and output reference before running the boundary comparison below.
Recursive Character Text Splitter describes its purpose as Split text trying to keep all related text together.
It accepts Document, Data or JSON in data_input and emits data as JSON. Its separators field has no exported list-encoding instructions, so a boundary sequence cannot be inferred from the field name. Check that syntax in the editor before comparing its behavior with Split Text.
The two splitters differ in accepted types and output types. A raw Message fits Split Text.data_inputs, but Message is not listed for Recursive Character Text Splitter.data_input. Choose a source representation each accepts; do not assume that similarly named size controls make the components interchangeable.
3. Change one boundary choice at a time
Chunk-boundary comparison: example configuration, not a recorded test
| Component | Catalogue inputs to set | Inspection before continuing |
|---|---|---|
| Split Text, trial A | data_inputs: synthetic source; chunk_size: 600; chunk_overlap: 0; separator: a newline; keep_separator: "End". | Inspect whether headings and exceptions remain interpretable in the Chunks table. |
| Split Text, trial B | Use the same source and chunk_size; change chunk_overlap to 80. | Compare repeated text and any newly preserved rule/exception pair. |
| Recursive Character Text Splitter | data_input: editor-accepted structured source; chunk_size: 600; chunk_overlap: 80; separators: editor-verified boundary sequence. | Inspect the JSON output without assuming it matches Split Text row layout. |
| Knowledge Base | knowledge_base: separately prepared trial base; search_query: labelled question; top_k: fixed across variants; include_metadata: true. | Keep corpus versions distinguishable when comparing Results. |
The values 600 and 80 are illustrative integers. The component export does not establish their size units, supported bounds or overlap behavior. Do not label them as tokens. Keep the source identical between trials and inspect whether cleaning removes separators or changes meaningful table text before making another adjustment.
4. Review each chunk as a standalone passage
For every chunk, ask whether a reader can identify its subject without seeing the previous chunk. Check that a table value retains its row and column meaning, that a bullet’s condition remains attached to the rule, and that headings have not become isolated fragments. Inspect the actual output order and metadata; the catalogue supplies no chunk identifier or source-offset schema.
Overlap can duplicate useful context, but duplicated wording can also consume the evidence space without adding a missing fact. Keep a record of which passage contains the main rule and which contains the exception. The goal is not the largest or smallest chunk count. It is the ability to retrieve enough attributable evidence for the questions you expect.
The export supplies no cleaning algorithm for clean_output, leaving whitespace and table-boundary preservation to inspection. Keep original documents for comparison. When text is lost in parsing before the splitter, changing chunk size cannot recover it.
5. Distinguish preview splitting from indexed chunks
The knowledge-base guide already covers ingestion settings, preview review and stored-chunk inspection. Follow it to prepare separate trial bases with the same source documents. A direct connection from Split Text output into built-in Knowledge Base ingestion remains unverified, as does algorithm equivalence between those paths.
Record the settings used for each indexed variant and examine the chunks actually stored. Read the component preview to diagnose boundaries; inspect stored chunks separately to learn what the ingestion path indexed. If you cannot control both paths consistently, compare indexed variants directly and describe component previews as a separate experiment.
Keep provider and model choices unchanged while testing document boundaries. References: base preparation · Language Model reference
6. Test paired facts rather than a covered keyword
| Boundary under inspection | Comparison to record | Failure to locate |
|---|---|---|
| Heading followed by its first sentence | Locate both in the zero-overlap and overlap variants. | The subject heading is detached from the rule it describes. |
| Exception at the next boundary | Identify which chunks hold the rule and qualification in each variant. | Extra overlap repeats the rule but still excludes the qualification. |
| Table header above a value row | Read the value chunk without consulting adjacent chunks. | The category survives but the column meaning disappears. |
| End of equipment section, start of visitor section | Mark the point where the subject changes in each preview. | A fragment joins two rules without their separate headings. |
For the equipment outline, compare the smallest chunk carrying the extension condition with the chunk carrying the return instruction. If overlap copies only the instruction, increasing it has not solved the missing qualification. If both facts survive but the subject heading disappears, the retrieved fragment may still be ambiguous. Record those as different boundary failures; their remedies may be moving a section boundary or retaining a heading rather than increasing overlap again.
Submit each labelled question to Knowledge Base.search_query with a constant top_k. Compare required evidence returned, metadata preserved and information actually supplied to any answer prompt. Have RAG evaluation check answer grounding, because a fluent guess about the extension could mask a lost exception.
7. How do I keep the chunking strategy specific to the corpus?
Do not use chunking changes to compensate for missing source documents, permissions or ambiguous questions. A retrieval failure may come from the query or embedding configuration rather than the boundary. Freeze those choices when isolating chunk behavior, then revisit them only when evidence points to that cause.
There is no recorded universal overlap ratio, chunk length or retrieval success rate for these components. Tables and documents with layout-dependent meaning may need additional source preparation. The catalogue does not prove table reconstruction or source metadata preservation. Keep a previous indexed variant until a replacement has passed the question set.
Chunking experiment usage
Plans and workflow capacity. Go: 5,000 Flowi Tokens; Plus: 25,000; Pro 100: 50,000; Pro 200: 100,000; Pro 500: 250,000 per month. Choose the capacity that fits your workflow, connected tools and review process. Book a demo
Separate ingestion and re-indexing work from retrieval trials.