For now, I searched around and tried to organize what I found:
Yes — the general idea looks implementable, and there is prior work fairly close to it.
The closest term I found is a query–document graph or query–document relevance graph: query nodes and document/chunk nodes form a weighted bipartite graph, and the edges represent estimated relevance.
The closest direct example is the ECIR 2024 paper Effective Adhoc Retrieval through Traversal of a Query-Document Graph. It connects generated query nodes to document nodes using neural relevance estimates, then traverses that graph to find documents that the first-stage retriever missed.
The main difference is that its query nodes are primarily generated from the corpus in advance. Your proposal seems to add live user queries as persistent nodes and reuse their relationships later. I would treat that persistent online-memory part as a second, separate experiment.
A useful way to split the pipeline
first-stage retrieval
↓
reranking
↓
optional graph-based candidate expansion
↓
reranking/fusion against the current query
↓
optional persistent-memory update
A reranker normally operates on candidates produced by an earlier retriever rather than searching the complete corpus efficiently by itself. The standard pattern is described in the Sentence Transformers Retrieve & Re-Rank documentation, and LightRAG currently supports a reranking-function stage.
I also would not treat contextual retrieval and reranking as alternatives. Contextualization changes what the initial retriever sees; reranking evaluates the resulting candidates more precisely. Anthropic’s Contextual Retrieval combines contextualized chunks, hybrid retrieval, and reranking in the same pipeline.
The default route I would test
| Variant |
What it isolates |
| A. Normal retrieval |
Baseline candidate recall and ranking |
| B. A + strong reranker |
Improvement caused by reranking alone |
| C. B + temporary graph expansion |
Whether graph traversal finds useful chunks missed by the first stage |
| D. C + persistent query memory |
Whether earlier query relationships improve later unseen queries |
| E. D + observed feedback |
Whether verified usefulness adds value beyond model predictions |
A conservative implementation of C could be:
1. Retrieve an initial set of chunks.
2. Rerank them against the current query.
3. Use the strongest chunks as graph seeds.
4. Collect a bounded number of previously unseen neighbors.
5. Rerank every added candidate against the current query.
6. Generate the answer from the final evidence set.
7. Separately decide whether anything from the run should be persisted.
This can be tested without permanently inserting the current query into the graph. Neo4j’s VectorCypherRetriever, for example, performs vector retrieval and then traverses the graph from the retrieved nodes. PyTerrier Adaptive Retrieval also provides an implementation of graph-based adaptive reranking within a bounded candidate process.
If C does not improve over B, persistent query nodes may not yet be necessary. If C helps, comparing C with D isolates the more distinctive part of the proposal:
Does retaining previous query relationships provide value beyond performing a temporary graph traversal for the current request?
The deciding factor is therefore not only how to write a query node, but how a later query reads it.
One possible read path is:
current query
↓ similarity search
previous QueryIntent
↓ verified relevance edges
candidate chunks
↓ reranking against the current query
final evidence
Without a defined read path, persistent query nodes may initially behave more like a retrieval log than an additional retrieval mechanism.
One other distinction seems important from the beginning:
the reranker predicted that a chunk was relevant
≠
the chunk was actually useful evidence
I would keep predicted relevance separate from observed usefulness, at least in the first prototype.
Nearby approaches and how they differ
1. Query–Document Graph: the closest structural match
Effective Adhoc Retrieval through Traversal of a Query-Document Graph starts from a limitation of normal two-stage retrieval:
- the first-stage retriever produces a candidate set;
- the reranker can improve the order within that set;
- a relevant document omitted from the candidate set cannot be recovered by reranking alone.
The paper builds a weighted bipartite graph:
generated query nodes
↕ estimated relevance
document nodes
During adaptive reranking, strong documents lead to associated query nodes, which can lead to other candidate documents.
This provides a concrete graph read path:
initial document
→ generated query
→ unseen related document
The distinction from the proposed online design is mainly lifecycle and provenance:
| Query–Document Graph paper |
Persistent live-query version |
| Queries are mainly generated from documents |
Queries originate from real requests |
| Graph is largely prepared offline |
Graph grows while the system is used |
| Primary goal is candidate recall |
Possible goals include recall, reuse, adaptation, or personalization |
| Edge estimates are evaluated within a retrieval method |
Edge validity may need long-term maintenance |
A search-friendly way to connect the proposal to existing terminology, without replacing the original wording, might be:
Depending on what the edges are intended to represent, this could also be viewed as an online query–document relevance graph layered alongside the knowledge graph.
2. Query-centric corpus indexing
Query-Centric Graph Retrieval Augmented Generation, or QCG-RAG, is another close direction.
It addresses the space between:
- very fine-grained entity graphs, which can lose surrounding context;
- coarse document graphs, which may miss nuanced relationships.
It generates query-oriented representations from chunks and uses a query-centric graph for multi-hop chunk retrieval.
This is relevant when the goal is:
- representing knowledge at the level of answerable questions;
- connecting chunks through questions they can answer;
- retaining more context than an entity label;
- reducing dependence on a rigid entity/relation schema.
It is still different from preserving actual user-query history. It is better understood as a query-granular corpus index.
That suggests distinguishing two node types:
SyntheticQuery
- generated from corpus content
- part of the search index
QueryEvent
- submitted during system use
- part of retrieval history
They can coexist, but their update rules, privacy constraints, and evaluation criteria are different.
3. Adaptive reranking without persistent query nodes
The broader idea of adding candidates during reranking is implemented in PyTerrier Adaptive Retrieval.
A simplified form is:
initial candidates
↓
rerank strongest documents
↓
add graph neighbors
↓
rerank the newly introduced candidates
The graph contains document-to-document connections rather than persistent query nodes, but it is a strong baseline for the candidate-recall part of the proposal.
If a document graph already recovers the missing evidence, live query nodes may not be needed for the first version.
4. Temporary query-specific reasoning structures
A different route is to construct a graph for the current question only.
LogicRAG, with a public implementation repository, decomposes a query into subproblems and constructs a dependency DAG at inference time:
question
↓
subquestions
↓
query-specific dependency DAG
↓
ordered retrieval
This may be closer when the underlying problem is organizing multi-step retrieval rather than learning from previous query histories.
HARR explores another temporary form of memory: it makes multi-step retrieval aware of what has already been retrieved during the current request.
These approaches distinguish several kinds of “query memory”:
| State |
Lifetime |
| Query-specific reasoning graph |
One request |
| Retrieval history |
One multi-step search |
| Persistent query/document memory |
Multiple future requests |
| Synthetic query index |
Until corpus reindexing |
The shorter-lived versions are useful controls before adding permanent storage.
5. Remembering successful paths instead of raw queries
ReMindRAG, with a public repository, stores graph-traversal experience in knowledge-graph edge embeddings and reuses it for similar later questions.
Its memory object is closer to:
past query
↓
successful or unsuccessful traversal
↓
memory associated with graph paths
↓
similar future query reuses promising routes
This may fit better if the intended benefit is:
Similar future questions should not need to rediscover the same multi-hop path.
In that case, a validated retrieval path may be more useful than a raw query node connected to every top-k reranker result.
6. Distilling useful query signals into document-side memory
RAG without Forgetting: Continual Query-Infused Key Memory proposes Evolving Retrieval Memory.
Instead of retaining each query as a graph node, it transfers useful query-time signals into document retrieval keys. Its updates are correctness-gated, selectively attributed, and bounded to reduce noise accumulation.
This gives another branch:
raw query history
versus
validated query signal distilled into document memory
Explicit query nodes are easier to inspect and audit. Distilled document-side memory may be cheaper to retrieve from and less prone to unbounded query-node growth.
These are different engineering trade-offs rather than mutually exclusive definitions of the same system.
7. Graph retrieval does not need to run for every query
Graph retrieval may be most useful for relational, multi-hop, or fragmented-evidence questions.
A possible router is:
simple factual query
→ ordinary retrieval + reranking
multi-hop or relational query
→ temporary graph expansion
similar validated history exists
→ persistent-memory branch
no useful graph result
→ ordinary retrieval fallback
This keeps graph traversal and graph writes from becoming mandatory overhead for requests already handled well by a normal retriever.
A possible data model, edge meanings, and memory policies
1. Separate event history from learned memory
A query being submitted does not necessarily mean that its retrieval result should influence future requests.
A useful separation could be:
QueryEvent
- the original request
- timestamp
- language
- user / tenant / session scope
- corpus version
RetrievalRun
- retriever and revision
- reranker and revision
- initial candidates
- graph-added candidates
- scores and ranks
- final evidence
QueryIntent
- optional reusable representation
- groups similar QueryEvents
Chunk / Document
- indexed evidence
- source and version metadata
This supports three different purposes:
| Layer |
Purpose |
QueryEvent |
Audit and reproducibility |
RetrievalRun |
Explain how candidates were produced |
QueryIntent |
Optional reuse across similar requests |
The complete event history can be stored without automatically promoting every result into learned retrieval memory.
2. Raw queries and reusable intents
Raw queries often contain paraphrases and language variants:
How does LightRAG work?
Explain the architecture of LightRAG.
What is the LightRAG retrieval process?
LightRAGの仕組みは？
An optional structure is:
(QueryEvent)-[:INSTANCE_OF]->(QueryIntent)
(QueryIntent)-[:VERIFIED_RELEVANCE]->(Chunk)
Potential benefits:
- fewer duplicate high-degree query nodes;
- reuse across paraphrases;
- a cleaner similarity-search target;
- preservation of original events for provenance.
Potential complications:
- distinct intentions may be merged;
- multilingual equivalence is imperfect;
- intent clusters may change with a new embedding model;
- wording-specific distinctions may disappear.
For that reason, QueryIntent is probably best treated as a derived layer rather than a replacement for the original event.
3. Distinguish different graph layers
Several kinds of edges may coexist:
Semantic knowledge graph
Entity ── explicit relation ── Entity
Document graph
Chunk ── citation / section / similarity ── Chunk
Predicted relevance graph
QueryIntent ── model-estimated relevance ── Chunk
Observed interaction graph
QueryEvent ── used / cited / accepted ── Chunk
They can share a graph database, but their semantics are not interchangeable.
A conventional knowledge-graph edge might encode:
LightRAG ── uses ── entity/relation extraction
A reranker-derived edge means something closer to:
For this query, candidate pool, and model revision,
this chunk received a high relevance rank.
The query-conditioned edge may preserve a fine-grained semantic association without providing the explicit relation semantics of an entity-relation edge. The two layers may therefore complement one another.
4. Use typed edges
Possible event or edge types include:
RETRIEVED
ADDED_BY_GRAPH
PREDICTED_RELEVANCE
USED_AS_EVIDENCE
CITED
CLICKED
USER_ACCEPTED
USER_REJECTED
These signals answer different questions:
| Signal |
What it tells us |
| Retrieved |
The first stage found the chunk |
| Added by graph |
Traversal introduced the chunk |
| Predicted relevance |
A model scored it highly |
| Used as evidence |
The generation stage selected it |
| Cited |
It appeared in the answer’s attribution |
| Accepted/rejected |
An evaluator provided feedback |
Keeping them separate makes it possible to test which signal improves future retrieval.
A generic RELATED_TO edge that mixes all of them would be harder to interpret or invalidate later.
5. Reranker scores need retrieval-run context
At least for Cohere’s models, the official reranking guidance says that rank is the primary output and that scores depend on the query and supplied passages. A score should not automatically be interpreted as a universal semantic distance.
Other rerankers expose logits, normalized scores, or architecture-specific outputs, so a stored relevance edge may need metadata such as:
raw_score
rank_within_run
top_k_membership
reranker model and revision
retriever configuration
candidate-pool or run ID
candidate-pool size
query timestamp
chunk and document version
Not every prototype needs all of these fields. The goal is to preserve enough information to explain, invalidate, or recompute an edge later.
6. Write-policy branches
Store every query as an event
Useful for:
- debugging;
- auditing;
- reproducing retrieval runs;
- analyzing request patterns.
Costs:
- duplicate queries;
- storage growth;
- privacy and retention obligations.
Store only deduplicated query intents
Useful for:
- repeated domain questions;
- reuse across paraphrases;
- a smaller graph.
Costs:
- possible incorrect merging;
- migration after embedding-model changes;
- weaker preservation of wording-specific details.
Store only verified query–chunk relationships
Useful for:
- limiting self-reinforcing model errors;
- constructing a higher-confidence retrieval memory.
Costs:
- requires evaluation or feedback;
- some queries may never receive a reliable verification signal.
Store successful retrieval paths
Useful for:
- repeated multi-hop patterns;
- avoiding repeated graph exploration.
This is closer to the ReMindRAG direction.
Distill validated signals into document memory
Useful for:
- continual adaptation;
- low query-time overhead;
- avoiding a very large query graph.
This is closer to Evolving Retrieval Memory.
A practical combination could be:
store QueryEvents for observability
↓
promote only validated relations or paths
into learned retrieval memory
7. Optional edge lifecycle
One possible implementation pattern is:
PENDING
model prediction only
ACTIVE
allowed in an experimental retrieval branch
VERIFIED
supported by an evaluation or usefulness signal
REJECTED
found to be misleading
EXPIRED
invalidated by time, corpus, permissions, or model changes
This is not a standard specification, but it separates “predicted once” from “safe to reuse.”
8. Read-policy defaults
A conservative read policy might be:
current query
↓
normal retrieval
↓
find a small number of similar QueryIntent nodes
↓
collect VERIFIED neighbors only
↓
deduplicate with normal candidates
↓
rerank everything against the current query
↓
apply the normal final-context budget
Important limits include:
max_similar_query_nodes
max_hops
max_neighbors_per_node
max_total_graph_candidates
allowed edge types
minimum edge state/confidence
allowed corpus versions
maximum memory age
The graph can serve as an additional candidate generator while the current-query reranker remains the final relevance check.
Evaluation, controls, and a small sanity check
1. Measure each stage separately
A final answer score alone may not reveal why the system improved.
Candidate retrieval
Measure:
- Recall@k before graph expansion;
- Recall@k after graph expansion;
- relevant chunks introduced only through the graph;
- irrelevant candidates added by traversal.
Ranking
Measure:
- MRR;
- nDCG@k;
- MAP or another domain-appropriate metric;
- ranking before and after reranking.
Sentence Transformers provides CrossEncoder reranking evaluators for measuring ranking behavior.
Answer evidence
Measure:
- answer accuracy or task-specific correctness;
- evidence precision;
- unsupported-context rate;
- contradictory-context rate;
- citation coverage where applicable.
Persistent-memory value
Measure:
- performance on future unseen queries;
- performance on new paraphrases;
- transfer to related but non-identical questions;
- degradation on unrelated questions;
- the result when memory retrieval is disabled.
System cost
Measure:
- total reranker pairs;
- graph reads and writes;
- end-to-end latency;
- node and edge growth;
- duplicate-query rate;
- average and tail fanout;
- storage size;
- fallback frequency.
2. Suggested ablation
A. first-stage retrieval only
B. A + reranker
C. B + transient graph expansion
D. C + persistent query or intent memory
E. D + verified feedback edges
Useful controls:
F. D with persistent-memory reads disabled
G. D with query–chunk edges shuffled
H. document-similarity graph under the same budget
I. semantic or retrieval-cache baseline
The comparisons isolate different mechanisms:
| Comparison |
Interpretation |
| A vs B |
Value of reranking |
| B vs C |
Value of temporary graph expansion |
| C vs D |
Additional value of persistent memory |
| D vs F |
Whether reading stored memory causes the gain |
| D vs G |
Whether the learned graph structure matters |
| D vs I |
Whether a graph beats simple previous-result reuse |
3. Match the retrieval budget
A graph system may look better simply because it evaluates more candidates.
At least one experiment should keep these approximately equal:
same first-stage top-k
same total reranker-pair budget
same maximum graph-added candidates
same final context-token budget
same generator model
same generation prompt
It can also be useful to report both:
- equal-cost performance;
- best unconstrained performance.
4. Use a temporal split for persistent memory
A random query split may leak close paraphrases into the memory graph.
A safer setup is:
memory-construction period
↓
fixed time cutoff
↓
evaluation on later queries
This tests the actual continual-memory claim:
Does earlier retrieval experience improve later requests?
5. Small controlled fixture
Before a larger benchmark, a tiny test can catch implementation errors:
Query Q
Chunk A:
clearly relevant and found by normal retrieval
Chunk B:
partially relevant
Chunk C:
lexically similar but irrelevant
Chunk D:
relevant but missed by normal retrieval;
it should be discoverable through the graph
Check:
- score and sorting direction;
- whether traversal reaches D;
- whether current-query reranking removes C;
- whether duplicate candidates are removed;
- whether incompatible or expired edges are filtered;
- whether the expansion budget is respected;
- whether D disappears when the relevant graph edge is shuffled or removed.
Operational notes for a later prototype
1. Fanout and graph-query cost
Query intents and broadly relevant chunks can become high-degree hubs.
Useful protections include:
max hops
max neighbors per node
max total expanded candidates
top-k edges per edge type
minimum confidence/state
time and version filters
There are related implementation reports in LightRAG’s Neo4j backend, although they do not imply that the same behavior must occur here.
A Neo4j batching issue documented a case where many individual graph calls were produced for one request and proposed batched operations. A separate hub-heavy graph read-path discussion discusses unbounded edge fanout and bounded top-k retrieval.
These are useful implementation warnings rather than evidence against query graphs.
2. Document and model versioning
A stored relevance edge can outlive the data or model that produced it.
Possible invalidation events include:
- source-document update;
- document deletion;
- rechunking;
- embedding-model replacement;
- reranker replacement;
- permissions change;
- corpus migration.
Possible policies include:
- stable document and chunk version IDs;
- cascading edge deletion;
- validity timestamps;
- filtering by corpus/model revision;
- lazy revalidation when an old edge is read;
- periodic cleanup;
- separate edge layers for different reranker revisions.
3. Self-reinforcing retrieval loops
A possible failure path is:
reranker makes one incorrect high-score prediction
↓
the edge is stored
↓
the chunk is retrieved more frequently
↓
more positive-looking interactions accumulate
↓
the incorrect chunk becomes increasingly prominent
Possible controls include:
- separating prediction from verification;
- not treating retrieval frequency as correctness;
- reranking graph candidates against the current query;
- preserving a graph-independent retrieval branch;
- using rejection signals;
- expiring unsupported edges;
- testing shuffled-edge and memory-disabled controls.
4. Privacy and tenant boundaries
Real user queries may contain:
- personal information;
- internal project names;
- confidential intentions;
- security-sensitive search terms;
- information absent from the indexed documents.
In a multi-user environment, persistent queries raise questions about:
- tenant isolation;
- access control;
- retention periods;
- deletion requests;
- whether raw text must be stored;
- whether an embedding or protected representation is sufficient;
- whether one user’s history may influence another user’s retrieval.
This is another reason to distinguish a private QueryEvent log from a reusable learned-memory layer.
5. Auditability
For an evolving retrieval system, it helps to be able to answer:
Why was this chunk retrieved?
Did normal retrieval or graph expansion introduce it?
Which previous query or path led to it?
Which model revision created the edge?
Was the relation predicted or observed?
Was it later verified or rejected?
Which corpus version did it refer to?
Typed edges and immutable retrieval-run metadata make these questions easier to answer than a single mutable relevance weight.
Summary
I would treat the proposal as two experiments:
- Does a query-aware graph improve retrieval beyond a strong first-stage retriever and reranker?
- Does persisting previous query relationships improve later unseen queries beyond a temporary query-aware graph?
The first question has close prior work in query–document graphs, adaptive reranking, and query-centric indexing.
The second appears to be the more distinctive part of the proposal. Its main choices are:
- what is stored: raw queries, intents, predicted edges, verified evidence, or successful paths;
- when a result is promoted into reusable memory;
- how a later query reads that memory;
- how reused candidates are rechecked against the current query;
- how memory is bounded, versioned, evaluated, and removed.
My default first route would therefore be:
hybrid retrieval
→ strong reranker
→ bounded transient graph expansion
→ rerank against the current query
→ only then test persistent, typed, validated memory
That sequence should make it easier to tell whether an improvement comes from the reranker, from graph traversal, or specifically from accumulated query memory.