Implementation
Design Source Permissions, Freshness and Deletion Before Building a Company RAG System
Before approving a company knowledge assistant, establish which sources may answer, who may access them and how every correction, revocation and deletion reaches the user-facing result.
Start with the approval decision, not the assistant demo
A technical leader is asked to approve one assistant for three useful but very different sources: access-controlled SharePoint documents, a centrally maintained policy file and licensed industry reporting. The demonstration looks convincing. Staff can ask one question and receive a fluent summary assembled from several documents.
Then the approval questions begin. One SharePoint document is restricted to a small team. The policy directory contains both current and superseded files. The reporting licence may restrict storage, redistribution or generated use. Nobody has yet shown what happens after a user loses access or a report is revoked.
This is not primarily a retrieval decision. It is a decision about whether source authority, permissions and lifecycle controls survive the complete route from source to answer.
The proposed architecture should be approved, limited or rejected according to evidence that it can:
distinguish authoritative sources from supporting or disputed material;
preserve document-level access boundaries;
propagate corrections, permission changes, revocations and deletions;
expose access-safe provenance;
abstain when it cannot support a safe answer; and
assign every material corpus decision to a named owner.
Start with a bounded workflow, defined user groups and a source inventory. Bring in the people responsible for each source, access control, security and applicable content-use terms. Check current primary documentation for the actual connectors, identity systems, indexes and retrieval tools under consideration.
This framework supports that pre-build decision. It is not a platform installation guide, product shortlist, legal opinion or comprehensive evaluation of ranking and generation quality. NIST similarly presents its AI Risk Management Framework as a voluntary way to incorporate trustworthiness considerations into the design, use and evaluation of AI systems.
Decide which sources may speak for the business
Create a source register before deciding how to chunk, embed or retrieve content. For each candidate, record:
source system and stable document identifier;
business owner and the person authorized to designate its status;
authority category and the scope within which that category applies;
version and effective, publication or expiry date;
replacement link when superseded;
intended user groups;
content-use restrictions; and
the date and owner of the next review.
Use categories that lead to a decision:
Authoritative: designated by the accountable owner to govern a defined question or process.
Supporting: useful context that cannot override an authoritative source.
Restricted: potentially valid, but available only to permitted identities and uses.
Superseded: replaced for current decisions, although it may remain relevant to historical questions.
Disputed: authority, applicability or accuracy has not been resolved.
Inadmissible: excluded because its use is not approved, its origin cannot be established or it falls outside the workflow.
Authority is a business designation, not a retrieval score. Research on reliability-aware RAG also identifies the limitation of relying only on query-document relevance when sources have different reliability, but an internal authority decision still needs named organizational ownership rather than an inferred model score (ACL Anthology).
In the running scenario, the policy owner might designate one controlled policy record as authoritative for current employee questions. The licensed report could remain supporting and restricted. An unattributed policy copy should stay disputed or be excluded until the owner resolves it.
Trace permission boundaries through every copy and query
Draw the complete information path. Include the source, connector, extracted text, chunks, metadata, index or vector store, cache, retrieval filter, prompt context, generated answer, citation and logs.
For every stage, ask three questions:
Whose identity applies? Identify the end user, service identity and any group mapping involved.
Where is authorization enforced? Name the system and rule that decide whether the item may continue.
What happens when the decision cannot be made? The safe behavior is to withhold the item, not broaden access.
SharePoint documents that its own search results are security trimmed, so users see content they have permission to view. It also allows site owners to control whether site, list or library content appears in search (Microsoft SharePoint documentation). That behavior supports permission-aware SharePoint search. It does not establish that a separately copied RAG index will inherit the same controls.
Microsoft Graph external connector items provide another bounded mechanism. Their ACL entries can associate grant or deny access with users, groups and defined broader identity types (ACL resource documentation). If an ACL is included in the documented item update request, the existing ACL collection is overwritten by the submitted collection (update documentation).
Those mechanisms still need to be connected correctly to source permissions and the requesting user's identity. A copied chunk without its source identifier and access metadata is already outside that chain. A retrieval filter applied after unauthorized content enters prompt context is too late. Citations and logs also need access controls because titles, snippets and query traces can reveal sensitive information.
Make corrections, revocations and deletions converge
Treat every source item as a lifecycle-managed record. Give it a stable identifier that connects the source document to its extracted content, chunks, index entries, cache records and citations.
For each event type, define a propagation route:
detect the source update, ACL change, correction, revocation, supersession or deletion;
place the event into a retryable synchronization process;
update or remove every derived record linked to the stable identifier;
invalidate caches and previously prepared answer material where applicable;
reconcile downstream state against the source; and
verify the result through the same retrieval path users access.
Microsoft's external-content guidance says that changes in a source must be synchronized with connection items and describes event-based and scheduled refresh approaches. It also documents creating, updating and deleting external items by stable item identifiers (Microsoft Graph connection guidance). The external item deletion endpoint returns 204 No Content when successful (delete documentation). That response describes the endpoint result, not the state of a separate vector store, application cache or generated answer layer.
Set freshness targets according to the consequence of stale information. A rapidly changing access rule may require event-driven propagation and a short convergence limit. A low-risk reference archive may tolerate a scheduled refresh. The important point is that the limit is explicit, owned and testable.
A deletion is complete only when the user-facing path no longer retrieves the item and the required downstream records have been removed or handled under the applicable retention rule. Record the source event, affected identifiers, synchronization attempts, verification result and responsible owner.
Resolve conflicts before retrieval rank can decide
Use a deterministic order before ranking retrieved passages:
Confirm that each candidate is admissible for the requesting user and intended use.
Prefer the source designated as authoritative for that subject and scope.
Check whether the source applies to the user, jurisdiction, department, product or time period in question.
Apply effective dates and version status.
Collapse true duplicates that represent the same source record.
Escalate unresolved conflicts to the named decision-maker.
Suppose the current policy file says expenses require one approval, while a licensed report describes a different industry practice. The licensed report may be relevant context, but it cannot override the company's designated policy. If two policy files both appear current and authoritative, the assistant should not let semantic similarity choose one. The conflict must follow the defined escalation route.
Older material is not automatically wrong. A superseded policy may be the correct source for a historical question about what applied before its replacement date. The source register therefore needs applicability dates, not just a current-versus-old label.
Show enough provenance to support or withhold an answer
A useful answer should expose enough information for an authorized user to inspect its basis. Depending on the workflow, show:
source name;
document title or access-safe identifier;
version;
effective or publication date;
retrieval time when freshness is material; and
a link that the requesting user is authorized to open.
Provenance must obey the same access boundary as the answer. If a user cannot know that a restricted investigation exists, the system must not reveal its title or a suggestive snippet while withholding only the link.
The workflow should abstain when:
no source has the required authority;
supporting evidence is inaccessible to the user;
identity or authorization cannot be established;
an important conflict remains unresolved;
the content is revoked, expired or outside its permitted use; or
available evidence does not meet the workflow's freshness rule.
An abstention should still be useful. It can state that the answer cannot be established from approved current sources and direct the user to the policy owner or another defined resolution path. It should not fill the gap with lower-authority material or model memory.
Assign an owner to every corpus decision
Separate business accountability from technical operation. One person may own the meaning and authority of a policy while another operates its connector and index.
Assign explicit decision rights for:
admitting or excluding a source;
designating authority and scope;
approving user groups and permission mappings;
reviewing content-use terms;
marking versions as superseded;
setting freshness and recertification intervals;
resolving conflicts;
processing deletion and revocation requests;
handling synchronization failures; and
escalating potential exposure incidents.
Each responsibility needs an expected response window proportionate to the workflow. A permission-removal failure affecting restricted documents needs a faster route than a routine metadata correction.
Add a handover rule. If an owner leaves or stops maintaining the source, authority should transfer formally or expire. An abandoned corpus must not remain silently authoritative because an index continues to retrieve it. Broader operating practices can be tied into Monitoring & Governance without confusing that service page with the official documentation for a source system.
Choose the narrowest architecture that meets the need
Generated synthesis is not always the requirement. Decide whether users need a new combined answer, an authorized document, a maintained standard response or a current field from a system of record. The comparison below keeps those routes tied to the same permission, authority and lifecycle requirements.
Choose according to the knowledge task and its controls
RAG over copied or indexed content
Generated synthesis across several governed sources may justify RAG when users need a combined answer rather than documents or fixed records. It also creates more places where permissions, freshness, content-use limits and provenance can fail. Approval therefore requires clear source authority, enforceable access at retrieval time, lifecycle control over every derived copy, access-safe citations and named operational ownership.
A narrower source-grounded route
Use permission-aware enterprise search when finding authorized documents is enough, an owner-maintained knowledge base when a small curated answer set is sufficient, or deterministic system-of-record queries when the answer should come from a defined field. A limited low-risk RAG pilot fits when synthesis matters but the wider corpus is not yet governable. Select the route by permission fidelity, freshness, source authority and failure consequences, not by demo appeal.
Build a pre-approval test matrix
Create one row per test with these fields: test identity, source state, expected behavior, observed behavior, evidence reference, owner and disposition. Use representative authorized and unauthorized identities, including a user whose access changes after ingestion.
Run each test through the user-facing route rather than querying only an intermediate store. Retain enough evidence to reproduce the decision without placing restricted document bodies, raw prompts or sensitive snippets into the review record. NIST's voluntary AI RMF Playbook organizes suggested risk-management actions around Govern, Map, Measure and Manage, which provides a useful general structure for assigning ownership and reviewing test evidence.
Pre-approval control tests
Restricted document: Request a known restricted item as an unauthorized identity. Pass only if retrieval, answer text, citations and logs expose no restricted content or unsafe metadata, supported by evidence from the user-facing path.
Permission loss and ACL update: Ingest an allowed item, remove the user's permission and propagate the ACL change. Pass only when the former user can no longer retrieve or receive the item through any answer surface, with the changed ACL and retrieval result retained as evidence.
Deleted document: Delete a source item and trigger the defined synchronization path. Pass only when the source identifier is absent from applicable indexes, vector stores, caches, derived records, answers and citations, as verified through retrieval rather than an API response alone.
Revoked licensed report: Mark a previously available report as revoked. Pass only if it stops contributing to retrieval and generation within the agreed limit, while the evidence record shows the revocation event, downstream checks and responsible owner.
Superseded policy: Introduce a replacement policy with a later effective version. Pass only if current questions use the replacement, the old version is marked as superseded and historical use remains limited to questions for which it is applicable.
Stale or duplicate evidence: Supply an expired copy and duplicate passages from the same source. Pass only if stale content is withheld or labelled according to policy and duplicates do not gain artificial authority through repetition.
Conflicting authoritative candidates: Provide two apparently authoritative sources with incompatible instructions. Pass only if the workflow applies the defined authority and applicability rules or escalates and abstains, rather than allowing retrieval rank to settle the dispute.
Unsafe citation: Retrieve evidence that the requesting user cannot safely open. Pass only if the answer withholds both the evidence and revealing citation metadata, while offering an access-safe next action.
Missing or ambiguous authority: Ask a question supported only by unowned or disputed material. Pass only if the workflow declines a definitive answer and identifies the appropriate owner or resolution route without presenting the disputed source as policy.
Failed identity resolution: Remove or corrupt the identity or group mapping needed for authorization. Pass only if retrieval fails closed, no fallback broadens access and the failure is recorded without copying sensitive source content into logs.
Required abstention: Test revoked, inaccessible, insufficiently fresh and unresolved evidence. Pass only if the expected abstention appears consistently in answer text and no unsupported answer is produced from model memory or lower-authority material.
Convergence sweep: After each change, query with representative authorized and unauthorized identities. Pass only when source state, index state, caches, derived records, answer text, citations and reviewable logs agree with the expected result.
Turn the evidence into approve, limit or reject
Approve the bounded architecture only when:
authoritative sources and owners are named;
access fails closed throughout the data path;
permission changes, corrections, revocations and deletions converge within agreed limits;
provenance is useful and access-safe;
required abstention behavior works; and
the test matrix contains reviewable passing evidence.
Limit the proposal when those controls work only for a smaller corpus, selected user group or low-risk pilot. Document excluded sources, prohibited uses and the evidence required before expansion. In the running scenario, this could mean approving only the centrally maintained policy corpus while leaving licensed reporting and sensitive SharePoint libraries outside the system.
Reject or pause the proposal when restricted content can surface, identity cannot be enforced, revoked material remains retrievable, applicable content-use terms remain unresolved, no owner accepts accountability or critical evidence is missing.
This decision can lead to a narrower and more useful design rather than a binary choice between a company-wide RAG system and no improvement at all. If the design proceeds, Workflow & Agent Implementation is the relevant owned service for building the controlled workflow, while vendor documentation remains the authority for product behavior.
Next step: test one disputed source and one permission change
Choose one bounded knowledge workflow you already understand. Identify the source authorized to answer one important question, then trace either a realistic permission removal or source revocation through every proposed copy, index, cache and answer surface.
The result should be one of three concrete outcomes: a documented pass, a narrower scope that can be operated safely or a specific unresolved control that blocks approval. That small exercise turns the SharePoint, policy-file and licensed-reporting scenario from a convincing demo into an evidence-based architecture decision.
Questions and answers
Do SharePoint permissions automatically carry into a separate RAG index?
No automatic inheritance should be assumed. SharePoint documents its own security-trimmed search behavior, but a separate ingestion, indexing and generation path needs an explicit identity and authorization design at every stage.
Is a successful deletion API response enough to prove that a document is gone?
No. It proves that the documented endpoint accepted the deletion in that system. Approval should depend on user-facing checks showing that the item is no longer available through indexes, caches, derived records, answers or citations.
When should a knowledge assistant refuse to answer?
It should abstain when authority is missing, identity or authorization cannot be established, required evidence is inaccessible, content is revoked or too stale, or an important source conflict remains unresolved.