Skip to main content
Style:
Size:

Chapter 9: Feeding the Machine (Securing RAG and Corporate Data)

If you buy an off-the-shelf Large Language Model (LLM) and ask it to summarize your company’s Q3 financial strategy, it will fail. It doesn't know your strategy. It only knows the public internet data it was trained on.

To make AI useful for the enterprise, companies use a technique called Retrieval-Augmented Generation (RAG).

RAG acts like an incredibly fast research assistant. When an employee asks the AI a question, the system searches your internal corporate documents, finds the relevant paragraphs, hands those paragraphs to the LLM, and says: "Read this internal data, and use it to answer the user's question."

This is a brilliant capability, but from an Identity and Access Management perspective, it introduces a terrifying data privacy nightmare.

The RAG Identity Crisis​

In a traditional file system (like SharePoint or Google Drive), Role-Based Access Control (RBAC) works perfectly.

  • Alice is in HR. She can see the Payroll.pdf file.
  • Bob is an Intern. The file system completely hides Payroll.pdf from him.

But in a RAG system, your corporate documents are ingested, shredded into chunks of text, converted into numbers, and stored in a Vector Database.

  • The Flaw: Many early Vector Databases were built by data scientists, not security engineers. They stripped away all the IAM permissions during the ingestion process.
  • The Impact: If Bob the Intern asks the corporate AI chatbot, "What is the CEO's salary?", the RAG system searches the Vector Database, finds the chunks of text that originated from the Payroll.pdf file, feeds them to the LLM, and happily tells Bob the CEO makes $2 million a year.

The AI just bypassed your entire HR security perimeter because the data lost its identity context.

The Solution: Entitlement Propagation & Authorization Architectures​

To secure a RAG pipeline, the authenticated context of the principal asking the question must travel all the way down to the Vector Database. This is called Entitlement Propagation.

However, how the vector database applies this authorization matters tremendously.

Pre-Filtering vs. Post-Filtering: The Security & DoS Trade-Off​

  1. Post-Filtering (Flawed Architecture): The system performs the approximate nearest neighbor (ANN / k-NN) vector search first across the entire database, retrieves the Top-10 most semantically relevant chunks, and then checks if the user has permission to view them.
    • Denial of Service (DoS): If the top 10 chunks all belong to restricted executive files, the filter strips all 10 out, returning zero results to a user who legitimately had secondary public files available.
    • Inference / Vector Space Leakage: An attacker can craft probing prompts to determine that confidential documents exist on specific topics based on search latency and empty result anomalies.
  2. Pre-Filtering (Zero Trust Architecture): The vector database indexes metadata tags (e.g., acl_allow_groups: ["finance_team", "all_employees"]). Before computing vector cosine similarities, the engine filters the search space strictly to chunk embeddings matching the user's validated identity claims.

Chunk-Level Security (CLS) vs. Document-Level Security (DLS)​

Enterprises rarely have documents where every single sentence shares the same classification. A 50-page corporate strategy document might contain 45 pages of general product roadmap and 5 pages of unreleased M&A financials.

  • Document-Level Security (DLS): Coarse-grained. If the document is tagged Internal, all 50 pages are exposed.
  • Chunk-Level Security (CLS): Fine-grained. During the text splitting/chunking pipeline, each individual text chunk inherits contextual classification tags (e.g., using automated NLP classifiers or section headers). Only chunks with classification: public or clearance: executive are returned based on the caller's JWT scope.

The Operational Challenge: Vector Embedding Staleness & ACL Sync​

What happens when an employee is transferred or an existing document's permissions change in SharePoint?

  • The Staleness Problem: Vector databases do not automatically track Active Directory ACL changes in real time. If Alice is removed from the "Mergers & Acquisitions" security group, but the vector index retains the stale metadata tag allowed_users: ["alice"], Alice can continue querying confidential M&A data through the AI assistant.
  • The Architecture Solution:
    1. Tag by Group/Role IDs, Not User IDs: Embed persistent group identifiers (e.g., Entra ID Group Object IDs) rather than individual user emails into chunk metadata.
    2. Event-Driven ACL Webhooks: Build real-time event bridges (e.g., Kafka / AWS EventBridge) connecting the enterprise IdP/HR system to the vector database metadata index, triggering near-instant ACL metadata updates upon group membership changes.
    3. Tenant Namespace Isolation: For multi-tenant environments, enforce physical partition or namespace isolation to mathematically prevent cross-tenant vector contamination.

The 5 Pillars of RAG Security​

  1. People: Data Owners must classify documents and define authoritative group boundaries before vector ingestion.
  2. Process: Implement event-driven ACL synchronization pipelines to eliminate vector metadata staleness.
  3. Technology: Deploy vector databases supporting native Pre-Filtering and Chunk-Level Security metadata schemas.
  4. Control: Enforce cryptographic Entitlement Propagation via user-bound OAuth tokens passed to the query engine.
  5. Impact: By pre-filtering vector retrieval and strictly tagging at the chunk level, unauthorized data is mathematically excluded before the LLM generates a response.

Interactive Simulator: Entitlement Filter Visualizer​

See how vector database ACL tags and identity tokens dynamically filter knowledge retrieval:

RAG Identity & Entitlement Simulator

See how Document-Level Security (DLS) and Access Control Lists filter RAG vector contexts before inference.

Vector Knowledge Base
Chunks tagged with IAM metadata ACLs:
Public (Level 1)
Our public product roadmap will launch next Tuesday.
Employee (Level 2)
The internal engineering cafeteria is scheduled for weekend maintenance.
Executive (Level 3)
Projected Q4 confidential acquisition target valuation exceeds $40M.
1. Present Authenticated Token
Active Context: Guest (Level 1)
2. Query the Assistant

Consultant's Corner: The "Garbage In, Garbage Out" Warning​

When you are hired to architect IAM for an enterprise AI rollout, you will often find that the client's underlying file permissions are a total disaster.

They will have "Global Share" folders containing accidentally uploaded passwords, unredacted customer data, and legacy HR files. In the past, this was protected by "security through obscurity"—nobody knew the files were there, so nobody looked for them.

AI destroys security through obscurity.

An LLM will instantly find, read, and summarize that forgotten global folder. If you feed garbage permissions into a RAG pipeline, you will build an incredibly efficient, automated data-leak machine. Before you deploy internal AI, you must enforce a massive cleanup of the source data permissions. If the client refuses, make sure they sign off on the risk in writing.

💡 Scenario & Solution: Vector Space Inference Leakage in Post-Filtered RAG and Event-Driven ACL Sync Architecture​

The Scenario: A pharmaceutical enterprise builds a RAG research assistant indexing patent applications. The vector database was configured using Post-Filtering. A junior chemist prompts the assistant: "Did our lab discover any new kinase inhibitors targeting Protein-X last month?" The search engine finds the top 5 confidential patent drafts, but because the chemist lacks the Patent-Restricted permission, the post-filter drops all 5 results and responds: "No results found." However, network latency logs reveal a 450ms semantic search followed by an authorization drop, and subsequent adversarial probing allows the user to deduce confidential research project codes.

Why It Happened: Post-filtering executed the vector search against restricted embeddings and leaked information via timing discrepancies and result exclusion patterns.

The Architecture Solution:

  1. Switch to Pre-Filtering Metadata Indices: Configure the vector index (e.g., Pinecone, Qdrant, pgvector) with composite HNSW + metadata filter graphs. The user's active groups (claims.groups) are evaluated prior to distance calculation.
  2. Chunk-Level Sensitivity Tagging: Ingest patent documents through a pipeline that automatically attaches fine-grained classification metadata (sensitivity_tier: tier_1_restricted) to each chunk.
  3. Dynamic ReBAC Entitlement Resolver: Integrate a lightweight ReBAC service (e.g., OpenFGA / SpiceDB) at the retrieval gateway that resolves the caller's real-time relationship graph and injects authorized resource IDs into the pre-filter clause before hitting the vector database.