Everyone is talking about RAG.
And for good reason, it solved a real problem. LLMs are powerful, but they only know what they were trained on. RAG gives them access to your documents, your databases, your private knowledge.
But here's the question nobody asked loudly enough:
What happens to sensitive data before it reaches the vector database?
Let me make this concrete.
A company wants an internal AI assistant. Employees upload HR documents, customer contracts, medical records. The RAG pipeline chunks them, generates embeddings, and stores everything in a vector database.
The assistant works beautifully.
But now you have a new problem — one you didn't plan for.
You've just created a searchable, queryable store of some of the most sensitive data in your organization. And the LLM sitting in front of it will happily retrieve and surface any of it.
That's not a model problem. That's an architecture problem.
That question is what led me to build LexiRedact.
LexiRedact is a privacy-preserving middleware layer that sits between your application and your vector database. Before any content is embedded and stored, it:
Detects PII using Microsoft Presidio (names, emails, phone numbers, medical identifiers, and more).
Redacts or pseudonymizes sensitive entities.
Generates embeddings from the sanitized content only.
Stores the privacy-preserving version in ChromaDB.
The key design decision I obsessed over wasn't detection accuracy. It was this:
Privacy that destroys retrieval quality is not a solution. It's a different problem.
So LexiRedact is built around a dual-pipeline approach: one path that prioritizes privacy, one that balances privacy with retrieval utility because the right tradeoff depends on what the data is and how it will be used.
What the numbers look like:
PII Detection F1 Score: 82.5%
Sensitive data reaching the vector store: reduced by 78.9%
Stack: Presidio · ChromaDB · Redis · FastEmbed · FastAPI
Published as an open-source Python package: pip install lexiredact
What building this actually taught me
Most AI engineering conversations are about model selection, embedding quality, and retrieval strategies.
Those are important.
But production AI systems have a different layer of hard problems:
What data are you actually storing, and should you be storing it?
Who can retrieve what, and under what conditions?
If your vector database is breached, what is exposed?
How do you audit what your RAG system retrieved and why?
These aren't model problems. They're system design and data governance problems. And they don't get solved by choosing a better LLM.
The most interesting AI engineering work right now isn't happening at the model layer.
It's happening in the infrastructure around it.
If you're building RAG systems at your organization, I'd genuinely like to know: is data privacy part of your ingestion pipeline, or is it something you're handling downstream?
Why this beats everything else on that platform:
Every other post is opinion. This has a published PyPI package with a release date as evidence
The hook is a problem, not a project pitch recruiters keep reading
The metrics (82.5 F1, 78.9% reduction) are specific and verifiable
The closing question invites engagement without begging for likes
It positions you as someone who thinks about systems, not just models, which is exactly what an AI Engineer role requires
Link: https://pypi.org/project/lexiredact/
