Employees chat with the HR handbook, IT policies and contract templates — answers come with citations, sensitive topics escalate cleanly.
In short: Employees chat with the HR handbook, IT policies, contract templates and Q&A from Confluence. Answers come with citations. Stack: Azure OpenAI Switzerland (data stays in the Swiss region), embeddings and retrieval, reranking, continuous evaluation on a gold set. PoC setup takes four to eight weeks; ongoing maintenance is handled by an internal RAG team or by us. Anti-hype: no "AI strategy reset" — one concrete function that solves one concrete problem, namely internal knowledge search that actually works.
Knowledge is scattered across PDFs, SharePoint, Confluence and Outlook threads. Employees spend more time searching than working — and often land on outdated versions. Classical full-text search fails on a simple thing: it finds the word, not the answer. Search "vacation" and you get 47 hits; the actual policy lives in a section that doesn't contain that word. Wikis don't get maintained because maintenance is no one's job — and once a wiki is considered stale, no one checks it again.
The pattern looks the same in every SMB: HR and IT get the same five to ten questions every week — vacation entitlement, expense policy, VPN access, onboarding checklist, sick leave — because the answers exist in the documents but are not findable. The result is interrupted work on both sides, knowledge silos, and onboarding cycles that keep getting longer. A RAG solution doesn't solve "everything", but it solves exactly this bottleneck: concrete question in, concrete answer with a source out.
The pipeline is conceptually simple — but operationally demanding to keep healthy. Documents from the various sources (SharePoint, Confluence, fileshare) are chunked — typically 300 to 800 tokens per chunk, with overlap so context doesn't get cut at a boundary. Each chunk is translated into an embedding vector and stored in a vector store (Azure AI Search, pgvector or similar). A user question goes through the same embedding step, the top-K relevant chunks are retrieved semantically, a reranker (a small cross-encoder model) re-sorts by actual relevance, and the LLM gets the top-N chunks as context plus the question. The answer is delivered with source links and a confidence hint.
Model-side, the whole thing runs on Azure OpenAI Switzerland — the data stays in the Swiss region and Microsoft does no training on your data. For particularly strict compliance, there's an on-premise variant with open-source models (Llama, Qwen) on your own hardware, with a performance tradeoff. Guardrails: topic classification before the LLM answer (sensitive topics escalate, see below), per-user and per-day cost caps, a confidence threshold (below the threshold the system says "I don't know" instead of guessing), and a continuously running eval gold set.
Important: the setup is only half the work. A RAG without continuous eval decays — every document change, every model update, every new chunking strategy can shift quality noticeably. Anyone who can't measure that won't notice when the system gets worse, until users simply stop asking. This is exactly where the engineering discipline sits: a gold set of 50 to 100 questions that runs automatically on every index or model change.
A Swiss SMB with around 80 employees — composite, not a real client — has accumulated 400 HR, IT and contract documents over the years. A mix of Markdown, PDF and Word, primarily in German with a handful of English tech policies. Before the RAG rollout: an HR inbox with 15 to 20 standard questions per week, and an IT inbox of similar volume. The answers exist — somewhere. After rollout, in a typical scenario: roughly 70% of the standard questions get answered by the RAG up front, with a source link back to the original document. HR and IT then only see the real edge cases — the ones where experience and context matter. Caveat: the 70% is an illustrative scenario figure, not a delivered benchmark; the actual ratio depends on document quality, question distribution and ongoing maintenance.
Typical effect: standard questions get answered in seconds with citations, HR and IT inbox volume drops noticeably, and employees learn to ask first instead of searching first. But the second half of the story matters more: setup is the beginning, not the end. Continuous evaluation — faithfulness, context precision, answer relevance on a 50- to 100-question gold set — keeps quality stable, otherwise the system decays with every document change. Without that discipline, hit rate sinks unnoticed, users stop asking, and internally the project gets written off as "well, it just didn't really work". With that discipline, the RAG stays a tool that actually gets used. Directionally: fewer routine tickets, shorter onboarding cycles, less knowledge-silo formation — not a published benchmark, but a pattern that recurs across real RAG projects.
Three metrics are standard: faithfulness (does the answer actually reflect what's in the sources — or is it hallucinating?), context precision (are the retrieved chunks actually relevant to the question?), answer relevance (does the answer address the question that was asked, or does it dodge?). In practice: a 50- to 100-question gold set is curated once and maintained, each question paired with a canonical answer and the correct source chunks. Automated scoring with RAGAS or a comparable framework measures each metric. On every index update, every model swap, every chunking change, the gold set runs — if faithfulness drops below threshold, that's a pre-production blocker. Without the gold set, the RAG is flying blind.
Standard setup: Azure OpenAI Switzerland (Region North/West). Embeddings and vector store sit in the same region; the data doesn't leave Switzerland. Important to know: Azure OpenAI does no training on your data — opt-in is default-off, contractually guaranteed by Microsoft. Logs are retained for 30 days for abuse monitoring and deleted afterwards. Anyone with even stricter requirements (sensitive contract data, industry-specific compliance) can switch to an on-premise variant: open-source models like Llama or Qwen on your own hardware, with a performance tradeoff on answer quality and latency. Which variant fits depends on protection needs — we clarify that before the PoC.
Topic classification runs before the LLM answer. Defined sensitive topics — termination, illness, workplace conflict, compensation — trigger an escalation path: no RAG answer, but a direct handoff to HR with a contextual briefing ("Person X has a question about Y, here's the conversation so far"). The reasoning: on sensitive topics, correct advice matters more than fast advice, and HR needs to know the question was asked rather than have it sink into a chat log. Sounds obvious, gets forgotten in practice all the time. Which topics count as sensitive is defined with HR before rollout.
Two cost blocks. PoC setup with 200 to 400 documents covers the eval gold set, chunking pipeline, first model setup, topic classification and four to eight weeks of delivery — the effort range depends on document volume, format variety (PDF/Word/HTML/Confluence exports) and the desired eval depth. Ongoing costs are Azure OpenAI token costs (scale with usage frequency), hosting and vector store, plus internal maintenance time (typically 0.1 to 0.2 FTE for document updates and quarterly eval reviews). For an SMB with 50 to 150 employees, monthly ongoing costs usually sit well below the cost of a single full-time hire. Note: ROI can't be cleanly calculated before the PoC — you test whether usage frequency and time saved justify the cost. We discuss concrete numbers individually based on your document volume and user setup; no PoC, no honest answer.
This use case sits in the AI Engineering practice and connects to several neighboring knowledge and document use cases.
Practice: LLM applications in production — RAG architectures, document intelligence, evaluation, guardrails, platform choice.
Pull structured data out of PDFs, invoices and contracts — with validation, confidence and escalation.
Agentic review of contracts and compliance documents against internal policies — with traceable reasoning.
Customer support in DE, EN, FR, IT — over the same knowledge base, with citations and tier-2 escalation.
RAG is worth it when the knowledge base is substantial (more than 200 documents) and search frequency is noticeable. With under 50 documents and rare use, a well-maintained wiki plus glossary is often cheaper and more honest. In a non-binding 30-minute discovery call we'll talk through whether your knowledge base and usage justify the effort.
Book a discovery call