Building a Secure Multi-LLM RAG Chatbot for WhatsApp

One of the core challenges of deploying generative AI inside a bank is trust: every answer needs to be traceable to an official source, and no sensitive document can leave the internal network. This ruled out most hosted LLM APIs for the retrieval step.

The architecture indexes official documentation into PostgreSQL using the pgvector extension, so the same database that already runs the bank's operational systems also serves as the vector store — no extra infrastructure to secure. A custom NLU module routes incoming WhatsApp messages to the right retrieval flow before a language model drafts the final answer.

To choose the right model, I benchmarked GPT-4o, Claude 3.5, Gemini 1.5 and a locally hosted Llama across latency, cost and confidentiality constraints — the local model ultimately handled the most sensitive query types, while hosted models cover lower-risk traffic.