Back to blog

Side Project ·

Building a Secure, High-Performance RAG Chatbot: Architecture and Development Journey

In modern web development, adding a smart conversational assistant to a site often means relying on third-party SaaS platforms with recurring fees, limited customization, and questionable privacy controls.

Source: github.com/pdaszko/sa-chatbot

For Szkolenia Skin Atelier, we set out to build a completely custom, self-hosted, and secure AI chatbot. It uses Retrieval-Augmented Generation (RAG) to answer user questions based directly on the actual content of the website, captures prospective leads, sends automatic notifications, and stands protected against DDoS and API abuse.

  • Next.js App Router for the frontend and localized routing.
  • Vercel AI SDK with Google Gemini for streaming conversational responses.
  • Local vector search and embeddings for secure Retrieval-Augmented Generation.
  • Nodemailer-powered lead capture and email notification workflow.

Chat Walkthrough

If the video does not start, open it directly: View raw MP4.

Technical Architecture: How it Works

The chatbot is built using Next.js, Vercel AI SDK, Google Gemini, and Nodemailer, running as a self-hosted Node.js application.

1. The RAG Pipeline (Retrieval-Augmented Generation)

Rather than retraining or fine-tuning an expensive model, we feed the website's factual content directly to the LLM at query time.

  • The Vector Database: A crawling script scrapes sitemaps and pages, processes the text into semantic paragraphs, and converts them into mathematical vector representations using the gemini-embedding-001 model. This is stored locally in a high-density, flat-file database (knowledge.json).
  • The Retrieval Step: When a user asks a question, the API dynamically embeds their question and calculates the mathematical Cosine Similaritybetween the question's vector and all paragraphs in our database.
  • Context Assembly:The top 8 most semantically relevant paragraphs are retrieved and injected into the LLM system prompt as the "source of truth".

2. Large Language Model & Streaming

The generation is handled by Google's Gemini 3.5 Flash model via the Vercel AI SDK. This ensures:

  • Conjugate Multilingual Support: The chatbot automatically detects the input language and answers in the same language.
  • Low Latency: Answers stream back to the UI character-by-character, keeping user engagement high.
  • Cost Efficiency: Running gemini-3.5-flash keeps per-query API costs under fractions of a cent.

3. Lead Capture & Notification Engine

When a user expresses interest in booking a training course or leaves contact details, the system handles it actively.

  • Tool Calling: The model invokes a JSON-schema-validated captureLead tool as soon as contact details appear.
  • SMTP Mailer: The backend connects to SMTP using Nodemailer to instantly email leads to administrators.

4. Dynamic Configuration Layer

To allow site owners to modify prompt guidelines, brand colors, welcome messages, links, and avatar icons without rewriting code, we isolated configuration into a JSON file. The API route and React widget fetch this configuration on demand, making updates instantaneous.

5. Zero-Dependency IP Rate Limiting

Publicly exposed AI endpoints are frequent targets for DDoS attacks and API quota exhaustion. To protect our backend, we developed a high-efficiency sliding-window rate limiter in pure TypeScript. It blocks abusers with HTTP 429 Too Many Requests before any vector calculation, Gemini token, or SMTP email is spent.

The Development Journey: Iterative Milestones

01

Knowledge Acquisition (The Crawler)

We began by creating a crawler script (generate-knowledge.ts). It targets XML sitemaps, extracts visible text, segments content into semantic chunks, and builds the local embedding database.

02

Core Chat and UI Widget

We implemented a responsive React widget with maximum/minimum view states, full mobile support, and an inline layout mode for page builders and iframe embedding.

03

Generalization & Decoupling

Initially built for a single site, we refactored the codebase to decouple business logic from configuration. We added a dedicated configuration endpoint and extracted hardcoded details into a flexible JSON format.

04

Production Hardening & Abuse Protection

As deployment neared, we hardened security with zero-cost, memory-cache rate limiting. The system was tested and tuned to block aggressive automation before it could consume quota.

05

Hydration and UI Polishing

We resolved edge cases like hydration mismatches caused by browser extensions or injected scripts, introducing targeted suppression flags to keep the console clean and the client render stable.

Key Takeaways

  • Self-hosting is viable: Building a custom RAG pipeline avoids costly monthly SaaS chatbot platforms.
  • Local memory is powerful: Persistent Node.js processes let us add enterprise-grade IP security without external caches.
  • Flexible architecture wins: Decoupling styling and prompt configuration into JSON makes the chatbot reusable across websites.