In this part of the course, we focus on designing a production-ready RAG pipeline that can serve as the core intelligence of a BIM conversational agent. This is not a toy setup and not a notebook demo. The goal is to design a system that can handle real project data, scale across multiple projects, and answer questions grounded in both standards and actual building models.

We start with RAG intentionally. Before worrying about APIs, authentication, or user interfaces, we need to make sure the system can reliably retrieve the right information. If retrieval fails, everything downstream fails with it. By isolating the RAG pipeline early, we can experiment, measure quality, and iterate quickly without the overhead of a full backend or frontend.

Throughout this article, we break down:

• What a production RAG pipeline actually looks like

• Why simple vector search is not enough

• How hybrid search, reranking, and context management fit together

• How do different types of BIM knowledge require different processing strategies

We also introduce the three core knowledge sources our BIM agent must reason over: IFC standard documentation, project-specific user documents, and IFC model files themselves. Each one behaves differently, and treating them as a single data source is one of the most common mistakes in real-world systems.

By the end of this part, you should have a clear mental model of how the RAG pipeline works, why it’s structured the way it is, and how it fits into a larger production system. In the next sections, we’ll move from concepts to implementation and start building the ingestion and processing layers step by step.

Get All My 8 Books With 60% Off

This is the third part of the Building a Production-Ready Agentic Chatbot Course. You can read the previous issues from here:

• Part 1: Introduction & Course Outline

• Part 2: System Architecture & Environment Setup

• Part 3: Designing Production-Ready RAG Pipeline [You are here!]

All the codes and data used in this course can be found in the course GitHub Repo

Why Start With RAG?

Get 30% off forever

You might be wondering why we’re building the RAG pipeline before the backend infrastructure or the frontend interface. There are good reasons for this approach.

First, the RAG pipeline is the core intelligence of the system. Everything else wraps around it. If the retrieval doesn’t work well, no amount of beautiful UI or robust backend architecture will make the system useful. Starting here lets us validate the most critical part first. We can test retrieval quality, experiment with different chunking strategies, and tune the system with real documents before we commit to specific API designs or user interfaces.

Second, the RAG pipeline can be developed and tested in isolation. We don’t need authentication, user management, or a frontend to validate that our document processing works. We can write Python scripts, run them locally, inspect the results, and iterate quickly. This tight feedback loop is valuable when you’re tuning a complex pipeline.

Third, understanding how the RAG pipeline works will inform our backend and frontend design decisions. Once we know what information the RAG system needs (project IDs, document metadata, conversation history), we can design database schemas and APIs that support it naturally. Once we know what the pipeline returns (retrieved chunks, confidence scores, source citations), we can design UI components that display this information effectively.

The alternative approach, building backend and frontend first and then trying to retrofit the AI logic later, usually results in awkward APIs, inefficient data flows, and systems that are hard to optimize. We’ve seen this pattern fail too many times, so we’re doing it right from the start.

Understanding RAG: The Complete Pipeline

Get 30% off forever

Before we dive into code, let’s understand what a production RAG pipeline actually does. RAG stands for Retrieval Augmented Generation, and it works by enhancing an LLM’s capabilities with relevant information retrieved from a knowledge base.

RAG is often introduced as a simple idea: retrieve relevant documents and pass them to a language model. That description is technically correct—but very incomplete. In production, a RAG system is a multi-stage pipeline with several moving parts, each one affecting accuracy, latency, and reliability.

This section walks through the full RAG pipeline as it’s built in real applications, from ingestion to generation, and highlights advanced techniques such as hybrid search, reranking, and context optimization.

Step 1: Data Ingestion and Preparation

Get All My 8 Books With 60% Off

Everything starts with the data. In production, documents are rarely clean PDFs or well-structured Markdown files. You’re usually dealing with a mix of:

• PDFs, Word files, and slides

• Web pages and internal wikis

• Logs, tickets, or database records

The ingestion step typically includes:

• Parsing and normalization (text extraction, encoding fixes)

• Metadata enrichment (source, timestamp, author, access level)

• Deduplication to avoid repeated content poisoning of retrieval results

Metadata is especially important in production. It allows you to filter retrieval results later (for example, by user role or document freshness) instead of relying on embeddings alone.

Step2: Chunking with Retrieval in Mind

Get 30% off forever

Chunking is not just about splitting text into fixed sizes. Poor chunking leads to poor retrieval, no matter how good your embeddings are.

Production-grade chunking often involves:

• Semantic chunking (splitting by meaning rather than character count)

• Overlapping windows to preserve context

• Structure-aware splitting (headings, tables, code blocks)

The goal is to create chunks that are:

• Large enough to be meaningful

• Small enough to be retrieved precisely

• Self-contained when read by the model

This step directly affects recall and precision downstream.

Step 3: Indexing and Vector Storage

Get All My 8 Books With 60% Off

Once chunks are ready, they are embedded and stored in a vector database. At this stage, you typically index:

• The embedding vector

• Raw text

• Metadata fields

In production systems, indexing is often incremental, not batch-only. New documents arrive continuously, and re-embedding everything is usually too expensive.

This is also where you decide how retrieval will work:

• Approximate nearest neighbor (ANN) indexes for speed

• Separate indexes per domain or tenant

• Time-based or namespace-based partitioning

Step 4: Query Understanding and Transformation

Get 30% off forever

User queries in real applications are messy. They may be:

• Too short

• Ambiguous

• Poorly phrased

• Multi-intent

Before retrieval, production systems often apply query transformations, such as:

• Query rewriting using an LLM

• Expanding acronyms and synonyms

• Generating multiple query variants

This step improves recall by aligning user language with how documents are written, without changing the original user intent.

Step 5: Hybrid Search: Dense + Sparse Retrieval

Get All My 8 Books With 60% Off

Pure vector search is rarely enough in production.

Hybrid search combines:

• Dense retrieval (embeddings, semantic similarity)

• Sparse retrieval (BM25, keyword matching)

Why this matters:

• Dense search is good at meaning

• Sparse search is good at exact terms, IDs, and rare keywords

In practice, hybrid search helps with:

• Technical terms

• Error codes

• Names, versions, and identifiers

Results from both retrieval methods are usually merged using weighted scores or rank fusion techniques.

Step 6: Reranking for Precision

Get 30% off forever

Read more

Most tutorials jump straight into coding an AI agent. They show you how to wire up an LLM to a prompt, maybe add a vector database, and call it a day. But when you try to turn that proof of concept into something real—something you can deploy, maintain, and scale—you realize that the agent logic is only about 20% of the work.

The other 80% is everything around it: how data flows through the system, how users authenticate, how errors are handled, how the frontend communicates with the backend, and how the whole thing gets deployed. These aren’t afterthoughts. They’re core architectural decisions that, if made poorly early on, will haunt you later.

In this tutorial, the second of the Building a Production-Ready Agentic Chatbot Course, we’re going to design the entire system before writing a single line of agent code. We’ll understand the problem space, break down the architecture into components, choose our technology stack, and set up a development environment that mirrors production as closely as possible.

By the end, you’ll have a clear mental model of what we’re building, why we’re building it this way, and a working development environment ready to start coding.

Get All My 8 Books With 60% Off

Table of Contents:

1. BIM and IFC Fundamentals

   1. What is BIM?

   2. What are IFC Files?

   3. Why is This Hard to Work With?

   4. Natural Language Access to BIM Data

2. Understanding the Problem Space

   1. What Success Looks Like for this Project?

3. System Architecture Design

   1. High-Level Architecture

      1. Frontend (React + TypeScript)

      2. Backend API (FastAPI)

      3. AI Agent Engine (LangChain/LangGraph)

      4. Database (PostgreSQL with pgvector)

      5. External Services

   2. Data Flow: How a Query Works

   3. Why This Architecture?

4. Technology Stack

   1. AI & Agent Layer

   2. Backend

   3. Frontend

   4. DevOps & Infrastructure

5. Development Environment Setup

   1. Step 1: Install Git

   2. Step 2: Install Docker

   3. Step 3: Install Python 3.11+

   4. Step 4: Install Node.js and npm

   5. Step 5: Choose Your IDE

   6. Step 6: Clone the Course Repository

   7. Step 7: Create Necessary Accounts

      1. OpenRouter Account

      2. Azure Account (for Blob Storage)

      3. GitHub Account

   8. Step 8: Set Up PostgreSQL Locally (with Docker)

   9. Step 9: Verify Your Setup

   10. Step 10: Set up Project Structure

       1. Why This Structure?

       2. Creating the Initial Structure

   11. Step 11: Push the changes

6. What’s Next?

This is the second issue of the Building a Production-Ready Agentic Chatbot Course. You can read the previous issues from here:

• Part 1: Introduction & Course Outline

• Part 2: System Architecture & Environment Setup [You are here!]

All the codes and data used in this course can be found in the course GitHub Repo

Get All My 8 Books With 60% Off

1. BIM and IFC Fundamentals

Get 50% off for 1 year

Before we design the system, we need to understand what we’re working with. If you’ve never encountered BIM or IFC files before, don’t worry—we’ll cover exactly what you need to know.

1.1. What is BIM?

Get 50% off for 1 year

Building Information Modeling (BIM) is a digital representation of the physical and functional characteristics of a building or infrastructure. Think of it as a highly detailed, intelligent 3D model that contains not just geometry, but rich data about every component.

Unlike traditional CAD drawings that are just lines and shapes, a BIM model knows that a wall is a wall, what material it’s made of, how thick it is, what room it separates, what fire rating it has, and which contractor is responsible for building it. Every door, beam, window, pipe, and cable has properties, relationships, and metadata.

BIM is the backbone of modern construction and infrastructure projects. Architects, engineers, contractors, and building owners all work with BIM models to design, analyze, construct, and maintain buildings and infrastructure.

Get All My 8 Books With 60% Off

1.2. What are IFC Files?

Get 50% off for 1 year

Industry Foundation Classes (IFC) is the open standard file format for BIM. It’s maintained by buildingSMART International and designed to enable interoperability between different BIM software tools.

An IFC file is essentially a database stored as a file. It contains:

• Geometric data: The 3D shapes of building elements

• Semantic data: What each element is (wall, door, column, etc.)

• Properties: Material, fire rating, thermal properties, manufacturer info

• Relationships: Which floor a room is on, which wall a window is in

• Metadata: Project info, ownership, timestamps, units

Here’s a simplified example of what IFC data looks like:

#1= IFCPROJECT('3MD_HkJ7X3Rw7jJCTkarBW',$,'Bridge Project',$,$,$,$,(#2),#3);
#10= IFCBEAM('2O2Fr$t4X7Zf8NOew3FKVS',$,'Main Beam',$,$,#11,#12,$);
#11= IFCLOCALPLACEMENT($,#20);
#12= IFCPRODUCTDEFINITIONSHAPE($,$,(#30));

Each line represents an entity (a project, a beam, a placement, etc.), and entities reference each other by ID. The structure is hierarchical and deeply interconnected.

1.3. Why is This Hard to Work With?

Get 50% off for 1 year

IFC files are powerful, but they’re not user-friendly. To extract information or make changes, you typically need:

1. Specialized BIM software (Revit, ArchiCAD, Tekla, etc.)

2. Deep understanding of the IFC schema (which has thousands of entity types)

3. Technical knowledge of how entities relate to each other

4. Programming skills, if you want to automate anything

For non-technical stakeholders—project managers, clients, contractors—getting a simple answer like “How many steel beams are in this bridge?” requires opening specialized software, navigating complex interfaces, and often asking a BIM coordinator for help.

This is the gap we’re addressing.

1.4. Natural Language Access to BIM Data

What if instead of navigating complex software, you could just ask:

• “How many steel beams are in the bridge?”

• “What’s the total weight of concrete in the foundation?”

• “Show me all elements on the second floor that are fire-rated.”

• “Change the material of beam B-104 to steel grade S355.”

This is what our conversational agent enables. It translates natural language into IFC operations and presents results in plain English. It democratizes access to BIM data.

Get All My 8 Books With 60% Off

2. Understanding the Problem Space

Get 50% off for 1 year

Let’s be specific about what we’re building and why it matters. IFC files contain valuable information, but accessing it requires expertise. On a typical construction project, you might have:

• Architects who need to check design compliance

• Engineers who need to analyze structural elements

• Contractors who need to understand what to build

• Project managers who need to track progress and quantities

• Clients who need to verify what they’re paying for

• Facility managers who need to understand the building for maintenance

Most of these people aren’t BIM specialists. They have questions, but getting answers currently requires either learning complex software or asking someone else to extract the data for them.

What Success Looks Like for this Project?

A successful system would allow any stakeholder to:

1. Query: Ask questions in natural language and get accurate answers

2. Explore: Follow up with clarifying questions and dive deeper

3. Modify: Make changes to the model through conversation

4. Verify: Check compliance and validate data

5. Automate: Run repetitive queries or modifications at scale

And all of this needs to work reliably, securely, and at scale, not just as a demo.

Get All My 8 Books With 60% Off

3. System Architecture

Get 50% off for 1 year

Now that we understand the problem, let’s design a system to solve it. We’ll build this in stages, but it’s important to see the complete picture first so every component makes sense in context.

3.1. High-Level Architecture.

Get 50% off for 1 year

Here’s the overall system architecture:

Let’s break down each major component and understand its role:

Get 50% off for 1 year

Read more

There are a lot of tutorials and blogs that show how to create a chatbot or wire up an LLM to a prompt. However, there is a shortage in resources that guide how to turn that into a reliable, maintainable system that handles real users, real data, and real constraints.

When you start building beyond a proof of concept, you quickly realize that a “working agent” is only a small part of the problem. You also need backend infrastructure, authentication and permissions, state management, streaming, error handling, observability, and a frontend that people can actually use.

This course focuses on that gap. We won’t stop at making an agent respond intelligently. We’ll build a complete, production-ready conversational system, end to end.

Throughout this course, we’ll build a BIM (Building Information Modeling) conversational agent that allows users to interact with complex building data using natural language.

You’ll learn how to build:

• A full-stack system: backend APIs, frontend interface, database, and infrastructure

• Production patterns: authentication, authorization, error handling, logging, and monitoring

• Scalable architecture: from a single-user setup to multi-tenant, multi-company usage

• Real-time streaming: for a better user experience and perceived performance

• Advanced AI workflows: RAG pipelines, OCR, query optimization, and custom tools

• Deployment workflows: containerization, CI/CD, and cloud deployment

• Incremental development: each part builds on the previous one with runnable code

By the end, you won’t just understand how conversational agents work, you’ll have built a system that could realistically be deployed and maintained. Let’s start with an overview of the course and cover in detail what you will learn in each part.

Get All My Books With 40% Off

Course Outline

Get 30% off forever

The course is organized into five main parts.

Part 1: System Architecture & Environment Setup

Get All My Books With 40% Off

We start with the foundation. Before writing agent logic, we design the system around it. In this part, we will cover:

• Production-oriented architecture design

• BIM and IFC domain fundamentals

• Technology stack selection and trade-offs

• Development environment setup (Docker, Git, IDEs)

• Project structure and organization

Part 2: Building the AI Engine

Get All My Books With 40% Off

This is where we build the core agent logic and reasoning layer. In this part, we will cover:

• Agent orchestration with LangChain / LangGraph

• Designing a RAG pipeline

• Text preprocessing, OCR, and chunking strategies

• Vector search with pgvector

• Query optimization techniques

• Custom tools for IFC operations using IfcOpenShell

• Conversation memory and state management

• Streaming model responses

Part 3: Backend Infrastructure

Get All My Books With 40% Off

A production agent needs a solid backend. In this part, we will cover:

• FastAPI application structure

• Database design with SQLAlchemy

• Authentication and authorization (OAuth, JWT)

• API design and versioning

• Streaming endpoints

• Background jobs and async processing

• Error handling and retries

• Logging, monitoring, and observability

• Basic security practices

Part 4: Frontend Interface

Get All My Books With 40% Off

Users need a clear, stable way to interact with the system. In this part, we will build:

• A React + TypeScript application

• Authentication flows and protected routes

• A real-time chat interface with streaming

• File upload and management UI

• Conversation history and state handling

• UI/UX patterns for conversational systems

• Responsive and accessible layouts

Part 5: Testing, Scaling & Deployment

Get All My Books With 40% Off

Finally, we prepare the system for real-world use. In this part, we will build:

• Unit, integration, and end-to-end testing

• Scaling to multi-tenant usage

• Company and user management

• Role-based access control (RBAC)

• Containerization with Docker

• Kubernetes basics

• CI/CD pipelines

• Cloud deployment

• Performance tuning and monitoring

Get 30% off forever

Course Prerequisites

Get All My Books With 40% Off

To get the most out of this course, you should be comfortable with intermediate Python and have a basic understanding of web development concepts such as HTTP, APIs, and how backend services communicate.

Familiarity with Git and working from the command line is important, and you should have a general idea of how large language models work, even if you haven’t built systems with them before.

Some experience with FastAPI or similar backend frameworks, basic React or JavaScript, Docker, SQL databases, or cloud platforms will be helpful, but it’s not required. These topics are introduced gradually, and you can follow along even if they’re new to you.

You don’t need any prior knowledge of BIM or IFC, experience building production systems, or a background in DevOps. Those aspects are covered as part of the learning journey, and the course assumes you’re here to learn them, not already master them.

Get 30% off forever

Expected Outcome

Get All My Books With 40% Off

By the end of the course, you will:

• Build a complete production system, not just a prototype

• Understand how to design systems that scale and remain maintainable

• Implement advanced AI workflows (RAG, tools, streaming)

• Deploy a real application to the cloud

• Own a strong portfolio project that demonstrates real-world skills

• Apply the same patterns to other domains (analytics, documents, support, etc.)

Most importantly, you’ll develop the confidence to take an AI product from idea to production.

How to get the most out of the course?

Get All My Books With 40% Off

This course is designed to be hands-on. The best way to follow it is to code as you read. Don’t rush through the articles or treat them as reference material only. You’ll learn the most by running the code yourself, encountering issues, and fixing them along the way. That friction is part of the learning process.

While the project focuses on the BIM and IFC domains, you shouldn’t feel limited by it. The architecture and patterns we build are intentionally domain-agnostic. You can adapt the same ideas to legal document analysis, medical report processing, code assistants, customer support systems, or any other problem you care about. Building a version that matters to you will make the course more engaging and more useful as a portfolio project.

Experimentation is encouraged. Change prompts, try different models, add or remove tools, and observe how the system behaves. These small experiments will help you understand not just how things work, but why certain design choices matter.

You’re also encouraged to engage with others following the course. Use GitHub Discussions and newsletter comments to ask questions, share progress, and learn from how others approach the same problems. Explaining your thinking and seeing alternative solutions is a powerful way to deepen your understanding.

Most importantly, try to think production-first as you build. Regularly ask yourself what happens when something fails, how the system would behave under load, and how you would debug issues in a real deployment. Developing this mindset is the real goal of the course.

Get 30% off forever

Frameworks and Tools

Get All My Books With 40% Off

Throughout this course, we will use a lot of frameworks and tools to bring our product to life.

1. AI & Agents

• LangChain / LangGraph

• pgvector

• OpenAI or other LLM providers

• Sentence Transformers

• IfcOpenShell

2. Backend

• FastAPI

• SQLAlchemy

• PostgreSQL

• Redis

• Pydantic

• Alembic

3. Frontend

• React + TypeScript

• Vite

• TanStack Query

• Tailwind CSS

• shadcn/ui

4. DevOps

• Docker & Docker Compose

• Kubernetes

• GitHub Actions

• Cloud deployment (AWS / Azure / GCP)

• Prometheus & Grafana

Course Resources & Logistics

Get All My Books With 40% Off

The new issue will be published every Sunday at 5:00 p.m. (EEST), allowing you to replicate the work and build your project throughout the week. Each article corresponds to a step towards our product, and the code will be updated to the course GitHub repository.

Discussions and questions happen in GitHub Discussions and newsletter comments. The course will be for To Data & Beyond paid subscribers. Make sure to subscribe to never miss any parts of the course. You can get a 30% discount on monthly and yearly subscriptions from here.

Get 30% off forever

Over the next several months, we’ll move from architecture decisions to deployment, step by step.

The goal isn’t to build the perfect system. It’s to understand how real conversational AI systems are designed, built, and maintained.

Next up: Designing the system architecture and setting up the working environment.

See you next Sunday!

This newsletter is a personal passion project, and your support helps keep it alive. If you would like to contribute, there are a few great ways:

• Subscribe. A paid subscription helps to make my writing sustainable and gives you access to additional content.*

• Grab a copy of my book Bundle. Get my 8 hands-on books and roadmaps for only 50% of the price

Thanks for reading, and for helping support independent writing and research!

Are you looking to start a career in data science and AI, but do not know how? I offer data science mentoring sessions and long-term career mentoring:

• 1-1 Mentoring sessions

• Long-term mentoring