Building a Personal Blog RAG Knowledge Base with LangChain + Milvus + Alibaba Cloud Bailian

目录

Preface

After so many years of blogging, I’ve accumulated nearly 800 posts covering frontend, backend, DevOps, automation, AI, and many other fields. But the problem is—even I often can’t find things I’ve written before. Every time I want to look up the implementation approach for some technical point, I have to dig through directories and search keywords, which is extremely inefficient.

RAG (Retrieval-Augmented Generation) has been all the rage lately, and I thought—could I turn my own blog into a dedicated knowledge-base Q&A system? No sooner said than done. I spent an afternoon building a complete solution based on LangChain + Milvus + Alibaba Cloud Bailian, and the results were unexpectedly good.

Technology Choices

Component	Choice	Notes
Vector database	Milvus 2.6	High performance, supports distribution, deployed on the LAN via K8s
Embedding	Alibaba Cloud Bailian text-embedding-v3	1024 dimensions, excellent Chinese performance
LLM	Alibaba Cloud Bailian qwen-turbo	Streaming output, fast response
Framework	LangChain	Handles document splitting, vector retrieval, and prompt engineering
Backend	FastAPI	Lightweight and efficient, with native SSE streaming support
Frontend	Plain HTML/CSS/JS	In the clean style of Zhihu Direct Answer

Overall Architecture

┌─────────────────┐     ┌──────────────┐     ┌─────────────────┐
│  User question   │────▶│  FastAPI     │────▶│  Milvus vector retrieval │
└─────────────────┘     └──────────────┘     └─────────────────┘
                               │
                               ▼
                        ┌──────────────┐
                        │  Alibaba Cloud Bailian │
                        │  qwen-turbo  │
                        └──────────────┘
                               │
                               ▼
                        ┌──────────────┐
                        │  SSE streaming output │
                        └──────────────┘

Core Implementation

1. Importing Blog Documents into Milvus

First I needed to clean, split, and vectorize the blog’s Markdown files, then store them in Milvus.

from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.embeddings import DashScopeEmbeddings
from pymilvus import MilvusClient, DataType

# Clean Markdown: strip YAML frontmatter and hexo tags, convert to plain text
def extract_text_from_markdown(md_content: str) -> str:
    md_content = re.sub(r'^---\s*\n.*?---\s*\n', '', md_content, flags=re.DOTALL)
    md_content = re.sub(r'{%.*?%}', '', md_content, flags=re.DOTALL)
    html = markdown(md_content)
    soup = BeautifulSoup(html, "html.parser")
    return soup.get_text(separator="\n").strip()

# Text splitting: split by heading level first to keep semantics coherent
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=800,
    chunk_overlap=150,
    separators=["\n## ", "\n### ", "\n\n", "\n", "。", "；", " ", ""],
)

# Batch-generate embeddings and insert into Milvus (25 per batch)
for i in range(0, len(split_docs), 25):
    batch = split_docs[i:i + 25]
    vectors = embeddings.embed_documents([d.page_content for d in batch])
    client.insert(collection_name="blog_knowledge", data=[...])

2. Two-Stage Retrieval + Rerank

Direct top-5 vector retrieval might not be precise enough, so I switched to a coarse-rank then fine-rank two-stage strategy:

1. Coarse rank: Recall 20 candidate documents from Milvus (COSINE similarity ≥ 0.45)
2. Fine rank: Re-rank the candidates with Alibaba Cloud Bailian’s gte-rerank model, taking the top 5

RETRIEVE_TOP_K = 20
RERANK_TOP_K = 5
SIMILARITY_THRESHOLD = 0.45

def retrieve_docs(query: str):
    # Stage 1: vector retrieval to recall candidates
    docs_with_score = vector_store.similarity_search_with_score(query, k=RETRIEVE_TOP_K)
    candidates = [(d, s) for d, s in docs_with_score if s >= SIMILARITY_THRESHOLD]
    if not candidates:
        return []
    
    # Stage 2: Rerank
    docs = [d for d, _ in candidates]
    resp = dashscope.TextReRank.call(
        model="gte-rerank",
        query=query,
        documents=[d.page_content for d in docs],
        top_n=RERANK_TOP_K,
        return_documents=False,
    )
    
    ranked = []
    for r in resp.output.results:
        idx = r["index"]
        if r["relevance_score"] >= 0.3:
            ranked.append(docs[idx])
    return ranked

After reranking, answer quality improved noticeably—especially for fuzzy queries, it can bring the truly relevant documents to the front.

3. Strict Prompt Engineering

This is the soul of the whole system—it must be restricted to answering only questions relevant to the knowledge base, to prevent LLM hallucination.

SYSTEM_PROMPT = """You are a professional blog knowledge-base Q&A assistant. Your only source of information is the blog knowledge-base snippets provided below.

[Rules]
1. You may only answer questions based on the provided knowledge-base snippets.
2. If the knowledge-base snippets contain no relevant information, or the user's question is completely unrelated to the blog content,
   you must reply directly: "This question is unrelated to the blog knowledge base, and I cannot answer it. Please try asking a question related to the blog content."
3. Do not fabricate information not present in the knowledge-base snippets.
4. Answer clearly, concisely, and in an organized way.

[Knowledge-base snippets]
{context}

Please answer the user's question based on the knowledge-base snippets above."""

4. Multi-Turn Dialogue and Session Memory

It supports multi-turn follow-ups. Once the session history exceeds 10 turns, it’s automatically compressed into a summary to avoid overly long context burning tokens.

MAX_HISTORY_TURNS = 10

def build_messages(session_id: str, user_question: str, context: str):
    session = sessions.setdefault(session_id, {"turns": [], "compressed_summary": ""})
    
    # Auto-compress history after 10 turns
    if len(session["turns"]) >= MAX_HISTORY_TURNS:
        summary = summarize_history(session["turns"])
        session["compressed_summary"] = summary
        session["turns"] = []
    
    messages = [{"role": "system", "content": SYSTEM_PROMPT.format(context=context)}]
    
    # Compressed summary + recent conversation history
    if session["compressed_summary"]:
        messages.append({"role": "system", "content": f"Summary of previous conversation: {session['compressed_summary']}"})
    for turn in session["turns"]:
        messages.append({"role": "user", "content": turn["user"]})
        messages.append({"role": "assistant", "content": turn["bot"]})
    messages.append({"role": "user", "content": user_question})
    
    return messages

5. FastAPI Streaming Interface

SSE (Server-Sent Events) is used to achieve a typewriter effect, making the experience more natural.

@app.post("/api/chat")
async def chat(request: ChatRequest):
    async def event_stream():
        docs = retrieve_docs(request.message)
        if not docs:
            yield f"data: {{'type': 'token', 'content': 'Refuse to answer'}}\n\n"
            return

        # Deduplicate sources to avoid citing the same article multiple times
        seen = set()
        sources = []
        for doc in docs:
            key = doc.metadata.get("source", "")
            if key and key not in seen:
                seen.add(key)
                sources.append({"title": ..., "source": key})

        async for chunk in llm.astream(messages):
            yield f"data: {{'type': 'token', 'content': chunk.content}}\n\n"

        yield f"data: {{'type': 'sources', 'sources': sources}}\n\n"
        yield "data: {'type': 'done'}\n\n"

    return StreamingResponse(event_stream(), media_type="text/event-stream")

6. Frontend Design

I referenced the clean style of Zhihu Direct Answer, implemented in plain HTML/CSS/JS with no framework dependencies:

• Homepage: Large title + rounded search box + recommended question tags
• Chat page: Collapsible history sidebar on the left + message area on the right
• Dark/light theme toggle: One-click switch in the top right, with state saved in localStorage
• Markdown rendering: Integrated marked.js, supporting code blocks, tables, lists, bold, etc.
• Message area scrolling: chat-messages has a fixed height and scrolls independently when content overflows, with the input box always pinned to the bottom

7. Self-Built Monitoring Dashboard

Not wanting to depend on third-party monitoring platforms like Langfuse, I built my own monitoring dashboard with SQLite + a pure frontend, with zero extra dependencies.

Each /api/chat request is automatically logged, recording:

• Time consumed at each stage (vector retrieval, rerank, LLM generation)
• Number of recalled documents and similarity scores
• Deduplicated source documents
• Input / Output token consumption
• Request status (success/refused/error)

The dashboard page /admin provides:

• Top stat cards: total requests, average time, average time per stage, total tokens
• Request list: with mini time-consumption bar charts, so you can spot the bottleneck at a glance
• Status filter + keyword search + pagination
• Click to expand details: time breakdown, source documents, token consumption, input/output content

Real-world data revealed that Input tokens far exceed Output tokens—because every request stuffs the system prompt + the context of 5 retrieved documents into the LLM together. This also explains why the number of vector-retrieved documents needs to be controlled: the longer the context, the higher the token cost.

Potholes I Stepped Into

1. Milvus’s Port Isn’t the Default 19530

My Milvus is exposed via a K8s NodePort, and the actual port is 30001. You can only confirm this with kubectl get svc:

$ kubectl get svc | grep milvus
milvus   NodePort   10.111.186.154   <none>   19530:30001/TCP,9091:30002/TCP

2. The Meaning of the Score in LangChain Milvus

For the COSINE metric, the score returned by LangChain’s similarity_search_with_score() is a similarity (larger means more similar), not a distance. At first I set a 0.4 threshold based on distance logic, which filtered out all the highly relevant documents.

3. Deduplicating Chunks from the Same Article

Once a blog post is split into multiple text blocks and stored in the vector store, retrieval may hit multiple chunks from the same article. If you don’t deduplicate, the source references will be displayed N times. The solution is to deduplicate the source path with a set() on the backend.

4. The Message Area Stretching the Page in a Flex Layout

At first, once there were many messages, they pushed the input box out of the viewport. The root cause is that in a Flex column layout, child elements default to min-height: auto, so even with flex: 1 and overflow-y: auto set, they won’t shrink.

The solution: add min-height: 0 to both chat-messages and its parent container main-area, and also set html, body to height: 100%; overflow: hidden to completely disable page-level scrolling.

5. Importing All 720 Articles Is Very Slow

720 articles split into 4400+ chunks, calling the Embedding API one by one, is extremely slow. The optimizations:

• DashScope’s embed_documents() already does batching internally
• Manually batch (25 per batch) and write directly to Milvus
• In the end, the full import takes about 6–8 minutes

Project Structure

blog_rag/
├── ingest.py                    # Blog import script
├── app.py                       # FastAPI backend
├── monitor.db                   # SQLite monitoring data (auto-generated)
├── static/
│   ├── index.html               # Frontend chat UI
│   └── admin.html               # Monitoring dashboard
├── langfuse-docker-compose.yml  # Langfuse self-hosted config (optional)
└── venv/                        # Python virtual environment

Validating the Results

• ✅ Relevant question: “How do I deploy Milvus?” → retrieves blog content and answers in detailed steps
• ❌ Irrelevant question: “What’s the weather like today?” → directly refuses: “This question is unrelated to the blog knowledge base…”

Next Steps

1. Full import: Currently only the latest 60 articles have been imported for testing; later I’ll load all 720
2. Image/link references: Images and external links mentioned in answers can be displayed as-is
3. Deploy and go live: Package with Docker Compose for easy migration

Summary

The whole project took less than half a day from zero to usable, thanks mainly to:

• LangChain abstracting the RAG flow very cleanly
• Milvus offering sufficiently strong vector retrieval performance
• Alibaba Cloud Bailian’s Embedding + LLM handled by one set of APIs, no need to juggle multiple platforms

What I’m most satisfied with is that prompt restriction—it makes the LLM honestly answer only things that are in the blog, without making things up. This is crucial when building a personal knowledge base.

中英对照

文章作者：阿文

文章链接： https://www.awen.me/post/a7103c45.html

版权声明：本博客所有文章除特别声明外，均采用 CC BY-NC-SA 4.0 许可协议。转载请注明来自 阿文的博客！