A Practical Guide to Dify External Knowledge Bases: Semantic Retrieval Based on Alibaba Cloud OSS Vector Search

目录

1. Why We Need an External Knowledge Base

Dify’s built-in knowledge base is great to use, but there’s one precondition: you have to upload your data to Dify’s servers. For scenarios where you already have massive amounts of documents stored on Alibaba Cloud OSS (like my blog articles or a company’s internal document library), re-uploading is neither elegant nor realistic.

The Dify External Knowledge Base API offers a perfect solution: Dify only takes responsibility for sending retrieval requests, while the actual knowledge base is maintained by your own service.

User question → Dify → your external knowledge base service → returns relevant documents → Dify passes them to the LLM

2. Comparing Two Retrieval Approaches

Before integrating vector retrieval, I first built a version using keyword matching. After comparing them, the gap was glaringly obvious:

Retrieval method	Result returned for query “grandma”	Principle
Keyword matching	“Let’s Talk About Hijacking.md”	“grand” and “ma” matched separately as single characters
Vector retrieval	“My Grandma Never Internalized Stress.md”	Semantic understanding of the concept “grandma”

The problem with keyword matching is that single-character matching in Chinese is far too prone to false hits. Almost every article contains the characters “grand” or “ma,” making the search results meaningless. Vector retrieval, based on semantic similarity, can genuinely understand the user’s query intent.

3. Alibaba Cloud OSS Vector Search

Alibaba Cloud OSS launched its vector search feature in 2024. Its core capabilities include:

• Automatic vectorization: Once enabled, OSS automatically extracts content features from text, images, audio, and video to generate vector indexes
• Semantic retrieval: Supports retrieval by natural-language description, such as “grandma,” “snowy outdoor surveillance,” or “2024 business analysis”
• Multimodal support: Unified retrieval across text, images, audio, and video
• Regional coverage: Already supported in major regions such as Hangzhou, Shanghai, Beijing, and Shenzhen

Configure the tag
bailian-datahub-access with value read

How to Enable It

In the OSS console → Data Indexing → select “Vector Search” mode to enable it. Once enabled, the system automatically scans existing files and builds the index; about 200 Markdown files are ready in just a few minutes.

Note: Vector search incurs API call charges. It’s recommended to evaluate the cost based on the actual volume of files.

4. Project Architecture

My project is based on the Go + Gin framework, with the overall architecture as follows:

Dify request
  │ POST /api/v1/external-kb/retrieval
  │ Authorization: Bearer {API_KEY}
  ▼
┌─────────────────────────────────────┐
│  ExternalKnowledgeHandler           │
│  ├─ Verify API Key                  │
│  ├─ Select retrieval mode per config│
│  │   ├─ semantic → OSSVectorService │
│  │   └─ keyword  → keyword matching │
│  └─ Download file content → return Dify format │
└─────────────────────────────────────┘
              │
    ┌─────────┴──────────┐
    ▼                    ▼
OSS SDK V2          OSS SDK V1
(vector search)     (file download)

Two Alibaba Cloud OSS SDKs are used here:

• V1 (aliyun-oss-go-sdk): Handles basic operations like file upload, download, and listing
• V2 (alibabacloud-oss-go-sdk-v2): Supports new features such as vector search

5. Core Code Implementation

5.1 Vector Retrieval Service Wrapper

type OSSVectorService struct {
    client     *oss.Client  // V2 SDK
    bucketName string
    region     string
}

func (s *OSSVectorService) SemanticSearch(ctx context.Context, query string, topK int64, mediaType string) ([]OSSVectorResult, error) {
    req := &oss.DoMetaQueryRequest{
        Bucket: oss.Ptr(s.bucketName),
        Mode:   oss.Ptr("semantic"),  // Key: semantic mode
        MetaQuery: &oss.MetaQuery{
            MaxResults: oss.Ptr(topK),
            Query:      oss.Ptr(query),
        },
    }
    
    if mediaType != "" {
        req.MetaQuery.MediaTypes = &oss.MetaQueryMediaTypes{
            MediaTypes: []string{mediaType},
        }
    }
    
    result, err := s.client.DoMetaQuery(ctx, req)
    // ... parse the returned file list
}

5.2 Integrating with the Dify API Spec

Dify external knowledge bases require implementing a standard retrieval interface:

Request:

{
    "knowledge_id": "awenblog-12",
    "query": "grandma",
    "retrieval_setting": {
        "top_k": 5,
        "score_threshold": 0.1
    }
}

Response:

{
    "records": [
        {
            "content": "Reading notes on 'My Grandma Never Internalized Stress'...",
            "score": 0.9,
            "title": "posts/2026/04/我的外婆从不内耗.md",
            "metadata": {
                "path": "posts/2026/04/我的外婆从不内耗.md",
                "url": "https://file.awen.me/..."
            }
        }
    ]
}

The core logic of the Handler:

func (h *ExternalKnowledgeHandler) Retrieval(c *gin.Context) {
    // 1. Verify API Key
    // 2. Parse the request
    // 3. Select the retrieval method based on SearchMode
    if h.cfg.SearchMode == "semantic" {
        records, err = h.semanticSearch(c, req.Query, req.RetrievalSetting)
    } else {
        records, err = h.keywordMatch(c, req.Query, req.RetrievalSetting)
    }
    // 4. Return the result
}

5.3 Cross-Bucket, Cross-Region Access

My blog articles are stored in the awenblog bucket in the Hangzhou region, while the project’s original configuration connects to file201503 in the Shanghai region. Here I need to dynamically create an OSS Client for the corresponding region:

func (s *OSSService) GetBucketWithRegion(bucketName, region, endpoint string) (*oss.Bucket, error) {
    // If the region is the same as the default config, reuse the existing client
    if region == "" || region == s.config.Region {
        return s.client.Bucket(bucketName)
    }
    
    // Otherwise create a new V4-signature Client
    client, err := oss.New(endpoint, accessKeyID, accessKeySecret, oss.AuthVersion(oss.AuthV4))
    client.SetRegion(region)
    return client.Bucket(bucketName)
}

6. Potholes I Stepped Into

Pothole 1: GOMODCACHE Permission Issue

During local compilation I got the error mkdir /usr/local/gopath: permission denied. The fix is to set a user-writable GOPATH:

export GOPATH=/home/wenjun/gopath
export GOMODCACHE=/home/wenjun/gopath/pkg/mod

Pothole 2: CGO and GLIBC Version Incompatibility

A binary compiled directly with go build threw GLIBC_2.34 not found when run on the server. Static compilation was required:

CGO_ENABLED=0 go build -ldflags='-s -w' -o bin/blog-api cmd/main.go

Pothole 3: Endpoint and Region Mismatch

awenblog is in the Hangzhou region, but the original Client connected to Shanghai. When EXTERNAL_KB_REGION isn’t configured correctly, you get AccessDenied: The bucket you are attempting to access must be addressed using the specified endpoint.

Pothole 4: Nginx 504 Timeout

Serially downloading 200 articles for keyword matching took longer than Nginx’s default timeout. The fix was to use goroutines for concurrent processing:

const maxWorkers = 15
fileCh := make(chan string, len(files))
resultCh := make(chan result, len(files))

for i := 0; i < maxWorkers; i++ {
    go func() {
        for file := range fileCh {
            // concurrent download and matching
        }
    }()
}

After the concurrency optimization, the response time dropped from several seconds to 0.2 seconds.

Pothole 5: Empty Array vs null

Dify requires {"records": []} to be returned when there are no matches, but Go’s nil slice serializes to null in JSON. I needed to add a fallback before returning:

if records == nil {
    records = []ExternalKBRecord{}
}

7. Config Quick Reference

# Dify external knowledge base config
EXTERNAL_KB_ENABLED=true
EXTERNAL_KB_API_KEY=sk-2esfdg3
EXTERNAL_KB_BUCKET=awenblog
EXTERNAL_KB_REGION=oss-cn-hangzhou
EXTERNAL_KB_PREFIX=
EXTERNAL_KB_SEARCH_MODE=semantic      # keyword | semantic
EXTERNAL_KB_MEDIA_TYPE=document       # image/video/audio/document

8. Showing the Results

Testing semantic retrieval with Postman:

curl -X POST https://api.api.com/api/v1/external-kb/retrieval \
  -H "Authorization: Bearer sk-2esfdg3" \
  -H "Content-Type: application/json" \
  -d '{
    "knowledge_id": "awenblog-12",
    "query": "grandma",
    "retrieval_setting": {"top_k": 5, "score_threshold": 0.1}
  }'

The returned result:

{
  "records": [
    {
      "content": "Reading notes on 'My Grandma Never Internalized Stress'...",
      "score": 0.9,
      "title": "posts/2026/04/我的外婆从不内耗.md",
      "metadata": {
        "path": "posts/2026/04/我的外婆从不内耗.md",
        "url": "https://xxx/..."
      }
    }
  ]
}

Then try searching “Docker,” “reading notes,” or “internalized stress”—they all return semantically relevant results, not literal hard matches.

9. Final Thoughts

Going from keyword matching to vector retrieval isn’t just a technical upgrade—it’s a leap from a “string game” to “semantic understanding.” When your knowledge base has hundreds or thousands of articles, users won’t remember each article’s title and keywords; they’ll just ask in natural language. Vector retrieval turns “can find it” into “finds the right thing.”

The advantage of Alibaba Cloud OSS vector search is zero operations: you don’t need to set up your own vector database, write embedding code, or handle vector index updates—OSS does all of it for you. For scenarios where you already heavily use OSS to store documents, this is one of the lowest-cost and best-performing RAG solutions.

The complete code is open-sourced in the blog_api project—feel free to reference it and discuss.

中英对照

文章作者：阿文

文章链接： https://www.awen.me/post/8f919b42.html

版权声明：本博客所有文章除特别声明外，均采用 CC BY-NC-SA 4.0 许可协议。转载请注明来自 阿文的博客！