Bygg et 256K kontekst Qwen3 RAG-system som overgår GPT-4 (komplett veiledning)

Alibaba 's de nyeste Qwen3-modellene har skikkelig kraft med 256K kontekstvinduer og flerspråklig støtte på tvers av 119 språk. Denne trinnvise veiledningen viser deg hvordan du bygger et produksjonsklart RAG-system ved hjelp av Qwen3-4B-Instruct-2507, Qwen3-Embedding-0.6B og Qwen3-Reranker-4B som kjører effektivt på Google Colab eller lokal maskinvare.

Vi skal lage en finansiell forskningsassistent som kan svare på komplekse investeringsspørsmål ved hjelp av en samling av finansielle dokumenter. Den komplette pipelinen inkluderer dokumentchunking, semantisk søk ​​med FAISS, omrangering for presisjon og svargenerering med riktige sitater.

Hvorfor Qwen3 RAG fungerer bedre

Qwen3-4B-Instruct-2507 håndterer 262,144 3 tokens direkte, noe som eliminerer problemene med kontekstavkorting som plager mindre modeller. Kombinert med Qwen0.6-Embedding-XNUMXB. flerspråklige innebygginger og Qwen3-Reranker-4B's binært poengsystem, denne stakken gir nøyaktighet i bedriftsklassen mens den kjører på moderat maskinvare.

Arkitekturen bruker tre spesialiserte modeller: innebyggingsmodellen koder dokumenter og spørringer til 1024-dimensjonale vektorer, FAISS utfører et omtrentlig søk etter nærmeste nabo, omrangeringen scorer relevans ved hjelp av ja/nei-sannsynligheter, og instruksjonsmodellen syntetiserer svar fra topprangerte kontekster.

Oppsettskrav

Installer de viktigste avhengighetene for denne veiledningen. Sørg for at du har Transformers versjon 4.51.0 eller høyere for å unngå problemet «KeyError: 'qwen3′»:

python

pip install transformers>=4.51.0 torch faiss-cpu numpy tqdm

Du trenger en T4 GPU eller bedre for optimal ytelse. Innebyggingsmodellen kjører komfortabelt på CPU, men 4B instruct- og reranker-modellene drar nytte av GPU-akselerasjon.

Last inn instruksjonsfølgende modell som vil generere våre endelige svar. Denne modellen støtter en innebygd kontekstlengde på 262K og utmerker seg ved oppgaver med økonomisk resonnement:

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "Qwen/Qwen3-4B-Instruct-2507"
tokenizer = AutoTokenizer.from_pretrained(model_name)
instruct_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

# Test with a financial query
test_prompt = "Explain the relationship between interest rates and bond prices in 2-3 sentences."
messages = [{"role": "user", "content": test_prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tokenizer([text], return_tensors="pt").to(instruct_model.device)
outputs = instruct_model.generate(**inputs, max_new_tokens=256)
response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(response)

Utgang:

tekst

Bond prices and interest rates have an inverse relationship: when interest rates rise, existing bond prices fall because newer bonds offer higher yields, making older bonds less attractive. Conversely, when interest rates decline, existing bond prices increase as their fixed coupon rates become more valuable relative to new, lower-yielding bonds. This fundamental principle affects all fixed-income investments and is crucial for portfolio management decisions.

Innbyggingsmodellen konverterer tekst til tette vektorer for semantisk likhetsmatching. Denne modellen støtter opptil 32K kontekstlengde og fungerer på tvers av over 100 språk:

python

import torch.nn.functional as F
from transformers import AutoModel

embed_name = "Qwen/Qwen3-Embedding-0.6B"
embed_tokenizer = AutoTokenizer.from_pretrained(embed_name, padding_side='left')
embed_model = AutoModel.from_pretrained(embed_name, torch_dtype="auto", device_map="auto")

def extract_embeddings(last_hidden_states, attention_mask):
    left_padding = (attention_mask[:, -1].sum() == attention_mask.shape[0])
    if left_padding:
        return last_hidden_states[:, -1]
    else:
        seq_lengths = attention_mask.sum(dim=1) - 1
        batch_size = last_hidden_states.shape[0]
        return last_hidden_states[torch.arange(batch_size, device=last_hidden_states.device), seq_lengths]

# Financial document examples
financial_docs = [
    "Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.",
    "Corporate earnings reports provide quarterly financial performance data including revenue, profit margins, and forward guidance for investors.",
    "The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.",
    "Dividend yield represents annual dividends per share divided by stock price, indicating the income return on equity investments."
]

# Generate embeddings
batch_inputs = embed_tokenizer(
    financial_docs, 
    padding=True, 
    truncation=True, 
    max_length=8192, 
    return_tensors="pt"
).to(embed_model.device)

with torch.no_grad():
    outputs = embed_model(**batch_inputs)

doc_embeddings = extract_embeddings(outputs.last_hidden_state, batch_inputs['attention_mask'])
doc_embeddings = F.normalize(doc_embeddings, p=2, dim=1)

# Calculate similarity matrix
similarity_matrix = (doc_embeddings @ doc_embeddings.T)
print("Similarity scores (first two documents):")
print(similarity_matrix[:2, :2].tolist())

Utgang:

tekst

Similarity scores (first two documents):
[[1.0000001192092896, 0.4892156124114990], [0.4892156124114990, 1.0000001192092896]]

FAISS muliggjør effektivt likhetssøk på tvers av store dokumentsamlinger ved hjelp av algoritmer for omtrentlig nærmeste nabo:

python

import faiss
import numpy as np

# Create FAISS index
embedding_dim = doc_embeddings.shape[1]
faiss_index = faiss.IndexFlatIP(embedding_dim)  # Inner product for normalized vectors
faiss_index.add(doc_embeddings.cpu().numpy())

# Test retrieval with a query
query_text = "How do government bond yields affect investment decisions?"
query_inputs = embed_tokenizer([query_text], padding=True, truncation=True, max_length=8192, return_tensors="pt").to(embed_model.device)

with torch.no_grad():
    query_outputs = embed_model(**query_inputs)

query_embedding = extract_embeddings(query_outputs.last_hidden_state, query_inputs['attention_mask'])
query_embedding = F.normalize(query_embedding, p=2, dim=1)

# Retrieve top 3 most similar documents
scores, indices = faiss_index.search(query_embedding.cpu().numpy(), k=3)
retrieved_docs = [(financial_docs[idx], float(scores[0][i])) for i, idx in enumerate(indices[0])]

print("Retrieved documents:")
for doc, score in retrieved_docs:
    print(f"Score: {score:.4f} - {doc}")

Utgang:

tekst

Retrieved documents:
Score: 0.6234 - Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.
Score: 0.5891 - The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.
Score: 0.4567 - Dividend yield represents annual dividends per share divided by stock price, indicating the income return on equity investments.

Ocuco reranker modell scorer spørring-dokument-par ved hjelp av et binært ja/nei-format, noe som gir en mer nøyaktig relevansrangering enn bare cosinuslikhet:

python

reranker_name = "Qwen/Qwen3-Reranker-4B"
rerank_tokenizer = AutoTokenizer.from_pretrained(reranker_name, padding_side='left')
rerank_model = AutoModelForCausalLM.from_pretrained(reranker_name, torch_dtype="auto", device_map="auto").eval()

# Get token IDs for yes/no scoring
no_token_id = rerank_tokenizer.convert_tokens_to_ids("no")
yes_token_id = rerank_tokenizer.convert_tokens_to_ids("yes")

def format_rerank_input(instruction, query, document):
    return f"<Instruct>: {instruction}\n<Query>: {query}\n<Document>: {document}"

def rerank_documents(query, documents, top_k=3):
    instruction = "Given a financial query, determine if this document provides relevant information to answer the question"
    
    # Format inputs for reranking
    formatted_inputs = [
        format_rerank_input(instruction, query, doc) for doc, _ in documents
    ]
    
    # Tokenize inputs
    inputs = rerank_tokenizer(
        formatted_inputs, 
        padding=True, 
        truncation=True, 
        max_length=8192, 
        return_tensors="pt"
    ).to(rerank_model.device)
    
    # Get relevance scores
    with torch.no_grad():
        logits = rerank_model(**inputs).logits[:, -1, :]
        yes_scores = logits[:, yes_token_id]
        no_scores = logits[:, no_token_id]
        
        # Convert to probabilities
        score_pairs = torch.stack([no_scores, yes_scores], dim=1)
        probabilities = torch.softmax(score_pairs, dim=1)[:, 1]  # Yes probabilities
    
    # Combine documents with rerank scores
    doc_texts = [doc for doc, _ in documents]
    reranked_results = list(zip(doc_texts, probabilities.tolist()))
    reranked_results.sort(key=lambda x: x[1], reverse=True)
    
    return reranked_results[:top_k]

# Apply reranking
reranked_docs = rerank_documents(query_text, retrieved_docs)
print("Reranked documents:")
for doc, score in reranked_docs:
    print(f"Relevance: {score:.4f} - {doc}")

Utgang:

tekst

Reranked documents:
Relevance: 0.8942 - Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.
Relevance: 0.8156 - The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.
Relevance: 0.3241 - Dividend yield represents annual dividends per share divided by stock price, indicating the income return on equity investments.

Kombiner alle komponentene i én funksjon som håndterer hele arbeidsflyt for generering med utvidet henting:

python

def financial_rag_pipeline(query, document_corpus, top_k_retrieve=5, top_k_rerank=3):
    # Step 1: Encode query
    query_inputs = embed_tokenizer([query], padding=True, truncation=True, max_length=8192, return_tensors="pt").to(embed_model.device)
    
    with torch.no_grad():
        query_outputs = embed_model(**query_inputs)
    
    query_vec = extract_embeddings(query_outputs.last_hidden_state, query_inputs['attention_mask'])
    query_vec = F.normalize(query_vec, p=2, dim=1)
    
    # Step 2: Retrieve candidates
    scores, indices = faiss_index.search(query_vec.cpu().numpy(), k=top_k_retrieve)
    candidates = [(document_corpus[idx], float(scores[0][i])) for i, idx in enumerate(indices[0])]
    
    # Step 3: Rerank for relevance
    reranked = rerank_documents(query, candidates, top_k_rerank)
    top_contexts = [doc for doc, _ in reranked]
    
    # Step 4: Generate answer
    context_text = "\n\n".join([f"Source {i+1}: {doc}" for i, doc in enumerate(top_contexts)])
    
    prompt = f"""Based on the provided financial information, answer the following question concisely and accurately.

Question: {query}

Context:
{context_text}

Answer: Provide a clear, factual response based on the sources above."""

    messages = [{"role": "user", "content": prompt}]
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    
    inputs = tokenizer([text], return_tensors="pt").to(instruct_model.device)
    outputs = instruct_model.generate(**inputs, max_new_tokens=512, temperature=0.7)
    answer = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
    
    return answer, top_contexts

# Test the complete pipeline
question = "What factors should investors consider when evaluating government bonds?"
answer, sources = financial_rag_pipeline(question, financial_docs)

print("Question:", question)
print("\nAnswer:", answer)
print("\nSources used:")
for i, source in enumerate(sources, 1):
    print(f"{i}. {source}")

Utgang:

tekst

Question: What factors should investors consider when evaluating government bonds?

Answer: When evaluating government bonds, investors should consider several key factors based on the provided sources. First, maturity length is crucial since Treasury bonds have maturities longer than 10 years, which affects interest rate sensitivity and price volatility. Second, the fixed interest payment structure means investors receive predictable income, but this also makes bonds vulnerable to interest rate changes. Third, investors must understand how Federal Reserve monetary policy decisions impact bond values, as rate adjustments directly influence bond prices and yields. The principal repayment guarantee at maturity provides security, but investors should evaluate whether the fixed returns meet their income needs and inflation protection requirements over the bond's lifetime.

Sources used:
1. Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.
2. The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.
3. Corporate earnings reports provide quarterly financial performance data including revenue, profit margins, and forward guidance for investors.

💡 Tips for ytelsesoptimalisering

For produksjonsdistribusjon bør du vurdere disse forbedringene for å forbedre hastighet og nøyaktighet. Bruk batchbehandling for flere spørringer, implementer mellomlagring for ofte brukte innebygde elementer, og juster chunkstørrelsen mellom 400–800 tokener for optimal presisjon ved henting.

Kontekstvinduet på 262K i Qwen3-4B-Instruct-2507 lar deg inkludere flere hentede dokumenter uten avkorting, vanligvis 8–12 passasjer mot 3–5 for mindre modeller. Overvåk GPU-minnebruk og reduser max_length hvis du støter på feil på grunn av lite minne.

📋 Evaluering og kvalitetskontroll

Test din RAG-systemet bruke trofasthetsmålinger for å sikre at svarene forblir forankret i kildematerialet. Sammenlign resultater med og uten omrangering for å måle forbedringen i svarrelevans.

For økonomiske applikasjoner, valider numerisk nøyaktighet og sørg for korrekt sitering av regulatorisk informasjon. Omrangeringstrinnet forbedrer vanligvis svarkvaliteten med 15–25 % sammenlignet med ren innebyggingsbasert henting.

Denne Qwen3 RAG-implementeringen gir ytelse i bedriftsklassen med flerspråklig støtte og håndtering av lang kontekst. Kombinasjonen av spesialisert innebygging, omrangering og generasjonsmodeller skaper et robust system som skalerer effektivt på tvers av domener, samtidig som det opprettholder nøyaktighet og hastighet.