256K kontekstiga Qwen3 RAG-süsteemi loomine, mis ületab GPT-4 jõudlust (täielik õpetus)

Alibaba's Uusimad Qwen3 mudelid pakuvad tõsist võimsust 256K kontekstiakent ja mitmekeelne tugi 119 keeles. See samm-sammult juhend näitab teile, kuidas luua Qwen3-4B-Instruct-2507, Qwen3-Embedding-0.6B ja Qwen3-Reranker-4B abil tootmisvalmis RAG-süsteem, mis töötab tõhusalt Google Colabis või kohalikus riistvaras.

Loome finantsuuringute assistendi, mis suudab vastata keerukatele investeerimisküsimustele, kasutades finantsdokumentide korpust. Täielik töövoog hõlmab dokumentide tükeldamist, semantilist otsingut FAISS-i abil, täpsuse suurendamiseks ümberjärjestamist ja vastuste genereerimist koos õigete viidetega.

Miks Qwen3 RAG paremini toimib

Qwen3-4B-Instruct-2507 haldab natiivselt 262,144 3 tokenit, kõrvaldades konteksti kärpimise probleemid, mis vaevavad väiksemaid mudeleid. Koos Qwen0.6-Embedding-XNUMXB-dega mitmekeelsed manustused ja Qwen3-Reranker-4B's binaarne punktisüsteem, see pinu pakub ettevõtte tasemel täpsust tagasihoidlikul riistvaral töötades.

Arhitektuur kasutab kolme spetsialiseeritud mudelit: manustamismudel kodeerib dokumendid ja päringud 1024-mõõtmelisteks vektoriteks, FAISS teostab ligikaudse lähima naabri otsingu, ümberjärjestaja hindab asjakohasust jah/ei tõenäosuste abil ja instruct-mudel sünteesib vastuseid kõrgeima asetusega kontekstidest.

Seadistamise nõuded

Paigalda selle õpetuse jaoks vajalikud sõltuvused. Veendu, et sul on Transformersi versioon 4.51.0 või uuem, et vältida „KeyError: 'qwen3′” probleemi:

püüton

pip install transformers>=4.51.0 torch faiss-cpu numpy tqdm

Optimaalse jõudluse saavutamiseks on vaja T4 või paremat graafikakaarti. Manustusmudel töötab protsessoril mugavalt, kuid 4B käskude ja ümberjärjestamise mudelid saavad kasu graafikakaardi kiirendusest.

Laadige käskude järgimise mudel, mis genereerib meie lõplikud vastused. See mudel toetab natiivset 262K konteksti pikkust ja paistab silma järgmiselt: finantsarutluste ülesanded:

püüton

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "Qwen/Qwen3-4B-Instruct-2507"
tokenizer = AutoTokenizer.from_pretrained(model_name)
instruct_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

# Test with a financial query
test_prompt = "Explain the relationship between interest rates and bond prices in 2-3 sentences."
messages = [{"role": "user", "content": test_prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tokenizer([text], return_tensors="pt").to(instruct_model.device)
outputs = instruct_model.generate(**inputs, max_new_tokens=256)
response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(response)

Väljund:

tekst

Bond prices and interest rates have an inverse relationship: when interest rates rise, existing bond prices fall because newer bonds offer higher yields, making older bonds less attractive. Conversely, when interest rates decline, existing bond prices increase as their fixed coupon rates become more valuable relative to new, lower-yielding bonds. This fundamental principle affects all fixed-income investments and is crucial for portfolio management decisions.

Manustamismudel teisendab teksti tihedateks vektoriteks semantilise sarnasuse sobitamiseks. See mudel toetab kuni 32K konteksti pikkust ja töötab enam kui 100 keeles:

püüton

import torch.nn.functional as F
from transformers import AutoModel

embed_name = "Qwen/Qwen3-Embedding-0.6B"
embed_tokenizer = AutoTokenizer.from_pretrained(embed_name, padding_side='left')
embed_model = AutoModel.from_pretrained(embed_name, torch_dtype="auto", device_map="auto")

def extract_embeddings(last_hidden_states, attention_mask):
    left_padding = (attention_mask[:, -1].sum() == attention_mask.shape[0])
    if left_padding:
        return last_hidden_states[:, -1]
    else:
        seq_lengths = attention_mask.sum(dim=1) - 1
        batch_size = last_hidden_states.shape[0]
        return last_hidden_states[torch.arange(batch_size, device=last_hidden_states.device), seq_lengths]

# Financial document examples
financial_docs = [
    "Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.",
    "Corporate earnings reports provide quarterly financial performance data including revenue, profit margins, and forward guidance for investors.",
    "The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.",
    "Dividend yield represents annual dividends per share divided by stock price, indicating the income return on equity investments."
]

# Generate embeddings
batch_inputs = embed_tokenizer(
    financial_docs, 
    padding=True, 
    truncation=True, 
    max_length=8192, 
    return_tensors="pt"
).to(embed_model.device)

with torch.no_grad():
    outputs = embed_model(**batch_inputs)

doc_embeddings = extract_embeddings(outputs.last_hidden_state, batch_inputs['attention_mask'])
doc_embeddings = F.normalize(doc_embeddings, p=2, dim=1)

# Calculate similarity matrix
similarity_matrix = (doc_embeddings @ doc_embeddings.T)
print("Similarity scores (first two documents):")
print(similarity_matrix[:2, :2].tolist())

Väljund:

tekst

Similarity scores (first two documents):
[[1.0000001192092896, 0.4892156124114990], [0.4892156124114990, 1.0000001192092896]]

FAISS võimaldab tõhusat sarnasuse otsingut suurtes dokumendikogudes, kasutades ligikaudseid lähima naabri algoritme:

püüton

import faiss
import numpy as np

# Create FAISS index
embedding_dim = doc_embeddings.shape[1]
faiss_index = faiss.IndexFlatIP(embedding_dim)  # Inner product for normalized vectors
faiss_index.add(doc_embeddings.cpu().numpy())

# Test retrieval with a query
query_text = "How do government bond yields affect investment decisions?"
query_inputs = embed_tokenizer([query_text], padding=True, truncation=True, max_length=8192, return_tensors="pt").to(embed_model.device)

with torch.no_grad():
    query_outputs = embed_model(**query_inputs)

query_embedding = extract_embeddings(query_outputs.last_hidden_state, query_inputs['attention_mask'])
query_embedding = F.normalize(query_embedding, p=2, dim=1)

# Retrieve top 3 most similar documents
scores, indices = faiss_index.search(query_embedding.cpu().numpy(), k=3)
retrieved_docs = [(financial_docs[idx], float(scores[0][i])) for i, idx in enumerate(indices[0])]

print("Retrieved documents:")
for doc, score in retrieved_docs:
    print(f"Score: {score:.4f} - {doc}")

Väljund:

tekst

Retrieved documents:
Score: 0.6234 - Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.
Score: 0.5891 - The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.
Score: 0.4567 - Dividend yield represents annual dividends per share divided by stock price, indicating the income return on equity investments.

. ümberjärjestaja mudel hindab päringu-dokumendi paare binaarse jah/ei vormingu abil, pakkudes täpsemat asjakohasuse järjestust kui ainult koosinussarnasus:

püüton

reranker_name = "Qwen/Qwen3-Reranker-4B"
rerank_tokenizer = AutoTokenizer.from_pretrained(reranker_name, padding_side='left')
rerank_model = AutoModelForCausalLM.from_pretrained(reranker_name, torch_dtype="auto", device_map="auto").eval()

# Get token IDs for yes/no scoring
no_token_id = rerank_tokenizer.convert_tokens_to_ids("no")
yes_token_id = rerank_tokenizer.convert_tokens_to_ids("yes")

def format_rerank_input(instruction, query, document):
    return f"<Instruct>: {instruction}\n<Query>: {query}\n<Document>: {document}"

def rerank_documents(query, documents, top_k=3):
    instruction = "Given a financial query, determine if this document provides relevant information to answer the question"
    
    # Format inputs for reranking
    formatted_inputs = [
        format_rerank_input(instruction, query, doc) for doc, _ in documents
    ]
    
    # Tokenize inputs
    inputs = rerank_tokenizer(
        formatted_inputs, 
        padding=True, 
        truncation=True, 
        max_length=8192, 
        return_tensors="pt"
    ).to(rerank_model.device)
    
    # Get relevance scores
    with torch.no_grad():
        logits = rerank_model(**inputs).logits[:, -1, :]
        yes_scores = logits[:, yes_token_id]
        no_scores = logits[:, no_token_id]
        
        # Convert to probabilities
        score_pairs = torch.stack([no_scores, yes_scores], dim=1)
        probabilities = torch.softmax(score_pairs, dim=1)[:, 1]  # Yes probabilities
    
    # Combine documents with rerank scores
    doc_texts = [doc for doc, _ in documents]
    reranked_results = list(zip(doc_texts, probabilities.tolist()))
    reranked_results.sort(key=lambda x: x[1], reverse=True)
    
    return reranked_results[:top_k]

# Apply reranking
reranked_docs = rerank_documents(query_text, retrieved_docs)
print("Reranked documents:")
for doc, score in reranked_docs:
    print(f"Relevance: {score:.4f} - {doc}")

Väljund:

tekst

Reranked documents:
Relevance: 0.8942 - Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.
Relevance: 0.8156 - The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.
Relevance: 0.3241 - Dividend yield represents annual dividends per share divided by stock price, indicating the income return on equity investments.

Kombineeri kõik komponendid üheks funktsiooniks, mis haldab kõiki otsingu abil laiendatud genereerimise töövoog:

püüton

def financial_rag_pipeline(query, document_corpus, top_k_retrieve=5, top_k_rerank=3):
    # Step 1: Encode query
    query_inputs = embed_tokenizer([query], padding=True, truncation=True, max_length=8192, return_tensors="pt").to(embed_model.device)
    
    with torch.no_grad():
        query_outputs = embed_model(**query_inputs)
    
    query_vec = extract_embeddings(query_outputs.last_hidden_state, query_inputs['attention_mask'])
    query_vec = F.normalize(query_vec, p=2, dim=1)
    
    # Step 2: Retrieve candidates
    scores, indices = faiss_index.search(query_vec.cpu().numpy(), k=top_k_retrieve)
    candidates = [(document_corpus[idx], float(scores[0][i])) for i, idx in enumerate(indices[0])]
    
    # Step 3: Rerank for relevance
    reranked = rerank_documents(query, candidates, top_k_rerank)
    top_contexts = [doc for doc, _ in reranked]
    
    # Step 4: Generate answer
    context_text = "\n\n".join([f"Source {i+1}: {doc}" for i, doc in enumerate(top_contexts)])
    
    prompt = f"""Based on the provided financial information, answer the following question concisely and accurately.

Question: {query}

Context:
{context_text}

Answer: Provide a clear, factual response based on the sources above."""

    messages = [{"role": "user", "content": prompt}]
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    
    inputs = tokenizer([text], return_tensors="pt").to(instruct_model.device)
    outputs = instruct_model.generate(**inputs, max_new_tokens=512, temperature=0.7)
    answer = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
    
    return answer, top_contexts

# Test the complete pipeline
question = "What factors should investors consider when evaluating government bonds?"
answer, sources = financial_rag_pipeline(question, financial_docs)

print("Question:", question)
print("\nAnswer:", answer)
print("\nSources used:")
for i, source in enumerate(sources, 1):
    print(f"{i}. {source}")

Väljund:

Tekst

Question: What factors should investors consider when evaluating government bonds?

Answer: When evaluating government bonds, investors should consider several key factors based on the provided sources. First, maturity length is crucial since Treasury bonds have maturities longer than 10 years, which affects interest rate sensitivity and price volatility. Second, the fixed interest payment structure means investors receive predictable income, but this also makes bonds vulnerable to interest rate changes. Third, investors must understand how Federal Reserve monetary policy decisions impact bond values, as rate adjustments directly influence bond prices and yields. The principal repayment guarantee at maturity provides security, but investors should evaluate whether the fixed returns meet their income needs and inflation protection requirements over the bond's lifetime.

Sources used:
1. Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.
2. The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.
3. Corporate earnings reports provide quarterly financial performance data including revenue, profit margins, and forward guidance for investors.

💡 Toimivuse optimeerimise näpunäited

Tootmiskeskkonnas juurutamiseks kaaluge neid täiustusi kiiruse ja täpsuse parandamiseks. Kasutage mitme päringu puhul partiitöötlust, rakendage vahemällu salvestamist sageli kasutatavate manustuste jaoks ja reguleerige tüki suurust vahemikus 400–800 märki, et saavutada optimaalne otsingu täpsus.

Qwen262-3B-Instruct-4 2507K kontekstiaken võimaldab teil kärpimata lisada rohkem hangitud dokumente, tavaliselt 8–12 lõiku võrreldes väiksemate mudelite 3–5 lõiguga. GPU mälukasutuse jälgimine ja vähendage max_length väärtust, kui ilmnevad mälu otsasaamise vead.

📋 Hindamine ja kvaliteedikontroll

Testi oma RAG-süsteem Kasutades usaldusväärsuse mõõdikuid, et tagada vastuste vastavus allikmaterjalile. Võrrelge tulemusi ümberjärjestamisega ja ilma selleta, et mõõta vastuste asjakohasuse paranemist.

Finantsrakenduste puhul valideerige numbriline täpsus ja tagage regulatiivse teabe korrektne viitamine. Ümberjärjestamise etapp parandab vastuste kvaliteeti tavaliselt 15–25% võrreldes puhta manustamisel põhineva otsinguga.

See Qwen3 RAG-i implementatsioon pakub ettevõtte tasemel jõudlust mitmekeelse toe ja pika konteksti käsitlemisega. Spetsialiseeritud manustamise, ümberjärjestamise ja põlvkonna mudelid loob tugeva süsteemi, mis skaleerub tõhusalt eri valdkondades, säilitades samal ajal täpsuse ja kiiruse.