Bumuo ng 256K Konteksto Qwen3 RAG System na Higit sa GPT-4 (Kumpletong Tutorial)

Ang mga pinakabagong modelo ng Qwen3 ng Alibaba ay may malaking kakayahan na may 256K context window at suporta sa maraming wika sa 119 na wika. Ipapakita sa iyo ng sunud-sunod na gabay na ito kung paano bumuo ng isang RAG system na handa nang gamitin para sa produksyon gamit ang Qwen3-4B-Instruct-2507, Qwen3-Embedding-0.6B, at Qwen3-Reranker-4B na mahusay na tumatakbo sa Google Colab o lokal na hardware.

Gagawa kami ng isang financial research assistant na makakasagot sa mga kumplikadong tanong sa pamumuhunan gamit ang isang corpus ng mga pinansyal na dokumento. Ang kumpletong pipeline ay kinabibilangan ng document chunking, semantic search sa FAISS, reranking para sa katumpakan, at answer generation na may wastong pagsipi.

Bakit Mas Gumagana ang Qwen3 RAG

Ang Qwen3-4B-Instruct-2507 ay humahawak ng 262,144 na token nang native, na nag-aalis ng mga isyu sa context truncation na sumasalot sa mas maliliit na modelo. Kasama ang multilingual embeddings ng Qwen3-Embedding-0.6B at binary scoring system ng Qwen3-Reranker-4B, ang stack na ito ay nagbibigay ng enterprise-grade accuracy habang tumatakbo sa katamtamang hardware.

Gumagamit ang arkitektura ng tatlong espesyalisadong modelo: ang modelo ng pag-embed ay nagko-code ng mga dokumento at query sa 1024-dimensional na mga vector, ang FAISS ay nagsasagawa ng tinatayang paghahanap sa pinakamalapit na kapitbahay, ang reranker ay nag-iiskor ng kaugnayan gamit ang mga probabilidad na oo/hindi, at ang modelo ng instruct ay nagsasama-sama ng mga sagot mula sa mga kontekstong may pinakamataas na ranggo.

Mga Kinakailangan sa Pag-setup

I-install ang mahahalagang dependencies para sa tutorial na ito. Tiyaking mayroon kang mga transformer na bersyon 4.51.0 o mas mataas para maiwasan ang “KeyError: 'qwen3′” na isyu:

python

pip install transformers>=4.51.0 torch faiss-cpu numpy tqdm

Kakailanganin mo ng T4 GPU o mas mahusay para sa pinakamainam na pagganap. Ang modelo ng pag-embed ay kumportable na tumatakbo sa CPU, ngunit nakikinabang ang 4B instruct at reranker na mga modelo mula sa GPU acceleration.

I-load ang modelo ng pagsunod sa instruksyon na bubuo ng ating mga pangwakas na sagot. Sinusuportahan ng modelong ito ang katutubong 262K na haba ng konteksto at mahusay sa mga gawain sa pangangatwiran sa pananalapi :

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "Qwen/Qwen3-4B-Instruct-2507"
tokenizer = AutoTokenizer.from_pretrained(model_name)
instruct_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

# Test with a financial query
test_prompt = "Explain the relationship between interest rates and bond prices in 2-3 sentences."
messages = [{"role": "user", "content": test_prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tokenizer([text], return_tensors="pt").to(instruct_model.device)
outputs = instruct_model.generate(**inputs, max_new_tokens=256)
response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(response)

output:

teksto

Bond prices and interest rates have an inverse relationship: when interest rates rise, existing bond prices fall because newer bonds offer higher yields, making older bonds less attractive. Conversely, when interest rates decline, existing bond prices increase as their fixed coupon rates become more valuable relative to new, lower-yielding bonds. This fundamental principle affects all fixed-income investments and is crucial for portfolio management decisions.

Ang modelo ng pag-embed ay nagko-convert ng teksto sa mga siksik na vector para sa pagtutugma ng pagkakatulad ng semantiko. Sinusuportahan ng modelong ito ang hanggang 32K na haba ng konteksto at gumagana sa 100+ wika:

python

import torch.nn.functional as F
from transformers import AutoModel

embed_name = "Qwen/Qwen3-Embedding-0.6B"
embed_tokenizer = AutoTokenizer.from_pretrained(embed_name, padding_side='left')
embed_model = AutoModel.from_pretrained(embed_name, torch_dtype="auto", device_map="auto")

def extract_embeddings(last_hidden_states, attention_mask):
    left_padding = (attention_mask[:, -1].sum() == attention_mask.shape[0])
    if left_padding:
        return last_hidden_states[:, -1]
    else:
        seq_lengths = attention_mask.sum(dim=1) - 1
        batch_size = last_hidden_states.shape[0]
        return last_hidden_states[torch.arange(batch_size, device=last_hidden_states.device), seq_lengths]

# Financial document examples
financial_docs = [
    "Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.",
    "Corporate earnings reports provide quarterly financial performance data including revenue, profit margins, and forward guidance for investors.",
    "The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.",
    "Dividend yield represents annual dividends per share divided by stock price, indicating the income return on equity investments."
]

# Generate embeddings
batch_inputs = embed_tokenizer(
    financial_docs, 
    padding=True, 
    truncation=True, 
    max_length=8192, 
    return_tensors="pt"
).to(embed_model.device)

with torch.no_grad():
    outputs = embed_model(**batch_inputs)

doc_embeddings = extract_embeddings(outputs.last_hidden_state, batch_inputs['attention_mask'])
doc_embeddings = F.normalize(doc_embeddings, p=2, dim=1)

# Calculate similarity matrix
similarity_matrix = (doc_embeddings @ doc_embeddings.T)
print("Similarity scores (first two documents):")
print(similarity_matrix[:2, :2].tolist())

output:

teksto

Similarity scores (first two documents):
[[1.0000001192092896, 0.4892156124114990], [0.4892156124114990, 1.0000001192092896]]

Ang FAISS ay nagbibigay-daan sa mahusay na paghahanap ng pagkakatulad sa malalaking koleksyon ng dokumento gamit ang tinatayang pinakamalapit na mga algorithm ng kapitbahay:

python

import faiss
import numpy as np

# Create FAISS index
embedding_dim = doc_embeddings.shape[1]
faiss_index = faiss.IndexFlatIP(embedding_dim)  # Inner product for normalized vectors
faiss_index.add(doc_embeddings.cpu().numpy())

# Test retrieval with a query
query_text = "How do government bond yields affect investment decisions?"
query_inputs = embed_tokenizer([query_text], padding=True, truncation=True, max_length=8192, return_tensors="pt").to(embed_model.device)

with torch.no_grad():
    query_outputs = embed_model(**query_inputs)

query_embedding = extract_embeddings(query_outputs.last_hidden_state, query_inputs['attention_mask'])
query_embedding = F.normalize(query_embedding, p=2, dim=1)

# Retrieve top 3 most similar documents
scores, indices = faiss_index.search(query_embedding.cpu().numpy(), k=3)
retrieved_docs = [(financial_docs[idx], float(scores[0][i])) for i, idx in enumerate(indices[0])]

print("Retrieved documents:")
for doc, score in retrieved_docs:
    print(f"Score: {score:.4f} - {doc}")

output:

teksto

Retrieved documents:
Score: 0.6234 - Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.
Score: 0.5891 - The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.
Score: 0.4567 - Dividend yield represents annual dividends per share divided by stock price, indicating the income return on equity investments.

Ang modelong reranker ay nag-iiskor ng mga pares ng query-document gamit ang binary na format na yes/no, na nagbibigay ng mas tumpak na relevance ranking kaysa sa cosine similarity lamang:

python

reranker_name = "Qwen/Qwen3-Reranker-4B"
rerank_tokenizer = AutoTokenizer.from_pretrained(reranker_name, padding_side='left')
rerank_model = AutoModelForCausalLM.from_pretrained(reranker_name, torch_dtype="auto", device_map="auto").eval()

# Get token IDs for yes/no scoring
no_token_id = rerank_tokenizer.convert_tokens_to_ids("no")
yes_token_id = rerank_tokenizer.convert_tokens_to_ids("yes")

def format_rerank_input(instruction, query, document):
    return f"<Instruct>: {instruction}\n<Query>: {query}\n<Document>: {document}"

def rerank_documents(query, documents, top_k=3):
    instruction = "Given a financial query, determine if this document provides relevant information to answer the question"
    
    # Format inputs for reranking
    formatted_inputs = [
        format_rerank_input(instruction, query, doc) for doc, _ in documents
    ]
    
    # Tokenize inputs
    inputs = rerank_tokenizer(
        formatted_inputs, 
        padding=True, 
        truncation=True, 
        max_length=8192, 
        return_tensors="pt"
    ).to(rerank_model.device)
    
    # Get relevance scores
    with torch.no_grad():
        logits = rerank_model(**inputs).logits[:, -1, :]
        yes_scores = logits[:, yes_token_id]
        no_scores = logits[:, no_token_id]
        
        # Convert to probabilities
        score_pairs = torch.stack([no_scores, yes_scores], dim=1)
        probabilities = torch.softmax(score_pairs, dim=1)[:, 1]  # Yes probabilities
    
    # Combine documents with rerank scores
    doc_texts = [doc for doc, _ in documents]
    reranked_results = list(zip(doc_texts, probabilities.tolist()))
    reranked_results.sort(key=lambda x: x[1], reverse=True)
    
    return reranked_results[:top_k]

# Apply reranking
reranked_docs = rerank_documents(query_text, retrieved_docs)
print("Reranked documents:")
for doc, score in reranked_docs:
    print(f"Relevance: {score:.4f} - {doc}")

output:

teksto

Reranked documents:
Relevance: 0.8942 - Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.
Relevance: 0.8156 - The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.
Relevance: 0.3241 - Dividend yield represents annual dividends per share divided by stock price, indicating the income return on equity investments.

Pagsamahin ang lahat ng bahagi sa isang function na humahawak sa buong daloy ng trabaho ng retrieval-augmented generation :

python

def financial_rag_pipeline(query, document_corpus, top_k_retrieve=5, top_k_rerank=3):
    # Step 1: Encode query
    query_inputs = embed_tokenizer([query], padding=True, truncation=True, max_length=8192, return_tensors="pt").to(embed_model.device)
    
    with torch.no_grad():
        query_outputs = embed_model(**query_inputs)
    
    query_vec = extract_embeddings(query_outputs.last_hidden_state, query_inputs['attention_mask'])
    query_vec = F.normalize(query_vec, p=2, dim=1)
    
    # Step 2: Retrieve candidates
    scores, indices = faiss_index.search(query_vec.cpu().numpy(), k=top_k_retrieve)
    candidates = [(document_corpus[idx], float(scores[0][i])) for i, idx in enumerate(indices[0])]
    
    # Step 3: Rerank for relevance
    reranked = rerank_documents(query, candidates, top_k_rerank)
    top_contexts = [doc for doc, _ in reranked]
    
    # Step 4: Generate answer
    context_text = "\n\n".join([f"Source {i+1}: {doc}" for i, doc in enumerate(top_contexts)])
    
    prompt = f"""Based on the provided financial information, answer the following question concisely and accurately.

Question: {query}

Context:
{context_text}

Answer: Provide a clear, factual response based on the sources above."""

    messages = [{"role": "user", "content": prompt}]
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    
    inputs = tokenizer([text], return_tensors="pt").to(instruct_model.device)
    outputs = instruct_model.generate(**inputs, max_new_tokens=512, temperature=0.7)
    answer = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
    
    return answer, top_contexts

# Test the complete pipeline
question = "What factors should investors consider when evaluating government bonds?"
answer, sources = financial_rag_pipeline(question, financial_docs)

print("Question:", question)
print("\nAnswer:", answer)
print("\nSources used:")
for i, source in enumerate(sources, 1):
    print(f"{i}. {source}")

output:

teksto

Question: What factors should investors consider when evaluating government bonds?

Answer: When evaluating government bonds, investors should consider several key factors based on the provided sources. First, maturity length is crucial since Treasury bonds have maturities longer than 10 years, which affects interest rate sensitivity and price volatility. Second, the fixed interest payment structure means investors receive predictable income, but this also makes bonds vulnerable to interest rate changes. Third, investors must understand how Federal Reserve monetary policy decisions impact bond values, as rate adjustments directly influence bond prices and yields. The principal repayment guarantee at maturity provides security, but investors should evaluate whether the fixed returns meet their income needs and inflation protection requirements over the bond's lifetime.

Sources used:
1. Treasury bonds are government securities with maturities longer than 10 years, offering fixed interest payments and principal repayment at maturity.
2. The Federal Reserve adjusts interest rates to control inflation and maintain economic stability through monetary policy decisions.
3. Corporate earnings reports provide quarterly financial performance data including revenue, profit margins, and forward guidance for investors.

💡 Mga Tip sa Pag-optimize ng Performance

Para sa deployment ng produksyon, isaalang-alang ang mga pagpapahusay na ito upang mapahusay ang bilis at katumpakan. Gumamit ng batch processing para sa maraming query, ipatupad ang caching para sa mga madalas na naa-access na embeddings, at isaayos ang laki ng chunk sa pagitan ng 400-800 token para sa pinakamainam na katumpakan sa pagkuha.

Ang 262K context window sa Qwen3-4B-Instruct-2507 ay nagbibigay-daan sa iyong magsama ng mas maraming nakuhang dokumento nang walang pagpuputol, karaniwang 8-12 sipi kumpara sa 3-5 para sa mas maliliit na modelo. Subaybayan ang paggamit ng memorya ng GPU at bawasan ang max_length kung makakaranas ka ng mga error na out-of-memory.

📋 Pagsusuri at Kontrol sa Kalidad

Subukan ang iyong RAG system gamit ang mga faithfulness metrics upang matiyak na ang mga sagot ay nananatiling nakabatay sa pinagmulang materyal. Ihambing ang mga output na mayroon at walang reranking upang masukat ang pagbuti sa kaugnayan ng sagot.

Para sa mga pinansiyal na aplikasyon, patunayan ang katumpakan ng numero at tiyakin ang wastong pagsipi ng impormasyon sa regulasyon. Karaniwang pinapahusay ng hakbang sa muling pagraranggo ang kalidad ng sagot nang 15-25% kumpara sa pagkuhang nakabatay sa purong pag-embed.

Ang implementasyong ito ng Qwen3 RAG ay nagbibigay ng pagganap na pang-enterprise-grade na may suportang multilingual at pangmatagalang paghawak. Ang kombinasyon ng mga espesyalisadong modelo ng pag-embed, muling pagraranggo, at pagbuo ay lumilikha ng isang matatag na sistema na mahusay na sumasaklaw sa iba't ibang domain habang pinapanatili ang katumpakan at bilis.