Choosing a Stack for a Company-Docs Q&A Bot: Pinecone + Claude vs. the Alternatives
The build itself is not the hard part. A retrieval-augmented Q&A bot over internal docs is roughly 300 lines of Python: chunk the corpus, embed it, store vectors, retrieve top-k on a question, stuff the chunks into a Claude prompt with a citation instruction. You can have a demo working in an afternoon.
The hard part is the first month after launch, when someone asks about the 2026 expense policy, the bot quotes the 2024 version, and Finance stops trusting it. Or when a contractor asks about parental leave and the bot cheerfully answers from a doc their group shouldn't see.
This guide compares the realistic stack options for the problem — internal docs Q&A with Pinecone as the vector store and Claude as the generator — and recommends a configuration per scenario. The reference implementation is Python, pinecone==10.3.0 / anthropic==2.4.0; verify every call against the SDK source of whatever you actually install.
What you're actually choosing
Four decisions dominate everything else:
- Retrieval back end — Pinecone serverless, Pinecone pod-based,
pgvector, or a managed RAG service. - Embedding model — this determines ceiling recall. Generation can't fix chunks you never retrieved.
- Chunking and metadata — the single most underrated decision. Heading paths,
doc_id,effective_date, andacl_groupsin metadata are what make citations and access control possible. - Orchestration — hand-rolled, a framework (LlamaIndex/LangChain), or fully managed.
Model choice is downstream of all four. Claude Sonnet-tier is the sensible default for generation; drop to Haiku-tier for high-volume extractive answers and reserve Opus-tier for multi-hop questions you route explicitly.
Options at a glance

图示:HEXOPS 原创配图(受选题启发,非复制原文图片) · 来源引用:dev.to
| Option | Stack | Best for | Main risk |
|---|---|---|---|
| A. Hand-rolled | Pinecone serverless + Claude + your own ingest | Teams wanting full control, 1k–100k chunks | You own chunking, sync, evals |
| B. Framework | LlamaIndex or LangChain on Pinecone | Fast prototyping, many file loaders | Abstraction drift, harder debugging |
| C. Managed RAG | Bedrock Knowledge Bases / Azure AI Search + Claude | Enterprises with existing cloud + ACL inheritance | Cost opacity, model lock-in |
| D. Postgres-native | pgvector + Claude |
Already on Postgres, <1M chunks | Manual hybrid search, index tuning |
| E. Lexical only | BM25/Elasticsearch + Claude | Compliance, exact-term lookups | Fails on paraphrase ("PTO" vs "annual leave") |

图示:来源页预览图(仅作编辑上下文;版权归属原发布方) · 来源引用:dev.to
Option E is the cautionary tale that motivates the whole project: keyword search works only if the user guesses the author's vocabulary. It's a useful complement to dense retrieval, not a substitute.
Criteria and weights
Scores below assume a mid-size company with a mixed Google Drive/Confluence corpus and per-team confidentiality. Re-weight for your context.
| Criterion | Weight | Why it dominates |
|---|---|---|
| Retrieval quality on your corpus | 30% | Answer quality is capped by recall@k |
| Access-control fidelity | 20% | A leaked HR doc costs more than a wrong answer |
| Sync/ops burden | 15% | Stale index = confidently wrong answers |
| Cost at your query volume | 15% | Embeddings + storage + tokens, monthly |
| p95 latency | 10% | Chat UX falls apart past ~3s |
| Portability / lock-in | 10% | You will swap a model within 18 months |
| Option | Retrieval | ACL | Ops | Cost | Latency | Portability | Weighted |
|---|---|---|---|---|---|---|---|
| A. Hand-rolled | 4 | 3 | 3 | 4 | 4 | 5 | 3.75 |
| B. Framework | 4 | 3 | 2 | 3 | 3 | 3 | 3.15 |
| C. Managed RAG | 4 | 5 | 5 | 2 | 3 | 1 | 3.65 |
D. pgvector |
3 | 4 | 3 | 4 | 4 | 5 | 3.65 |
| E. Lexical only | 1 | 4 | 5 | 5 | 5 | 4 | 3.50 |
Note what the scores say: managed RAG wins on ACL fidelity and ops, and loses badly on portability and cost. Hand-rolled wins overall, but only if you actually build the ACL filter — its 3 on that axis assumes you do.
The reference stack, briefly
Chunking: 400–800 tokens with 10–15% overlap, split on headings first, never split a table row from its header. Metadata per chunk: doc_id, version, effective_date, acl_groups, source_url, title, heading_path.
Embeddings: OpenAI text-embedding-3-small is the cost-efficient default at 1536 dims. text-embedding-3-large buys recall at roughly double the cost. Cohere embed-v3 and BGE-M3 are the picks for multilingual or self-hosted.
Retrieval: namespace per environment, metadata filter per caller's groups, top_k 8–12, hybrid BM25 union for exact-term queries, rerank the union down to 5–6 before generation.
from pinecone import Pinecone
import anthropic, os
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
index = pc.Index("company-docs")
client = anthropic.Anthropic()
def answer(question: str, groups: list[str]):
hits = index.query(
vector=embed(question), # your embedding call
top_k=10,
namespace="prod",
include_metadata=True,
filter={"acl_groups": {"$in": groups}}, # never omit this
)["matches"]
if not hits or hits[0]["score"] < 0.72:
return "I couldn't find that in the indexed docs.", []
ctx = "\n\n".join(
f"[{i+1}] {m['metadata']['title']} "
f"(effective {m['metadata']['effective_date']})\n{m['metadata']['text']}"
for i, m in enumerate(hits)
)
msg = client.messages.create(
model="claude-sonnet-tier",
max_tokens=700,
system=(
"Answer only from the numbered context.



