OpenAI Assistants API File Search Tutorial

If you want an AI assistant that can answer questions using your own documents without building a retrieval‑augmented generation pipeline from scratch, the OpenAI Assistants API’s file search tool is the fastest way to get there. This tutorial walks through the entire workflow—creating an assistant, uploading files, configuring a vector store, enabling citations, and managing costs—using clear, friendly explanations and ready‑to‑run Python snippets. By the end, you’ll have a working assistant that can search PDFs, Markdown, and other supported formats while giving you traceable sources for every answer.

Why the Assistants API’s File Search Matters

Large language models are powerful, but they only know what was in their training data. When you need the model to reference internal policies, product specs, or financial reports, you have to bring that information into the conversation. The file search tool does exactly that: it takes your documents, automatically chunks them, creates embeddings, stores them in a vector store, and then retrieves the most relevant passages whenever the assistant decides a search is needed. The result is a RAG‑style experience without you having to manage an embedding model, a vector database, or custom chunking logic.

Beyond retrieval, the assistant also returns built‑in citations. Each time it pulls information from a file, it adds an annotation that points back to the source chunk, making it easy for users to verify the answer—a feature that turns a curious demo into a trustworthy tool for professional settings.

Prerequisites

Before diving into code, make sure you have:

• An OpenAI account with API access (you can create a project‑specific key in the dashboard).
• Python 3.9 or newer installed.
• A virtual environment tool (like venv or conda) to keep dependencies isolated.
• The files you want to search—PDF, .md, .docx, .txt, or .json work out of the box.

Install the official OpenAI SDK:

pip install --upgrade openai

Create a .env file in your project root and add your key:

OPENAI_API_KEY=sk‑your‑project‑key

All code examples assume the SDK is initialized with client = OpenAI().

Step 1: Create an Assistant with File Search Enabled

The assistant is the persona that will use the tools you give it. For a document‑Q&A bot, we enable only the file search tool (you can add others later).

from openai import OpenAI

client = OpenAI()

assistant = client.beta.assistants.create(
    name="Policy Helper",
    instructions=(
        "You are a helpful assistant that answers questions using the uploaded policy documents. "
        "When you use information from a file, cite the source."
    ),
    model="gpt-4o",
    tools=[{"type": "file_search"}],
)

print(f"Assistant created: {assistant.id}")

The instructions field guides the model’s behavior; here we explicitly ask it to cite sources, which helps the citation mechanism later.

Step 2: Upload Files and Build a Vector Store

File search relies on a vector store—a container that holds your documents after they’ve been parsed, chunked, and embedded. You can create the store and upload files in one call using the convenient upload_and_poll helper, which blocks until indexing finishes.

# 1. Create the vector store
vector_store = client.vector_stores.create(name="Company Policies")

# 2. Prepare file streams (open in binary mode)
file_paths = ["policies/remote-work.pdf", "policies/security-guide.md"]
file_streams = [open(path, "rb") for path in file_paths]

# 3. Upload and wait for processing
file_batch = client.vector_stores.file_batches.upload_and_poll(
    vector_store_id=vector_store.id,
    files=file_streams,
)

print(f"Upload status: {file_batch.status}")
print(f"File counts: {file_batch.file_counts}")

The SDK handles chunking (default 800‑token chunks with 400‑token overlap), embedding with text-embedding-3-large, and storage. You don’t need to know the internal details unless you want to tweak them later (see the “Customizing Chunking and Ranking” section).

Step 3: Link the Vector Store to the Assistant

Now that the store exists, tell the assistant to use it via the tool_resources parameter.

assistant = client.beta.assistants.update(
    assistant_id=assistant.id,
    tool_resources={
        "file_search": {"vector_store_ids": [vector_store.id]}
    },
)

print("Vector store attached to assistant")

At this point, any thread that runs with this assistant can search across all files in the vector store.

Step 4: Create a Thread and Ask a Question

A thread holds the conversation history. You can create a fresh thread for each user session or reuse one if you want persistence.

thread = client.beta.threads.create()

# User message – feel free to attach additional files here if needed
client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="What does the remote‑work policy say about equipment reimbursement?",
)

# Kick off a run
run = client.beta.threads.runs.create_and_poll(
    thread_id=thread.id,
    assistant_id=assistant.id,
)

print(f"Run completed with status: {run.status}")

create_and_poll is handy for simple scripts; for production UIs you might prefer streaming (see later).

Step 5: Retrieve the Answer and Show Citations

When the run finishes, the assistant’s reply is in the thread’s message list. The real power appears in the annotations field, which contains file citations.

messages = client.beta.threads.messages.list(thread_id=thread.id, order="desc", limit=1)
msg = messages.data[0]

if msg.role == "assistant":
    text_block = msg.content[0].text
    print("Assistant:", text_block.value)

    # Show citations
    citations = []
    for ann in text_block.annotations:
        if hasattr(ann, "file_citation"):
            cited_file = client.files.retrieve(ann.file_citation.file_id)
            citations.append(f"[{ann.index}] {cited_file.filename}")

    if citations:
        print("\nSources:")
        for line in citations:
            print(line)

You’ll see something like:

Assistant: According to the remote‑work policy, employees receive a stipend of $150 per year for home office equipment【0†remote-work.pdf】.

Sources:
[0] remote-work.pdf

The bracketed numbers (【0†…】) are inserted automatically by the API; you can strip them or re‑format them for your UI.

Step 6: Streaming Responses for a Better UX

If you’re building a chat interface, streaming tokens as they’re generated feels more responsive. Replace the polling run with a stream and provide an event handler.

from typing_extensions import override
from openai import AssistantEventHandler

class SimpleHandler(AssistantEventHandler):
    @override
    def on_text_created(self, text) -> None:
        print("\nassistant > ", end="", flush=True)

    @override
    def on_text_delta(self, delta, snapshot):
        print(delta.value, end="", flush=True)

    @override
    def on_tool_call_created(self, tool_call):
        print(f"\n\nassistant > {tool_call.type}\n", flush=True)

    @override
    def on_message_done(self, message) -> None:
        # Print citations after the full message arrives
        text_block = message.content[0].text
        citations = []
        for ann in text_block.annotations:
            if getattr(ann, "file_citation", None):
                cited = client.files.retrieve(ann.file_citation.file_id)
                citations.append(f"[{ann.index}] {cited.filename}")
        if citations:
            print("\n\nSources:")
            for c in citations:
                print(c)

with client.beta.threads.runs.stream(
    thread_id=thread.id,
    assistant_id=assistant.id,
    event_handler=SimpleHandler(),
) as stream:
    stream.until_done()

The handler prints each delta as it arrives, giving a typing‑effect experience, and prints citations once the message is complete.

Customizing Chunking and Ranking

While the defaults work for most documents, you can override chunk size, overlap, and the number of chunks returned to the model. This is useful when you have highly structured content (e.g., tables) where the default splitting might break important context.

Adjusting Chunking Strategy

When uploading files, you can specify a chunking_strategy per file.

file_batch = client.vector_stores.file_batches.upload_and_poll(
    vector_store_id=vector_store.id,
    files=[
        {
            "file_id": file.id,
            "chunking_strategy": {
                "type": "static",
                "max_chunk_size_tokens": 1200,
                "chunk_overlap_tokens": 300,
            },
        }
    ],
)

Constraints:

• max_chunk_size_tokens must be between 100 and 4096.
• chunk_overlap_tokens must be non‑negative and not exceed half of max_chunk_size_tokens.

Controlling How Many Chunks Are Used

The assistant pulls a limited number of chunks into its context window. You can raise or lower this limit when you create the assistant or when you start a run.

assistant = client.beta.assistants.create(
    name="Detailed Analyst",
    instructions="Answer with deep detail from the source documents.",
    model="gpt-4o",
    tools=[{"type": "file_search"}],
    tool_resources={
        "file_search": {
            "vector_store_ids": [vector_store.id],
            "max_num_results": 30,   # increase from default 20
        },
    },
)

Or override per‑run:

run = client.beta.threads.runs.create_and_poll(
    thread_id=thread.id,
    assistant_id=assistant.id,
    tools=[{"type": "file_search", "file_search": {"max_num_results": 30}}],
)

Tuning the Ranker

If you notice irrelevant chunks influencing answers, you can adjust the ranker’s scoring threshold or the balance between semantic (embedding) and keyword signals.

assistant = client.beta.assistants.update(
    assistant_id=assistant.id,
    tool_resources={
        "file_search": {
            "vector_store_ids": [vector_store.id],
            "ranking_options": {
                "ranker": "default_2024_08_21",
                "score_threshold": 0.3,   # only keep chunks with score ≥0.3
                "hybrid_search": {
                    "embedding_weight": 0.7,
                    "text_weight": 0.3,
                },
            },
        }
    },
)

A higher score_threshold cuts off low‑scoring chunks, potentially improving precision at the cost of recall. Adjusting embedding_weight vs. text_weight lets you favor semantic similarity or exact keyword matches.

Managing Costs

The Assistants API incurs three main cost categories:

Expiration Policies

You can tell OpenAI to automatically delete a vector store after a period of inactivity.

vector_store = client.vector_stores.create_and_poll(
    name="Temporary Policies",
    file_ids=[file1.id, file2.id],
    expires_after={
        "anchor": "last_active_at",
        "days": 3,   # delete after 3 days of no use
    },
)

Thread‑level vector stores (those created via message attachments) already have a default 7‑day expiration; you can override this the same way.

Cleanup Script

Periodically run a script that:

1. Lists all vector stores.
2. Checks the last_active_at timestamp.
3. Deletes stores older than your chosen threshold.
4. Optionally deletes orphaned files (those not attached to any assistant or thread).

from datetime import datetime, timedelta, timezone

cutoff = datetime.now(timezone.utc) - timedelta(days=30)
for vs in client.vector_stores.list():
    if vs.updated_at < cutoff:
        client.vector_stores.delete(vs.id)
        print(f"Deleted vector store {vs.id}")

Best Practices from Real‑World Use

1. Start Small – Begin with a few dozen documents to validate the workflow before scaling to thousands.
2. Leverage Both Assistant‑ and Thread‑Level Stores – Keep a global knowledge base at the assistant level for static policies, and allow users to attach case‑specific files per thread.
3. Always Show Citations – They build trust and let users verify answers, especially in regulated industries.
4. Monitor Usage – The OpenAI dashboard provides token and storage metrics; set alerts to avoid surprise bills.
5. Handle Unsupported Formats Gracefully – If you encounter a file type the API can’t parse (e.g., a proprietary CAD format), convert it to Markdown or JSON before uploading, or use function calling to fetch data from your own system.
6. Test Edge Cases – Try queries that should return no results, very long documents, and ambiguous phrasing to see how the assistant behaves.

Going Beyond File Search

The Assistants API also offers Code Interpreter for data analysis and Function Calling for invoking your own backend services. You can combine all three in a single assistant:

assistant = client.beta.assistants.create(
    name="Full‑Featured Assistant",
    instructions=(
        "You can search documents, run Python code for analysis, and call custom APIs when needed. "
        "Always cite sources for file‑based answers."
    ),
    model="gpt-4o",
    tools=[
        {"type": "file_search"},
        {"type": "code_interpreter"},
        {"type": "function"},
    ],
    # ... add function definitions and vector store attachments as shown earlier
)

With this setup, the assistant can, for example, retrieve a sales report, compute quarterly growth with Python, and then call your internal CRM API to enrich the answer—all while still providing traceable citations for the document‑derived parts.

Wrap‑Up

You now have a complete, end‑to‑end understanding of how to use the OpenAI Assistants API’s file search tool:

• Create an assistant with the file_search tool.
• Upload documents and let the SDK build a vector store.
• Attach that store to the assistant (or to individual threads).
• Run queries, stream responses, and extract built‑in citations.
• Fine‑tune chunking, ranking, and result limits to match your content.
• Manage storage costs with expiration policies and periodic cleanup.

Armed with this knowledge, you can move from a simple demo to a production‑ready document‑Q&A assistant that saves users time, reduces manual searching, and gives them confidence in the AI’s answers through transparent sourcing. Happy building!