Vector Databases in Depth: The Heart of RAG

In the previous chapter we saw the full RAG flow with a single example paragraph. But in reality, the HR knowledge base of a mid-sized company easily has hundreds of documents — manuals, policies, annual updates, template contracts — that, once chunked, can turn into tens of thousands of individual chunks.

That is where a vector database stops being an implementation detail and becomes a real piece of architecture.

What is a vector database?

Unlike a traditional query that searches by exact match (“give me every record where department = HR”), a vector database searches by semantic closeness (“give me the chunks that mean something similar to this question”).

Analogy: a traditional database is like a catalog ordered alphabetically — you need to know the exact title or the author. A vector database is like a librarian who has read every book and organized them in a space where books on similar topics sit physically close. You ask “how do paid vacations work?” and it walks you to the employee-benefits section, even if no book has the word “vacation” in the title.

Why a traditional index is not enough, with concrete numbers

A typical embeddings vector has 768 dimensions. In standard precision (float32, 4 bytes per number), that is 3,072 bytes per vector — just for the meaning of one chunk, without the original text. With 50,000 chunks: roughly 150MB in vectors, about 50MB in text, and around 200MB total for a mid-sized base.

The real problem is not storage — it is search. A traditional database is excellent at “give me the documents where date = 2026” (indexable with B-trees). But “which of these 50,000 vectors is mathematically closest to this other one?” is not a simple ordering relation that a B-tree can index. The most direct exact method would be to compute similarity against each of the 50,000 vectors, one by one — feasible at that scale, expensive with millions.

That does not mean SQL and vector search are mutually exclusive: extensions like pgvector add vector indexes to PostgreSQL. The important distinction is between a traditional index over scalar values and a specialized index for nearest neighbors.

Indexing: how HNSW avoids comparing against everything

HNSW (Hierarchical Navigable Small World) organizes the vectors in layers: an upper layer with few highly connected points (like seeing only the main highways on a map), and lower layers that get denser and denser (like seeing every street).

The airport analogy: imagine an airport with 50,000 gates, looking for the one closest to a café. Linear search: you walk past all 50,000, one by one. HNSW: the airport is organized in levels — at the top level you only see the main terminals, you walk toward the closest one, you drop a level, you repeat. At each step you discard entire regions without visiting them.

The practical result: linear search grows with the size of the base (O(N) — twice the documents means roughly twice the comparisons). HNSW search typically shows approximately logarithmic growth (O(log N) under typical conditions) — growth that is much slower than linear. That difference in behavior — not a specific millisecond figure, which depends enormously on hardware, data, and configuration — is what lets systems with millions of vectors answer in times that are usable in production.

HNSW is an approximate index — it trades a small chance of missing the theoretically optimal result for much higher speed. In practice that loss can be tuned with configuration parameters (see the code section below) and should be measured as recall on real data.

Binary quantization: compressing vectors for more speed

3,072 bytes per vector at full precision. Binary quantization (used by Qdrant, Milvus, and others) compresses each of the 768 numbers to a single bit: above a given reference point → 1, below → 0. Result: 768 bits = 96 bytes per vector — 32 times smaller.

Analogy: it is like converting a high-resolution photo to 2-tone black and white. You lose detail, but the image is much smaller and faster to process.

Comparing bit strings is much cheaper than comparing full decimal vectors. Qdrant reports speedups of up to 40× for its implementation, although the real result depends on dimensionality and on the distribution of the vectors; it is not a guarantee for every dataset.

Two-stage strategy, common in production:

First filter (fast): search for candidates using the binary representation — bitwise operations, very fast.
Second filter (precise): take the best candidates from the first filter and recompute full similarity using the original uncompressed vectors, only over that reduced subset.

The result combines the speed of binary search with the precision of the full vectors. The size of the set you re-evaluate should be tuned with retrieval evals, not assumed as universal.

The architecture of a vector database (with Qdrant)

Collection: the container for every chunk — like a “table” in SQL. All of its vectors share dimensionality and distance metric.

Points: each chunk is a point, with a vector, a payload (JSON metadata), and a unique ID:

{
  "id": "policy_12345",
  "vector": [0.12, -0.34, 0.56, "... 768 numbers ..."],
  "payload": {
    "texto": "Employees with 2 to 5 years of seniority are entitled to 20 business days of vacation per year.",
    "documento": "politica_vacaciones_2026",
    "seccion": "antiguedad",
    "fecha_actualizacion": "2026-01-15",
    "vigente": true,
    "departamento": "RRHH"
  }
}

Distance metric:

Metric	Description	When to use it
Cosine	Measures the angle between vectors, independent of magnitude	When the embeddings model recommends cosine similarity
Euclidean	Straight-line distance	When the geometry and magnitude of the trained space are meaningful
Dot product	Combines direction and magnitude	When the model was trained for dot product; with normalized vectors it is equivalent to cosine

The metric is not chosen only by the modality of the data: it must match the function used when the embeddings model was trained.

Filters: solving the stale-version problem

What happens if the vacation policy was updated, but the old document is still in the base? Similarity search could retrieve the outdated version just fine — vector similarity does not know which version is current.

Filters combine vector search with exact conditions on the payload:

from qdrant_client import models

resultados = client.query_points(
    collection_name="politicas_rrhh",
    query=vector_pregunta.tolist(),
    query_filter=models.Filter(
        must=[
            models.FieldCondition(
                key="vigente",
                match=models.MatchValue(value=True),
            ),
            models.FieldCondition(
                key="departamento",
                match=models.MatchValue(value="RRHH"),
            ),
        ]
    ),
    limit=5,
    with_payload=True,
).points

The system is not searching “the closest thing among everything” — it is searching “the closest thing, only among what also satisfies this exact condition”. You keep both documents (2025 and 2026), with vigente: true only on the current one, and the filter makes sure only the correct version is retrieved.

In large collections it is worth creating payload indexes for fields used frequently in filters; otherwise the exact filter can become a bottleneck too.

Beyond text: vectors for images, audio, and video

Yes, you can vectorize images, audio, and video — but there is a key distinction worth locking in.

The vector is a numeric representation of the content. It is stored in the vector database. The original object (the image, the audio, the video) is stored in object storage (S3, Google Cloud Storage, a local bucket) with a reference in the payload.

Analogy: the vector is like the thematic summary of a book in the catalog; the original object is the physical book on the shelf; the payload is the reference that says which aisle it is in.

For images: models like CLIP (from OpenAI, generates text and image embeddings in the same vector space — letting you search images with text and vice versa), ResNet (extracts computer-vision features), or DINOv2 (from Meta, self-supervised). The process: the image is vectorized with CLIP, the vector goes into the database together with the URL of the real image in the payload. When someone searches in text (“show me images of products with scratches”), the question is vectorized with the same CLIP model and, because text and image share a vector space, it finds the relevant images.

For audio: CLAP (Contrastive Language-Audio Pretraining) generates joint representations of audio and text. Wav2Vec 2.0 produces audio representations useful for downstream tasks. Whisper is used mainly to transcribe: a common strategy is to convert the audio to text and generate the embedding of that transcription. The original audio file stays in object storage.

For video: you can extract keyframes and vectorize them as images, combine frame embeddings with captions, or use specific models like VideoCLIP, which aligns video and language representations. A long video is usually split into segments, each with its own vector, timestamps, and a reference to the original file.

Real use cases: visual search in e-commerce (“find shoes similar to this photo”), voice assistants that retrieve context beyond literal matches, and search for relevant moments inside hours of video.

Use cases, in one word: “match”

Domain	Example	What gets vectorized
Recommendation	Spotify adjusting suggestions based on what you skip or repeat	Songs and preference signals
Semantic search	Our HR system	Document text and questions
Fraud detection	Atypical transactions in finance	Transactional behavior
CBIR	Similar images in a catalog	Images with models like CLIP or DINOv2
Video search	The moment someone talks about “transformer” in a conference	Frames, segments, and captions
Computational biology	Proteins with similar sequences or structures	Amino-acid sequences or structures

Security and access control

Our HR base probably has information that not everyone should query the same way — salary policies, for example.

Role-based filtering (RBAC):

from qdrant_client import models

def buscar_con_acceso(consulta, usuario):
    vector = modelo_embeddings.encode(consulta)

    if usuario.rol == "admin":
        filtro = None
    elif usuario.rol == "rrhh":
        filtro = models.Filter(
            must=[
                models.FieldCondition(
                    key="departamento",
                    match=models.MatchValue(value="RRHH"),
                )
            ]
        )
    else:
        filtro = models.Filter(
            must=[
                models.FieldCondition(
                    key="departamento",
                    match=models.MatchValue(value=usuario.departamento),
                ),
                models.FieldCondition(
                    key="nivel",
                    match=models.MatchValue(value="publico"),
                ),
            ]
        )

    resultados = client.query_points(
        collection_name="politicas_empresa",
        query=vector.tolist(),
        query_filter=filtro,
        limit=5,
        with_payload=True,
    ).points
    registrar_consulta(usuario.id, consulta, resultados)
    return resultados

The authorization filter must be built on the server from an already-authenticated identity; it must never be accepted directly from client-controlled parameters. You also need encryption at rest and in transit, secrets management, isolation between tenants, and audit logs. The concrete algorithm and standards depend on the threat model and the applicable regime: a regulation like GDPR requires controls adequate to the risk, it does not by itself prescribe a single encryption figure or algorithm.

Practical implementation with Qdrant

docker run --rm -p 6333:6333 -p 6334:6334 \
  -v "$(pwd)/qdrant_storage:/qdrant/storage" \
  qdrant/qdrant

from qdrant_client import QdrantClient, models

client = QdrantClient(url="http://localhost:6333")

client.create_collection(
    collection_name="politicas_rrhh",
    vectors_config=models.VectorParams(
        size=768,
        distance=models.Distance.COSINE,
    ),
    hnsw_config=models.HnswConfigDiff(
        m=16,              # more connections: more recall and more memory
        ef_construct=100,  # more candidates at build time: more recall, slower indexing
    ),
)

# Insert once, during indexing
points = []
for i, chunk in enumerate(chunks):
    embedding = modelo_embeddings.encode(chunk["texto"])
    points.append(
        models.PointStruct(
            id=i,
            vector=embedding.tolist(),
            payload={
                "texto": chunk["texto"],
                "documento": chunk["documento"],
                "vigente": True,
            },
        )
    )

client.upsert(
    collection_name="politicas_rrhh",
    points=points,
)

# Search with a filter on every query
def buscar_en_rrhh(pregunta, solo_vigentes=True):
    vector_pregunta = modelo_embeddings.encode(pregunta)
    filtro = (
        models.Filter(
            must=[
                models.FieldCondition(
                    key="vigente",
                    match=models.MatchValue(value=True),
                )
            ]
        )
        if solo_vigentes
        else None
    )

    resultados = client.query_points(
        collection_name="politicas_rrhh",
        query=vector_pregunta.tolist(),
        query_filter=filtro,
        search_params=models.SearchParams(
            hnsw_ef=128,  # more candidates: more recall, slower search
        ),
        limit=3,
        with_payload=True,
    ).points

    return [hit.payload["texto"] for hit in resultados]

m controls how many connections each node in the graph keeps, and ef_construct how much the index builder explores. At query time, hnsw_ef controls how many candidates the search walks: raising it usually improves recall, at the cost of more latency. The values shown are a didactic starting point, not a universal configuration; they should be measured on the real embeddings and load.

Chapter summary

A vector database is a search engine by meaning, not only by exact match.
Linear search grows with the size of the base (O(N)); indexes like HNSW typically achieve approximately logarithmic growth under typical conditions.
It is organized as collections → points → vector + payload. Payload metadata enables filters that combine semantic search with exact conditions.
It handles multimodality: the vector goes to the vector database, the original object (image, audio, video) goes to object storage, connected by a reference.
It needs real security controls: role-based filtering, encryption, isolation, and auditing.

In the next chapter (MCP, 5c) we will look at how the model does not only query information, but executes actions on the real world. The vector database solves “what should I know?”; MCP solves “what should I do?”.