Free tools Windows power users keep installed
One-click scans. No signup required.
Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
You have documents, products, images, or support records and want to find the items most related to a query—even when the wording is different. A vector database can help by storing machine-generated representations called embeddings and retrieving the closest matches. But it is not automatically required: PostgreSQL with pgvector, an embedded library, or a local vector store may be the better starting point.
This guide updates the core ideas in DZone Refcard #396, Getting Started With Vector Databases, by Miguel Garcia. The Refcard was published in April 2024 and uses a Weaviate fashion-retail example. Its concepts remain useful, but provider APIs, pricing, and recommended deployment patterns change. Use current vendor documentation before copying provider-specific code.
What problem does a vector database solve?
Traditional databases are excellent at exact operations: find the row where id = 123, return products in the t-shirts category, or search text for a particular term. Vector search addresses a different question:
Which records are most similar in meaning, appearance, sound, or other learned characteristics to this query?
For example, a keyword search for comfortable summer clothing
may not find a product described as lightweight relaxed-fit cotton apparel
. An embedding model can represent both phrases in a numerical space where related concepts tend to be near each other.
Common uses include:
- Semantic search: finding relevant documents despite different wording.
- Recommendations: finding products, images, songs, or articles similar to an item.
- Retrieval-augmented generation (RAG): retrieving source passages before an AI model generates an answer.
- Multimodal retrieval: searching images, audio, video, or text using compatible representations.
- Anomaly detection and clustering: identifying unusual or naturally grouped records.
A vector database does not replace a relational or document database. It is an infrastructure option for workloads where learned similarity is important. Many production systems use both: a relational database for authoritative business data and a vector index for retrieval.
Vector databases in one diagram
raw content
→ chunking or preprocessing
→ embedding model
→ vector + metadata
→ vector index
→ query embedding
→ nearest-neighbor search
→ filtering and ranking
→ application or LLMThe database stores and searches vectors. The embedding model supplies the representation. This distinction matters: a vector database does not understand language or meaning independently. The quality of semantic behavior depends heavily on the model, the data, the chunking strategy, and the search configuration.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Core concepts
Embeddings
An embedding is a numerical representation produced by a machine-learning model. A text embedding might represent a sentence as hundreds or thousands of floating-point values. Related inputs often occupy nearby regions of the model’s vector space.
Different modalities normally require different models. Text, images, audio, and video need representations designed for their data. A query and the stored records must use compatible model semantics. You should not embed documents with one unrelated model and query them with another simply because both return arrays of numbers.
Dimensions
Dimension is the number of components in a vector. A 768-dimensional embedding is an array containing 768 numerical values. Dimension affects storage, memory, index-building time, query cost, and often latency.
Higher dimensionality may preserve more information, but it does not automatically improve retrieval. The useful choice is the model and dimensionality that perform well on your actual task. Common implementation failures include:
- Creating an index with a dimension different from the embedding model’s output.
- Changing models without re-embedding existing records.
- Comparing vectors generated by incompatible models.
- Assuming more dimensions necessarily produce better answers.
- Ignoring the difference between dense and sparse representations.
Store the model name, model version, dimension, and preprocessing configuration with your collection or deployment metadata. That information makes migrations and debugging much easier.
Similarity metrics
Nearest-neighbor search ranks vectors using a distance or similarity function:
Rank #2
- Cosine similarity compares vector orientation and is common for normalized semantic embeddings.
- Dot product, or inner product, can be useful when vector magnitude carries meaning. For normalized vectors, it is closely related to cosine similarity.
- Euclidean distance measures straight-line geometric distance.
The correct metric depends on the embedding model and workload. Do not choose one solely because it is popular, and do not compare raw scores from different metrics or models as though they have a universal meaning.
Indexes and approximate search
A brute-force search compares a query with every stored vector. This is accurate and useful for small collections or evaluation, but becomes expensive as the corpus grows.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteApproximate nearest-neighbor (ANN) indexes reduce search work by organizing the vector space. Common families include:
- HNSW: a graph-based index that often provides strong recall and low latency, at the cost of memory and index-building work.
- IVF or IVFFlat: partitions vectors into regions and searches selected partitions. It can reduce work but requires tuning and may miss relevant neighbors.
- Product quantization and related compression: reduce memory usage and storage size, potentially trading away some accuracy.
Milvus documentation identifies HNSW and IVFFlat among the vector indexes used for efficient retrieval. The important point is that index settings are workload-dependent. Evaluate recall@k, latency, throughput, index-build time, memory use, and update behavior rather than relying on a default configuration.
In general:
more speed and lower memory often means
less exactness, more tuning, or lower recallMetadata and filters
A useful vector record contains more than a vector. It normally includes an identifier, source content, and metadata:
{
"id": "product-123",
"vector": [0.12, -0.04, 0.88],
"text": "Red relaxed-fit cotton T-shirt",
"metadata": {
"category": "t-shirts",
"color": "red",
"tenant_id": "shop-42",
"source": "catalog",
"updated_at": "2026-08-18T00:00:00Z"
}
}Metadata enables filtering by tenant, category, permissions, language, date, availability, or source. It also lets the application return citations, update records, delete stale data, and apply business rules after retrieval.
Filtering must be designed with authorization in mind. A tenant filter is not merely a relevance feature: if it is missing or incorrectly applied, one customer may receive another customer’s data. Check whether filtering occurs during candidate retrieval or only after a top-k result is produced, because those behaviors can affect recall.
Do you actually need a vector database?
No. A dedicated vector database is usually justified when you need persistent storage, high concurrency, horizontal scaling, replication, metadata filtering, operational APIs, backups, recovery, multitenancy, or independent vector-search scaling.
For smaller workloads, alternatives may be simpler:
| Option | Best for | Main advantage | Main drawback |
|---|---|---|---|
| Managed vector service | Fast production setup | Low operational burden | Ongoing cost, provider APIs, and lock-in risk |
| Self-hosted Qdrant, Weaviate, or Milvus | Control and portability | Deployment flexibility | Your team owns upgrades, backups, security, and scaling |
PostgreSQL + pgvector |
Existing SQL applications | One platform for transactions, joins, and vectors | May not fit extreme vector scale or independent retrieval scaling |
| Chroma or LanceDB | Prototypes and local applications | Developer simplicity | Less operational depth for large distributed deployments |
| FAISS | Research and offline similarity search | Application-managed control and performance | Not a complete durable, multiuser database |
PostgreSQL with pgvector is particularly attractive when your application already depends on PostgreSQL and needs SQL joins, transactions, and moderate-scale similarity search. FAISS is an indexing library, not a turnkey service with authentication, backups, metadata APIs, and multiuser operations.
Build a minimal semantic-search prototype
The provider-neutral lifecycle is:
- Choose an embedding model appropriate for the language, modality, and domain.
- Split documents into meaningful chunks.
- Generate one embedding for each chunk or record.
- Create a collection, index, or table using the correct dimension and metric.
- Insert vectors, identifiers, source text, and metadata.
- Embed a user query with the same model.
- Run nearest-neighbor search with an initial
top_k. - Apply tenant and business filters.
- Inspect the returned records and scores.
- Delete the test collection or records when finished if the provider charges for retained resources.
The following is conceptual pseudocode, not a drop-in SDK example:
documents = load_documents()
chunks = split_into_chunks(documents)
vectors = [embed(chunk.text) for chunk in chunks]
store.create_collection(
name="knowledge",
dimension=len(vectors[0]),
metric="cosine"
)
store.upsert([
{
"id": chunk.id,
"vector": vector,
"metadata": {
"text": chunk.text,
"source": chunk.source
}
}
for chunk, vector in zip(chunks, vectors)
])
query_vector = embed("How do I reset my password?")
results = store.search(
vector=query_vector,
top_k=5,
filter={"source": "help-center"}
)For a current managed onboarding path, Pinecone’s documentation shows installation with pip install pinecone, a client initialized with from pinecone import Pinecone, and a quickstart covering index creation, text upsert, search, and cleanup. Follow the current Pinecone quickstart rather than copying older Refcard code.
For a local file-backed experiment, the Milvus Lite quickstart demonstrates:
from pymilvus import MilvusClient
client = MilvusClient("milvus_demo.db")Milvus Lite can reduce setup friction, while larger Milvus deployments target distributed workloads. Weaviate’s current quickstart documents both a Weaviate Cloud path and a local Docker path. Provider APIs change, so verify client versions, authentication, collection schemas, and cleanup commands against current documentation.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteFrom semantic search to RAG
Retrieval-augmented generation adds an answer-generation step:
- Ingest and chunk authoritative documents.
- Embed the chunks and store them with source identifiers and metadata.
- Embed the user’s question.
- Retrieve candidate chunks.
- Optionally rerank them with a second model.
- Place selected context into the language model’s prompt.
- Generate an answer with citations or source references.
RAG can improve grounding, but it does not guarantee correctness or eliminate hallucinations. Incorrect chunking, weak embeddings, stale records, missing filters, low recall, prompt injection in retrieved documents, and poor citation handling can all produce bad answers.
Use source identifiers so every generated claim can be traced to an authoritative record. Apply authorization before placing retrieved text into a prompt. Retrieved documents are untrusted input and may contain instructions designed to manipulate the model.
Keyword, vector, and hybrid retrieval
Vector search is not a universal replacement for lexical search. Product IDs, error codes, names, email addresses, legal terms, and exact numbers often require keyword or field matching. A robust search system may combine:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- Dense vector retrieval for semantic similarity.
- Lexical retrieval for exact terms and identifiers.
- Metadata filters for authorization and business constraints.
- Reranking to reorder a candidate set.
Hybrid search is not automatically better. Scores from different retrieval systems must be normalized or weighted, and the combination should be evaluated on representative queries.
How to evaluate retrieval
A high similarity score is not proof that a result is correct. Build a small evaluation set containing real queries and expected relevant records. Measure:
- Recall@k: how often relevant records appear in the first k results.
- Precision@k: how many of the first k results are relevant.
- Answer quality: whether an RAG response is supported, complete, and appropriately cited.
- Latency: including embedding, filtering, retrieval, and reranking.
- Throughput: sustained reads and writes at expected concurrency.
- Cost: embedding calls, storage, memory, queries, network, replicas, and operations.
Test with the actual embedding model, vector dimension, corpus, metadata filters, update frequency, and concurrency. A benchmark on another dataset or vendor’s preferred configuration does not prove which system is fastest for your application.
Choosing an implementation
Managed services
Managed services reduce the work of provisioning, upgrades, availability, and scaling. They can be a good fit when a team wants to reach production quickly and does not want to operate a database cluster. Trade-offs include usage-based pricing, plan minimums, data-residency constraints, provider-specific APIs, and migration risk.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Pinecone’s pricing page showed the following figures on August 18, 2026: Starter free, Builder at $20 per month, Standard with a $50 monthly minimum, and Enterprise with a $500 monthly minimum. These are time-sensitive published signals, not universal production costs; additional usage varies by operation, plan, cloud, and region. Check the current pricing page and cost estimator for your workload.
Weaviate offers cloud and local deployment choices. Qdrant offers self-hosted and cloud paths and directs customers to a workload-based pricing calculator rather than one fixed price. Milvus Lite provides a local entry point, while Zilliz Cloud provides a managed Milvus option. Do not assume open-source software is cost-free: infrastructure, backups, networking, support, and operations still have a price.
Self-hosting
Self-hosting can provide deployment, data-location, and customization control. It may be attractive for Kubernetes-based organizations or regulated workloads. However, the team becomes responsible for capacity planning, upgrades, security, replication, backups, recovery testing, monitoring, and performance tuning.
PostgreSQL versus a dedicated vector database
Start with PostgreSQL and pgvector when relational data, transactions, SQL joins, and operational simplicity dominate. Evaluate a dedicated system when vector search is the primary workload, the corpus or query volume is large, specialized filtering or compression is needed, or vector search must scale independently from transactional traffic.
Recommended Free Tools
Production checklist
- Model: record the model, version, dimension, language coverage, and preprocessing rules.
- Chunking: preserve enough context without creating oversized or repetitive chunks.
- Schema: include stable IDs, source references, timestamps, tenant identifiers, and deletion state.
- Filtering: enforce tenant, permission, region, language, and availability constraints.
- Freshness: re-embed changed content and remove deleted or superseded records.
- Search: test dense, lexical, and hybrid retrieval where exact terms matter.
- Indexing: tune ANN parameters against recall and latency targets.
- Security: protect API keys, rotate secrets, encrypt connections, and restrict access.
- Privacy: define retention and deletion rules for vectors, metadata, source text, queries, and logs.
- Reliability: configure backups and perform an actual restore test.
- Observability: monitor latency, empty-result rates, query volume, index growth, errors, and cost.
- Portability: maintain exportable source data and embeddings, and avoid burying business logic in an irreversible provider API.
Common mistakes and recovery steps
Dimension mismatch
If insertion or querying fails because dimensions differ, inspect the model output and collection schema. Do not truncate or pad vectors casually. Create a correctly configured collection and re-embed the records if the model changed.
Best Value
- Used Book in Good Condition
Wrong query model
If results are nonsensical after a deployment change, confirm that query embeddings use the same compatible model and preprocessing as stored embeddings. A model migration normally requires re-embedding the corpus.
Poor chunking
If answers omit context, chunks may be too small or split across headings, tables, and explanations. If results contain too much unrelated material, chunks may be too large. Evaluate several chunking strategies using the same query set.
Vector-only retrieval misses exact matches
Add lexical search or exact field lookup for identifiers, codes, and names. Hybrid retrieval should be tested rather than assumed to improve every query.
Filters destroy recall
Check whether the filter is valid, whether the filtered subset contains relevant data, and whether the search engine applies the filter during candidate generation. A restrictive filter can correctly return no result even when the unfiltered search looks strong.
Costs grow unexpectedly
Inspect vector count, dimension, metadata size, replicas, index memory, embedding-call frequency, query fan-out, network traffic, and plan minimums. Delete test indexes and records when they are no longer needed.
A practical decision tree
Already centered on PostgreSQL?
→ Try pgvector first.
Need a local prototype?
→ Try Milvus Lite, Chroma, LanceDB, or FAISS.
Need managed production with minimal operations?
→ Evaluate Pinecone, Weaviate Cloud, Qdrant Cloud, or Zilliz Cloud.
Need self-hosting and distributed scale?
→ Evaluate Milvus, Qdrant, or Weaviate.
Need exact identifiers as well as semantic meaning?
→ Use hybrid lexical + vector retrieval.The best choice depends on corpus size, query volume, filtering, update patterns, compliance requirements, team skills, and cost—not on a generic ranking of vendors.
About the DZone Refcard
DZone’s Getting Started With Vector Databases is Refcard #396, written by Miguel Garcia. It introduces vector-database fundamentals, key concepts, data preparation, collection creation, querying, and output through a fashion-retail similarity-search example using Weaviate. It is available as an online reference and free PDF.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Its accessible progression is a useful starting point. For a current implementation, supplement it with provider documentation, especially for authentication, SDK syntax, index configuration, pricing, filtering behavior, and deletion semantics. The concepts transfer across systems, but APIs and operational guarantees do not.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
