Architecture and operating model
Vector database selection cannot rely on an ANN latency benchmark alone. Recall, metadata filters, updates/deletes, tenant isolation, backup, consistency, operational skill and total volume all matter. Exact brute-force results with the same embeddings and distance metric provide the quality reference.
HNSW graph search can provide high recall and fast queries while graph memory and build cost grow. IVFFlat partitions vectors into lists and trades speed against recall using probe counts, often with lighter builds. Behavior varies by implementation and data distribution.
Metadata filtering is part of retrieval security. Applying tenant or document-class filters after ANN can exhaust candidates or process unauthorized data. Test selectivity, payload/scalar indexes and iterative scan behavior with the actual query mix.
- 1Embedding and exact reference
- 2Index build and metadata schema
- 3Filtered ANN query
- 4Recall, latency and maintenance measurement
Design parameters
- Recall target
- Compare with exact top-k and require a threshold such as recall@10 ≥ 95% for critical queries.
- Filter selectivity
- Measure latency and recall when 50%, 10%, 1% and 0.1% of rows remain.
- Write profile
- Measure daily inserts, deletes and re-embedding against index maintenance and vacuum/compaction.
- Memory budget
- Separate vectors, graph/list structures, metadata indexes, cache and replicas.
Worked example
Ten million 1,536-dimensional float32 vectors consume about 61.4 GB for raw values alone. IDs, metadata, index, WAL/replicas and working space can multiply this. Quantization reduces memory but its recall and reranking impact must be measured on the same corpus.
If HNSW ef_search 40 yields 91% recall@10 at 18 ms p95 and 120 yields 97% at 34 ms, choose from the SLO and cost of misses rather than the setting name. If a 0.1% tenant filter starves results, tune the filter-aware plan separately.
Example commands: replace lab values and confirm permissions and software versions before use.
vector-bench load --vectors 10000000 --dims 1536 --filters tenant,doctype
vector-bench run --indexes hnsw,ivfflat --k 10 --filter-selectivity 0.5,0.1,0.01,0.001
vector-bench compare --ground-truth exact --metrics recall,p95,qps,memoryTroubleshooting
| Observation | Likely cause / distinction | Verification |
|---|---|---|
| Filtered query returns too few rows | ANN candidates are exhausted after filtering or no filter index exists. | Compare candidate counts, probe/ef settings and exact results before and after filters. |
| Recall drops on recent data | Distribution changed or clusters/index need maintenance. | Measure recall by time slice and inspect centroid/list or graph maintenance. |
| Disk grows after deletes | Tombstones or compaction/vacuum policy do not reclaim space. | Compare live rows, index size, deleted ratio and maintenance jobs. |
Acceptance checks
- Pin embedding version and distance metric in the collection schema.
- Keep exact-reference query sets.
- Measure recall and p95 across filter-selectivity slices.
- Test production impact during insert, delete and reindex.
- Enforce tenant authorization in query filters and access layers.
- Measure backup, restore and index rebuild time.
Related concepts
Indexes and execution plans
An index can reduce scanned data for suitable queries, but adds write, maintenance and storage costs. Evaluate column order, selectivity, filters and sorting together. Slow queries are not always caused by missing indexes: estimates, locks, storage latency or excessive data transfer may dominate. Inspect representative execution plans and actual row counts. Measure read improvement alongside write overhead, and avoid uncontrolled production maintenance.
Latency distribution
Latency is the time between starting an operation and receiving its result. An average can hide a small number of very slow operations; medians and p95/p99 percentiles answer different questions. Network RTT, storage waits, processor queues and application processing contribute to end-to-end time. State whether measurements come from the client or server. Check whether increases coincide with traffic growth, maintenance or capacity limits. Record normal and peak-hour baselines before selecting an alert threshold.
Capacity and usable headroom
Raw capacity is not the capacity available to applications. RAID or erasure coding, filesystems, reserved space, metadata, snapshots and growth headroom are separate deductions. TB and TiB representations also change the displayed number. Write calculations with units, establish protected usable capacity, then subtract operating reserves. Track growth rate as well as current utilization. The projected exhaustion date should leave enough time to procure and deploy additional capacity.
IOPS and block size
IOPS measures operations per second, whereas MB/s measures transferred data. Identical IOPS values produce very different bandwidth at different block sizes. For example, 10,000 operations/s at 4 KiB is approximately 39.1 MiB/s. Read/write mix, random versus sequential access and protection mechanisms affect the result. Measure actual block distributions and concurrency rather than transferring a benchmark result directly to production. High disk IOPS does not prove good application response; latency must be examined alongside it.
Application consistency
A bootable copy does not prove application-data consistency. Operating-system caches, database logs and write ordering across disks or services affect the result. A crash-consistent copy resembles recovery after an unexpected shutdown; an application-consistent copy follows supported application preparation and write coordination. Validate transaction integrity, relationships between records and application behavior after recovery, rather than only counting files. Confirm backup integration, application version and reported errors before assuming that consistency was achieved.
Cost boundaries
Cost includes more than purchase price or a monthly resource bill. Licensing scope, storage, egress, backup, support, operating effort and disruption impact are separate items. Compare equal service levels, capacity and time periods; a cheaper option may exclude an obligation. Compare forecast and actual spending by resource or business unit. Assess underuse risk for commitments and uncontrolled growth for flexible models. Verify pricing and licensing conditions against official documents at decision time.
Primary documentation
Prepared by the Doz Teknoloji technical team using the primary references below. Calculations and lab scenarios state their assumptions; validate the applicable product version before rollout.