Architecture and operating model
Quantization represents weights and sometimes activations or KV cache at lower precision to reduce memory and bandwidth. FP16/BF16 is a common production baseline; INT8 and INT4 provide more compression, but kernel support, calibration and quality loss depend on model and hardware.
Post-training quantization converts an existing model using calibration data or weight-only methods. Quantization-aware training simulates lower precision during training and can preserve quality for some tasks. GPTQ, AWQ, bitsandbytes and FP8 differ in weight/activation scope and hardware kernels.
Do not accept quality using aggregate perplexity alone. Compare target-language tasks, long context, code, numerical reasoning, tool schemas and RAG citations. Memory savings may not become throughput if dequantization, unsupported kernels or small batches dominate.
- 1FP16/BF16 reference
- 2Quantization and kernel selection
- 3Equal-quality load benchmark
- 4Artifact record and controlled rollout
Design parameters
- Weight lower bound
- For 8B parameters, raw weights are roughly 16 GB at FP16, 8 GB at INT8 and 4 GB at INT4.
- Calibration set
- Represent real languages, domains, context lengths and output formats.
- Kernel compatibility
- Confirm from logs that GPU, runtime and format select the supported optimized kernel.
- Quality budget
- Set separate maximum drops for task success, critical errors, format compliance and safety.
Worked example
An FP16 release uses 23 GB total GPU memory at 12,000 token/s while INT4 uses 10 GB at 18,000 token/s. If Turkish legal accuracy falls from 91% to 84%, the 50% throughput gain is unacceptable for that use case; try higher or mixed precision.
INT8 may fit two models on one GPU, but if shared memory bandwidth raises p95 TTFT from 600 to 980 ms, fitting is not capacity evidence.
Example commands: replace lab values and confirm permissions and software versions before use.
quantize build --model model.safetensors --method int8 --calibration eval/calibration-v3.jsonl
quantize bench --baseline bf16 --candidate int8 --trace traffic.jsonl
quantize gate --max-quality-drop 0.02 --max-ttft-p95-ms 900Troubleshooting
| Observation | Likely cause / distinction | Verification |
|---|---|---|
| Memory falls but speed does not improve | Optimized kernels are absent or dequantization/CPU dominates. | Inspect runtime kernel logs, profiler and batch/context distribution. |
| Only some tasks regress | Calibration coverage or sensitive layers are poorly represented. | Analyze error slices and try mixed precision for affected layers. |
| Model fails on another GPU | Format, compute capability or runtime version is incompatible. | Match artifact quantization method and support matrix to target hardware. |
Acceptance checks
- Pin FP16/BF16 reference outputs and performance.
- Record quantization artifact and calibration-data versions.
- Run quality regression on real task slices.
- Measure TTFT, TPOT, token/s and memory on identical traces.
- Verify kernel selection on target GPU/runtime combinations.
- Rollback automatically when canary quality or error thresholds fail.
Related concepts
Capacity and usable headroom
Raw capacity is not the capacity available to applications. RAID or erasure coding, filesystems, reserved space, metadata, snapshots and growth headroom are separate deductions. TB and TiB representations also change the displayed number. Write calculations with units, establish protected usable capacity, then subtract operating reserves. Track growth rate as well as current utilization. The projected exhaustion date should leave enough time to procure and deploy additional capacity.
Bandwidth and useful throughput
Link capacity differs from useful application throughput. Protocol headers, encryption, retransmissions, small files and storage waits reduce net transfer speed. Keep bits and bytes distinct: 1 Gbit/s corresponds to a theoretical 125 MB/s, not an application performance guarantee. Estimate transfer time as data size divided by measured useful throughput. Observe the network, source reads and destination writes together to locate the bottleneck. Consider temporary slowdowns and competing workloads as well as average speed.
Latency distribution
Latency is the time between starting an operation and receiving its result. An average can hide a small number of very slow operations; medians and p95/p99 percentiles answer different questions. Network RTT, storage waits, processor queues and application processing contribute to end-to-end time. State whether measurements come from the client or server. Check whether increases coincide with traffic growth, maintenance or capacity limits. Record normal and peak-hour baselines before selecting an alert threshold.
Cost boundaries
Cost includes more than purchase price or a monthly resource bill. Licensing scope, storage, egress, backup, support, operating effort and disruption impact are separate items. Compare equal service levels, capacity and time periods; a cheaper option may exclude an obligation. Compare forecast and actual spending by resource or business unit. Assess underuse risk for commitments and uncontrolled growth for flexible models. Verify pricing and licensing conditions against official documents at decision time.
Dependencies and restart order
Services commonly depend on identity, DNS, time, networking, databases and licensing. Record a dependency graph describing conditions for operation, rather than only a list of equipment. Recovery order follows that graph; circular dependencies may require emergency access paths. Distinguish restored infrastructure from resumed business activity. Assign an owner, validation method and alternative access path to each dependency. Test assumptions by deliberately making one component unavailable in a controlled end-to-end exercise.
Application consistency
A bootable copy does not prove application-data consistency. Operating-system caches, database logs and write ordering across disks or services affect the result. A crash-consistent copy resembles recovery after an unexpected shutdown; an application-consistent copy follows supported application preparation and write coordination. Validate transaction integrity, relationships between records and application behavior after recovery, rather than only counting files. Confirm backup integration, application version and reported errors before assuming that consistency was achieved.
Primary documentation
Prepared by the Doz Teknoloji technical team using the primary references below. Calculations and lab scenarios state their assumptions; validate the applicable product version before rollout.