LEARNING IN PUBLIC · ENGINEERING LOGS
Research,
made practical.
Research notes rewritten as practical English articles, plus first-hand investigations from edge AI, VLA models, inference performance, and production systems.
RECOMMENDED READING PATH
Follow the ideas as they evolve.Foundation→Quantization→Deployment
Quantization is a systems decision
A practical field guide to INT8, INT4, FP8, NVFP4, and the measurements that decide whether lower precision is actually better.
Attention, viewed from the edge
The Transformer shape flow, why training parallelizes, and where autoregressive inference becomes a memory and latency problem.
INT8 without folklore
Range mapping, calibration, partial quantization, and a production workflow for models that must remain accurate at the edge.
SmoothQuant: moving the hard part offline
Why activation outliers break naive W8A8 quantization, and how equivalent scaling restores an efficient INT8 path.
GPTQ: quantizing with second-order information
How Hessian-aware error compensation, lazy updates, and Cholesky reformulation made 3–4 bit LLM quantization practical.
AWQ: protect what activations reveal
An engineering explanation of salient channels, INT4 weight-only quantization, and why the runtime matters.
LUT-GEMM: low-bit weights without reconstructing them
How binary-coded weights become lookup-table indices—and when that changes decode latency rather than only checkpoint size.
SpinQuant: rotate before you round
Why learned rotations tame activation outliers and make W4A4KV4 much less destructive than naive low-bit quantization.
ParoQuant: pairwise rotations that survive deployment
From learned Givens rotations to an INT4 TensorRT pipeline—and what my reduced-calibration experiment actually showed.
VLA-Perf: How to reason about VLA inference
A practical map of compute, memory, network, and action-generation bottlenecks in Vision-Language-Action systems.
TensorRT Edge-LLM: four fixes from controlled experiments
How I separated compiler fusion bugs, lossy AWQ repacking, and a silent InternVL3 export failure—and verified each fix on NVIDIA edge hardware.