Vision-Language-Action
Building compact VLA models that connect visual perception, language understanding, and physical action.
- Small VLA architecture
- Embodied intelligence
- Multimodal policies
Senior Edge AI Engineer
Small models.
Big machines.
I’m Ho Thinh Hung, an Edge AI engineer building compact Vision-Language-Action models and the inference systems that make them useful in the physical world.
HO THINH HUNG
EDGE AIVLAMODEL OPT
100K+model downloads
4NVIDIA runtime fixes
2.11×VLA policy speedup
01 / MY POINT OF VIEW
The most exciting AI is the kind that can see, reason, and act.
I like the moment when a model leaves the lab and meets real constraints: limited memory, strict latency, noisy sensors, and a machine that has to make the right move.
That’s where I work—between model architecture and hardware, turning ambitious multimodal ideas into dependable edge systems.
02 / WHAT I DO
Deep focus,
across the stack.
Three areas where I spend most of my time.
Model Optimization
Making capable models fit real memory, power, and latency budgets without losing what makes them useful.
- Quantization & compression
- TensorRT & ONNX
- CUDA optimization
Production AI Systems
Designing reliable, observable systems for real-time computer vision and high-throughput inference.
- Real-time pipelines
- GPU serving
- Edge deployment
03 / SELECTED WORK
Things I’ve
built and explored.
BF16FP8NVFP4AWQ
MEMORY ↓
15.42 → 19.37 req/s
MODEL OPTIMIZATION · FEATURED
Model Quantization Recipes
Reproducible FP8, NVFP4, AWQ, and TensorRT workflows for LLM and speech models—from preparation to deployment-oriented validation.
ModelOptFP8NVFP4AWQ
View project ↗
2.11×POLICY SPEEDUP
VLA OPTIMIZATION
Model Optimization Notes
Hands-on notes and benchmarks for X-VLA, SmolVLA, quantization, pruning, and edge deployment.
VLATensorRTPruna
View project ↗FP16 → INT8 → INT4NVIDIA EDGE RUNTIME · BUG FIXES
Four bugs. One symptom.
Controlled experiments separated Myelin and CASK fusion failures from lossy AWQ repacking and a silent InternVL3 exporter failure.
TensorRTJetson ThorAWQNVFP4
Read the experiment →04 / PRECISION LAB
Choose the
right trade-off.
Real measurements from my ModelOpt recipes—including BF16, FP8, NVFP4, INT8 SmoothQuant, and INT4 AWQ.
SELECTED RECIPEFP8
Balanced throughput and model footprint with a small WER change.
Memory2.55 GB
Latency · RTF ↓0.0152
Throughput ↑19.37 req/s
Accuracy · WER ↓7.60%
Qwen3-ASR-1.7B · vLLM · RTX 5090Benchmark source ↗
05 / LEARNING IN PUBLIC
Learning
in public.
Paper notes and first-hand engineering experiments, rewritten as practical English articles about the decisions behind fast, reliable edge AI.
TENSORRT · DEBUGGINGEXPERIMENT
Four bugs. One symptom.
Controlled experiments that isolated four garbled-generation failures across FP16, NVFP4, AWQ, and InternVL3—with NVIDIA maintainer confirmation.
REPRODUCE→ISOLATE→VERIFY
Read experiment →
VLA · PERFORMANCENEW
How Fast Can I Run My VLA?
Inference performance across H100, RTX, and Jetson—covering model bottlenecks, deployment splits, and action generation strategies.
VISION→VLM→ACTION
Read article ↗
QUANTIZATIONINT4
AWQ
Activation-aware weight quantization, salient channels, and why TinyChat matters for real runtime gains.
Read article ↗QUANTIZATIONW8A8
SmoothQuant
Moving activation difficulty into weights for efficient, accurate post-training quantization of large models.
Read article ↗06 / TOOLKIT
Tools are choices.
Outcomes are the point.
Core
Python · C++ · CUDA
Inference
TensorRT · Triton · vLLM · ONNX
Vision
DeepStream · YOLO · OpenCV · VLMs
Systems
Docker · Kafka · Redis · WebRTC