01 / AI · Claim vs. Check · Analysis
4-bit method cuts reasoning costs in a vision-language model
The researchers designed QL-VLM to preserve how a model retrieves information from images and text and to keep its reasoning aligned with the full-precision model.

Researchers reported that their 4-bit QL-VLM method reached 90.43% accuracy on ScienceQA-IMG, an image question-answering benchmark.
The paper examined vision-language models, which process images and text together. The researchers said that even when these models compress their reasoning steps into internal numerical representations, running them at full precision still takes substantial computation.
The researchers designed QL-VLM to lower numerical precision while preserving how the model retrieves information and uses it in later reasoning steps.
Cost Reduction Factors Relative to Explicit Reasoning
| Bit operations | 16.64 | |
|---|---|---|
| Measured latency | 6.4 |
Reasoning at lower precision
The paper described a quantisation method that maps values to a table of unevenly spaced representative values. The researchers chose those values to preserve the structure of the rotary keys used to retrieve information.
The researchers also used distillation, a training method that uses the full-precision model as a reference for the lower-precision model. They aligned intermediate reasoning values, retrieval results and predictions to keep later predictions consistent with those of the full-precision model.
Computing costs compared with text-based reasoning
The paper identified Qwen2-VL-2B-Instruct as the model used in the test. In the researchers’ published benchmark results, accuracy improved by 40.33% over the baseline.
The researchers compared QL-VLM with explicit reasoning, which writes out intermediate steps as text. They reported fewer bit operations and shorter measured latency.
The abstract did not identify the hardware used to measure latency.
AI-written by Gyeol Han