Verified, then logged.
Sun 22 Nov 2026 · 00:20 KST / 15:20 UTC한국어 | ENSearch /

01 / AI · Analysis

NAVER Cloud's multimodal training throughput rises to 150.2%

A rebuilt vision encoder and rebalanced image load drove the gain, in a company-published result measured on one 30B model.

A multi-GPU server tray pulled out of a data center rack, with an out-of-focus technician behind it facing a monitor filled with a grid of photo thumbnails. AI-generated image

NAVER Cloud, the cloud arm of South Korean internet company NAVER, said it raised multimodal training throughput on a 30B model to 150.2% of unoptimised Megatron Bridge, NVIDIA's model-training software. The company-published figure appeared in an NVIDIA developer blog post dated 18 Aug 2026. The metric is tokens processed per second per GPU, and the experiment ran on an NVIDIA B200 cluster.

A vision-language model, which learns from images and text together, adds image-processing work to every training step. According to the post, the baseline used the same parallelism settings, and the optimisations were stacked on top in turn. The figure therefore measures gains in the training path, not added hardware.

The largest single gain in the NVIDIA post came from the vision encoder, the component that turns images into tokens. The encoder trains alongside the language model and directly affected the time of every step.

Throughput gain by optimization step (company-reported)

Storage I/O2.5
Pack reordering13.3
Activation recomputation5.4
Native encoder23.9
Source developer.nvidia.com · as of 2026-08-18 · unit: %Chart: made by LOGIT

Rebuilding the vision encoder delivered the largest gain

The researchers wrote that they first connected NAVER's in-house vision encoder through its Hugging Face implementation unchanged. On that path the encoder sat outside Megatron's parallelism, and a single GPU had to handle high-resolution images. Activation recomputation, a technique that saves memory by recalculating intermediate values, raised the sequence-length limit from 8K to 16K.

The next step reimplemented the same encoder as Megatron blocks. The main configuration uses tensor parallelism of 1 (TP=1) and expert parallelism of 8 (EP=8). The NVIDIA post attributed the throughput gain in this setup to Megatron's optimised kernels.

In company-published figures in the post, peak GPU memory in this setup moved from 164.7GB to 166.3GB. With TP=4 and EP=2, the native encoder lifted throughput from 100% on the Hugging Face path to 150.3%, a separate comparison from the headline figure. Peak GPU memory in that setup fell from 103.1GB to 97.3GB.

Rebalancing image load across GPUs

According to the NVIDIA post, default packing, which groups samples into training sequences, filled fixed 8K sequences with whole samples and reached a company-published efficiency of 99.4%. That method balanced only the language-model load, and packs of equal length still held different numbers of image tokens. The GPU holding the most image-heavy pack slowed every step.

The researchers wrote that they redistributed packs among GPU workers by image-token count. The reordering does not cross global batch boundaries, so the set of samples seen by each optimiser step stays the same.

The deciding variable in the post's measurements was the communication method. A synchronous version that waited for pack exchanges gained only 1.7%, a company-published figure. The rest of the gain came from an asynchronous design that ran the exchanges alongside training computation.

What the 150.2% figure covers

The NVIDIA post describes a single test subject: a 30B MoE model in the Nemotron-3-Nano style, where MoE (mixture of experts) is a design that shares work among several expert neural networks. The data was NAVER Cloud's production training set of more than 2 trillion tokens from over 100 sources. The company did not disclose the reduction in training time or the number of GPUs in the cluster.

The same post cited HyperCLOVA X 32B Think, part of NAVER Cloud's HyperCLOVA X foundation model family, as a production model with the same architecture. The experiment, however, used the 30B model, and public documents give no basis for extending the result to the whole product family.

The two companies' documents describe the scope of the company-published figure differently. A NAVER technical post said the four optimisation areas combined raised throughput on the 30B MoE vision-language model by 50.2% in total. The NVIDIA post said the chart carrying the figure covers only the first three areas, because the fourth concerned correctness rather than speed.

Donghaeng Media Sidae, a South Korean news outlet, reported the result on 9 Nov 2026 as a NAVER Cloud announcement. The same figure had already appeared in the NVIDIA post of 18 Aug 2026.

The cost left by context parallelism

The fourth area was a fix for parallelism errors, which entered Megatron Bridge through PR #4784, a pull request, the NVIDIA post said. CP, or context parallelism, splits a long context across several GPUs, and results from this area sit outside the headline figure.

Expanding context parallelism from CP=1 (16K) to CP=2 (24K) carried a 38% cost before mitigation, a company-published figure. Vision DP when CP, a feature that encodes each image only once per CP group, raised throughput in the CP=2 run from 100% to 120%, also company-published.

NAVER named building frontier AI models through the Nemotron Coalition as the first item in its collaboration with NVIDIA. Inews24, a South Korean IT news outlet, reported that NVIDIA vice-president Jonathan Cohen introduced the model collaboration with NAVER Cloud at a Seoul event on 9 Nov 2026. Edaily, a South Korean business daily, reported that NAVER Cloud shares pre-training and post-training data and reinforcement-learning environments with the coalition.

The figures NAVER published measure the volume of tokens GPUs processed, not model quality. Public documents give neither the effect of the speed-up on model quality nor a release date for the collaborative model. NAVER said in the NVIDIA post that it is extending this foundation to Omni-style multimodal training and plans to share further results.