Calibration text for 4-bit weight-only quantization of Nemotron 3.5 Super VL, checkpoint

Technical report three of three on the Nemotron 3.5 Super VL study. Prepared for the NVIDIA Nemotron model team, October 1, 2026, revised October 13, 2026. Measurements taken on 8×H200.

Abstract

A simulated law firm, the legal customer of report 2, wants to serve NVIDIA Nemotron 3.5 Super VL, checkpoint BF16-09212026, at lower cost by storing its weights at 4 bits, and asks whether the quantizer must be calibrated on its own legal documents. Because customer documents are proprietary, a public legal corpus stands in for them.

Thoughtworks built three 4-bit weight-only GPTQ versions of the model, calibrated on legal or on generic text. Each makes the weights 3.41 times smaller, from 231.8 GiB to 68.1 GiB, and raises cross-entropy on held-out legal text by 0.32% to 0.61%. When the legal and generic calibration texts contain the same number of tokens, no difference is detected between them, and the interval on the difference lies inside a margin fixed before scoring. In this scenario the firm does not need to supply its own documents for calibration.

For a real customer, Thoughtworks recommends calibrating on generic text and accepting the quantized model only after comparing it with the original model on held-out customer documents. All results were measured on the pre-release checkpoint BF16-09212026, before GA (general availability).

The GA checkpoints are the completed post-training release, with improvements across multimodal, verbosity, chat and instruction following, and the results have not been re-measured on them.

1. Setup

In the simulated scenario, the law firm of report two serves Nemotron 3.5 Super VL and wants to lower its serving cost by storing the weights at four bits instead of 16. GPTQ, the quantization method used in this study, sets each layer's 4-bit weights using a small sample of text, the calibration text. The firm asks whether that text must come from its own legal documents or whether generic text works as well. For this demonstration the study cannot use customer documents, so a public legal corpus stands in for them (Section 1.1).

The builds quantize the original checkpoint BF16-09212026 with no LoRA adapter from report 2 merged in. Each is compared with the unquantized checkpoint on cross-entropy, a measure of how well a model predicts held-out text, where lower is better.

1.1 Question and design

The legal corpus is the one used in report two: the CaseHOLD train split and six subsets of nvidia/Nemotron-Pretraining-Legal-v1, with general replay. The 128 legal calibration records were drawn at random (seed 20260920) from its legal rows and cover CaseHOLD and four of the six subsets: Definition-Classification, Function-Of-Decision, ToSDR-QA, and eCFR-QA. None came from ToS-Clause-Understanding or Diversity-Jurisdiction. The question is whether legal calibration text keeps the quantized model's cross-entropy on held-out legal text closer to the original model's than generic text with the same number of tokens.

GPTQ derives its settings from the calibration tokens, so two calibration sets with the same number of records but different record lengths also differ in how much text the quantizer sees. At 128 records each, the generic set holds 1.81 times as many tokens as the legal set. The study therefore built three versions:

• Legal, 128 records. The domain build.

• Generic, 71 records. The token-matched control. Its 65,083 tokens are 7.7% fewer than the legal set's 70,518.

• Generic, 128 records. The record-matched build. The 71-record set is its first 71 records, so the pair compares two calibration volumes of one corpus.

   

Healthcare, the study's second simulated customer, was not quantized; the comparison covers the legal corpus only.

1.2 Checkpoint

All builds start from BF16-09212026, NVIDIA's early-access post-training release of Nemotron 3.5 Super VL, downloaded as 80 files totaling about 249 GB (231.8 GiB). Its language model has 512 routed experts and one shared expert in each mixture-of-experts layer, with 22 routed experts active per token, and a hidden size of 4,096.

1.3 Quantization recipe

All three builds use the same quantization script, settings and calibration draw seed (20260920) and differ only in their calibration text. The quantization settings exported with the three builds are byte-identical.

The module counts in Table 1 were read from each built model's weight index.

Choice of quantizer. The comparison needs a quantizer that reaches the routed experts, which hold 92% of the checkpoint's parameters, and that uses the calibration text when it sets the weights. LLM Compressor's GPTQModifier meets both: it sets each weight from statistics of the calibration tokens and quantizes all 40,960 routed-expert projections and the 80 shared-expert projections (Table 1).

Its round-to-nearest W4A16 path reads no calibration data, so builds differing only in calibration text would be identical and could not answer the question.

1.4 Calibration sets

NVIDIA's Model Optimizer example multinode_fsdp2_ptq.slurm uses 512 calibration samples for a 120B Super model, and the Model Optimizer hf_ptq README documents a range of 128 to 512. The legal and record-matched builds use 128 samples, the low end of that range, and the token-matched build uses 71.

1.5 Evaluation

Each build and the unquantized reference were served by vLLM 0.28.1rc1.dev628+g2a02f6efe on two GPUs (tensor parallel width 2) with expert parallelism enabled. For each evaluation record, a client asked the server for the log-probability of every token and computed the mean cross-entropy in nats, so the score describes each model as served.

The evaluation set is the first 512 legal records of the validation split of an earlier version of the study's legal corpus, which splits the same source rows between training and validation differently from the version used in report 2. It contains 86,349 scored tokens, a median of 158 per record and a range of 63 to 560. The 128 legal calibration records came from the training split of report 2's version, and overlap between them and the 512 evaluation records was not checked.

Differences between two models were computed with a paired bootstrap over records (10,000 draws, seed 20260922): each draw resamples records and recomputes both models on the same sample, so each record keeps its token weight. Intervals are 95% percentile intervals.

Acceptance criterion, fixed before any build was scored. A build passes when the 95% interval on its cross-entropy difference from the unquantized reference lies entirely within 0.01 nats. The prediction fixed with it was that the difference between legal and generic calibration would also fall within that margin. The 0.01-nat margin is the study's own choice, about 0.7% of the reference cross-entropy, and no published measurement links it to a loss of task accuracy.

Measurement repeatability. Three repeat scorings of the unquantized reference on four GPUs returned the same mean, 1.432691 nats, over the same 86,349 tokens, so repeated serving adds no variation at this precision. Moving the reference from four GPUs to two changed it by +0.000146 nats, and enabling expert parallelism on two GPUs changed it by -0.000487 nats. All four models are therefore compared under one configuration, two GPUs with expert parallelism.

1.6 Relation to NVFP4

NVFP4 is NVIDIA's 4-bit floating-point format, which Blackwell GPUs execute in four-bit arithmetic. This study ran on H200, and Hopper GPUs have no four-bit floating-point arithmetic, so the builds could not use four-bit activations. The builds in this report are therefore weight-only 4-bit integer builds with BF16 activations. The calibration findings concern these builds, and nothing in this report measures NVFP4.

2. Results

All three builds raise held-out legal cross-entropy above the unquantized reference, by 0.32% to 0.61%, and no difference is detected between legal and generic calibration at matched token volume.

2.1 Each build against the unquantized reference

All three builds raise cross-entropy; the lower ends of their intervals run from +0.002657 to +0.006485 nats. The legal and token-matched builds meet the 0.01-nat acceptance criterion. The 128-record generic build does not: its difference, +0.008774 nats, is inside the margin, but the upper end of its interval, +0.011187 nats, is outside.

2.2 Domain against volume

With the same number of calibration tokens, legal and generic calibration differ by +0.002026 nats, and the interval includes zero, so no difference is detected. The interval, 0.004430 nats wide, lies entirely inside the ±0.01-nat margin, as predicted before scoring. A 95% interval inside the margin is equivalent to two one-sided tests at the 2.5% level, so for these builds on these 512 records, a domain effect larger than 0.01 nats in either direction is ruled out at that level. Within the interval, legal calibration ranges from 0.000197 nats worse to 0.004233 nats better than generic. The bootstrap resamples evaluation records only, and each calibration set was quantized once, so the interval does not include variation between repeated builds (Section 3).

Because all three contrasts use the same records, the record-matched contrast, +0.004227 nats, splits exactly into the token-matched domain contrast, +0.002026, and the volume contrast, +0.002201. Comparing at equal record count would therefore credit legal text with 0.004227 nats, of which 0.002201 is the difference between two generic builds that differ only in calibration volume.

Within the generic corpus, the larger calibration set produced the higher cross-entropy: the 128-record set adds 57 records and 62,495 tokens to the 71-record set, and its build scored 0.002201 nats higher. It also contains three more records cut off at the 2,048-token cap. Each build was made once, so this report does not attribute the 0.002201-nat difference to calibration volume rather than to variation between builds.

2.3 Footprint and serving

The three builds have identical byte counts, and their resident memory at idle matches the reference to within 2 MiB per card, because vLLM gives the memory left after loading weights to the key-value cache. The key-value cache size was not recorded, so this report gives no measured gain in capacity or throughput.

Throughput, latency and key-value cache capacity were not measured on any build, so the serving saving that motivates the scenario is not quantified. On weights alone, the 4-bit model occupies 68.1 GiB, which fits on one H200 with 139.81 GiB usable (report 1, Table 1) and leaves 71.8 GiB for activations and the key-value cache; the unquantized weights, 231.8 GiB, need at least two cards.

The builds cannot be served on four GPUs. The shared expert's down_proj has an input dimension of 5,376, which four GPUs split into 1,344 columns each, or 10.5 groups of 128; two GPUs give 21 whole groups. By the same arithmetic eight GPUs give 5.25 groups and one GPU gives 42; neither was tried. The limit follows from group size 128 against this dimension, so it applies to any build of the checkpoint that quantizes the shared expert's down_proj at that group size and is served by an engine that requires whole groups on each GPU. For this reason all four models in Tables 3 and 4 were served on two GPUs.

2.4 Mixture sweep

A second experiment, varying the share of legal text in a mixed calibration set, was planned to run only if the three builds showed a domain effect. Table 4 detects no difference at matched volume, 95% interval [-0.000197, +0.004233] nats, so it was not run.

3. Limitations

• One build per calibration set. Each calibration set was quantized once, so variation between repeated builds is unmeasured. The 0.002201-nat difference between the two generic builds is the same size as the domain contrast, and both may include that variation.

• One evaluation measure. Cross-entropy on 512 short legal records, median 158 scored tokens, is the only score. No benchmark accuracy, throughput, latency, key-value cache capacity, long-document or image evaluation was run on the quantized builds, so whether 4-bit weights change answers, and how much they save in serving cost, is unmeasured.

• Calibration and evaluation splits. The calibration and evaluation records come from two builds of the legal corpus that split the source rows differently, and their overlap was not checked.

• Margin chosen by the study. The 0.01-nat margin was fixed before scoring, and no published result converts it to a loss of task accuracy.

• Unequal truncation. The 2,048-token cap truncates four generic records in the 128-record set, one in the 71-record set and no legal record.

• Low calibration volume. All three sets hold between 65,083 and 127,578 tokens over 71 to 128 records, at or below the low end of NVIDIA's documented 128 to 512 samples, and no larger set was tested.

• One scheme, one domain, one hardware type. Only W4A16 GPTQ was compared, only on legal text and only on H200; NVFP4 with four-bit activations was not tested.

• Serving width. Of the GPU counts tried, these builds serve on two GPUs and refuse four; one and eight were not tried.

• Build and score records. The LLM Compressor version is not written into the build records; the export records compressed-tensors 0.18.1a20260919. The four score files do not record serving width or engine flags, which come from the file names and the comparison record.

• Original model. The builds quantize BF16-09212026 without a report 2 adapter merged, while a customer would quantize the adapted model.

• Checkpoint. All results are on BF16-09212026 and none has been re-measured on the GA checkpoints.

   

4. Conclusion

In the simulated scenario, a law firm wants to store Nemotron 3.5 Super VL's weights at 4 bits and asks whether the quantizer must be calibrated on its own documents. On BF16-09212026, GPTQ W4A16, with the layers listed in Table 1 kept at BF16, makes the weights 3.41 times smaller and raises held-out legal cross-entropy by 0.32% to 0.61%. The legal and token-matched generic builds meet the 0.01-nat acceptance criterion; the 128-record generic build does not. With the same number of calibration tokens, legal and generic calibration differ by 0.002026 nats: no difference is detected, the interval lies inside the margin fixed before scoring, and the mixture experiment was not needed. Comparing at equal record count would credit legal text with 0.004227 nats, of which 0.002201 is the difference between the two generic builds. Each result rests on one build per calibration set and on cross-entropy alone (Section 3). These builds serve on two GPUs and not on four, because group size 128 does not divide the shared expert's 5,376-wide down_proj into whole groups across four GPUs.

For a real customer, Thoughtworks recommends calibrating the quantizer on generic text and accepting the quantized model only after comparing it with the unquantized model on a held-out part of the customer's own documents. The customer's documents are then needed for that comparison and not for calibration.