TL;DR:
Post-training an open-weight model is now an afternoon's work. We taught NVIDIA Nemotron 3.5 Super VL a multi-step chart-reading skill: locate the subplot, find each of its axes, read its tick labels, count them and add the counts. Accuracy on that task rose from 17% to 78% after 3.3 hours of training on four NVIDIA H200 GPUs.
A specialized open model can run on infrastructure you control. A 4-bit build cut the model's weights from 232 GiB to 68 GiB, so they fit on one H200 instead of two, and needed no customer documents to build.
Legal and healthcare adapters trained in about 3 hours each. On held-out documents they predicted the reference text 9.4 and 5.4 percentage points more accurately than the base model. We describe the test a customer should run before deploying an adapter.
Why we ran these tests
Enterprises are moving more of their AI work onto open-weight models they run themselves, mainly for cost and sovereignty. A token sourcing strategy decides, workload by workload, what to rent from a frontier lab and what to own: how much intelligence the task needs, what it costs at the volume you run, how specific the domain is, where the data is allowed to go, and what intellectual property the prompts carry.
In short, frontier models earn their price on high-complexity reasoning, strategy and creative work, and owned open-weight models take the specific, sensitive and high-volume work, on infrastructure you control, whether your own data center or a neo-cloud provider (NCP). Where the task needs it, post-training improves the model's performance in your domain.
Before moving a workload to an owned model ask these questions: can the model learn my task quickly, can I run it on hardware I have, and can I tell whether the training worked before I deploy it. We put NVIDIA Nemotron 3.5 Super VL to all three tests.
Our approach
NVIDIA Nemotron 3.5 Super VL is NVIDIA's open vision-language model. It is a mixture of experts with about 120 billion parameters, of which about 12 billion are active for each token, and it reads images and text together.
As an early access partner, Thoughtworks ran three studies on it, each for a simulated customer: a consulting firm that extracts figures from charts, a law firm and a healthcare provider. We simulate because a real customer's documents cannot be published; the method is what transfers. All training ran on four NVIDIA H200 GPUs in about 3 hours per adapter. No customer data was used at any stage.
All results were measured on NVIDIA's 09/21 pre-release checkpoint, before general availability. The GA checkpoints are the completed post-training release, with improvements across multimodal, verbosity, chat and instruction following. We have not re-measured our results on them. Each section names its full technical report.
Teaching the model to read charts
The consulting firm pulls figures out of charts in its own reports. One recurring step is counting the labeled ticks across a chart's axes. Out of the box, the model already read a single labeled tick correctly 95% of the time and identified a chart's subplot layout 98% of the time. Counting across every axis combines those skills, and it is the kind of specific, repeated task post-training is for. We chose it after scoring the model on ten public vision benchmarks, as the chart question with the most room to gain.
We trained an adapter, a small trainable layer that sits beside the frozen model, on the language model only and left the vision encoder, the part that reads the image, unchanged. The training data was 24,000 charts rendered from plotting code, so the correct count for every chart was known by construction. No chart from the evaluation set was used. A customer would follow the same arrangement: generated charts for training, its own charts only for evaluation.
On 224 charts from published research papers, taken from a public benchmark of chart questions, accuracy rose from 17.0% to 78.1%. The benchmark scores answers with an AI judge; scoring the same answers with a simple rule that reads the final number gave the same two figures. On 300 held-out generated charts, it rose from 18.3% to 99.3%. Training took 3.3 hours.
There was a cost, and we report it. On the 3,776 benchmark questions of other types, the adapter answered 104 fewer correctly than the base model, a net loss of 2.8 percentage points, most of it on two question types about colorbar labels, the color scale beside a heatmap. Across all 4,000 questions, accuracy still rose, from 88.8% to 89.6%. For a production adapter, we would mix general chart data into training to reduce that loss; we have not measured the effect of that mix.
Two things in this experiment deserve a leader's attention. The training data was generated, so the customer does not need thousands of hand-labeled examples. And the evaluation set stayed separate, which is what makes the 78% a fair measure: the model had never seen those charts.
Read the vision post-training of NVIDIA Nemotron 3.5 Super VL technical report.
Legal and healthcare, and how to know it worked
The law firm and the healthcare provider want the model adapted to their documents, and each needs a way to check that adaptation helped before paying for it. We trained three adapters on public data, with one percent of each corpus held out before training, in about 3 hours each. Then we measured the gain three ways, and they disagreed.
Held-out documents. We measured how often the model predicts the next word of a reference answer correctly. For the law firm, that rose from 68.8% to 78.2%, and to 76.8% for a second legal adapter. For the healthcare provider, it rose from 70.7% to 76.1%. All three gains have 95% confidence intervals that exclude zero. The limit of this measure is that it scores the reference text, not an answer the model wrote.
A public benchmark. On 3,600 questions of a public legal multiple-choice benchmark, neither legal adapter scored higher than the base model; each was about 1.4 points lower in one of two comparisons. We also built a benchmark from 2019 to 2026 United States court opinions, and the unadapted model answered 92% of it correctly, which leaves too little room to separate two models.
An AI judge. Before trusting a judge model to compare adapter and base answers, we checked it by having it compare two samples of the same base model. Its verdict moved by up to 35 points when we reworded its instructions, and it favored the longer answer. We withdrew the judged comparison.
In our Nemotron 3.5 Lightning study, a blind AI judge preferred the legal adapter in 75% of comparisons. For Super VL, the judge control above showed that a judge alone could not decide between the two models, partly because the adapter's answers were much shorter. We therefore recommend a test that needs no judge: hold out part of your own documents before training and compare the adapted and unadapted models on them, question by question.
In addition, training on single-letter answers shortened the legal adapter's output: its median response fell from 1,400 tokens to 151, and it ran out of its 16,384-token budget on 6 questions where the base model ran out on 188. Shorter answers are cheaper answers, when they are right.
Read the measuring legal and healthcare domain adaptation of NVIDIA Nemotron 3.5 Super VL technical report.
Fitting it on one GPU and serving it faster
A model you own has to fit the hardware you have. The law firm wants to serve the model from less GPU memory. Quantization stores the weights at lower precision, using a small sample of calibration text to set them. The question was whether that text has to come from the firm's own documents, which would mean handing them over.
Each of our three 4-bit builds cut the weights from 231.8 GiB to 68.1 GiB, a factor of 3.4. That fits on one H200, which has 139.8 GiB of usable memory. The original weights need at least two. On 512 held-out legal records, prediction error rose by 0.3% to 0.6%. And with the amount of calibration text matched, generic text gave the same result as legal text, within a margin we set before measuring. The customer keeps its documents.
We did not measure throughput, latency or the memory left for serving, so we cannot put a dollar figure on the saving. Our builds store 4-bit integer weights and keep activations at 16 bits, because the H200 has no 4-bit floating-point arithmetic. NVIDIA's NVFP4 build in the GA release uses a 4-bit floating-point format whose speed advantage needs Blackwell GPUs, and we did not test it.
The checkpoint also includes a multi-token prediction head: a small built-in module that drafts the next token for the full model to verify in one pass, a technique called speculative decoding. With the head on, single-request output on 20 math prompts rose from 202.9 to 316.9 tokens per second, a factor of 1.56, on four H200 GPUs, with nothing extra trained. Each setting ran once, and we did not check that the outputs were identical with the head on and off. This used the 09/21 checkpoint's head; the GA checkpoints ship a boosted one, which we have not measured.
Read the calibration text for 4-bit quantization of NVIDIA Nemotron 3.5 Super VL technical report.
Three experiments, one sourcing decision
Put the three together and they describe the owned tier of a token sourcing strategy. No customer data left our environment, the weights fit on one GPU, and the held-out test told us which adapters worked.
An organization starting with NVIDIA Nemotron 3.5 Super VL can run the same sequence we did. Pick one workload whose data cannot leave your environment and whose volume is steady. Hold out part of your own data before training anything. Train an adapter in an afternoon, on internal, generated or public data. Test it on the held-out set, question by question, on answers the model writes, and decide from that.
Put it behind a control plane, the layer that routes each request to a model, so the switch from a rented model to an owned one is invisible to the teams that use it and visible to the people who pay for it.
Related reading: Putting NVIDIA Nemotron 3.5 Lightning to the test.