How Specializing the Vocabulary of SLMs Can Cut Inference Cost by 77% in Real Applications

Models Datasets Spaces Buckets new Docs Enterprise Pricing Website Tasks HuggingChat Collections Languages Organizations Community Blog Posts Daily Papers Hardware Learn Discord Forum GitHub Solutions Team & Enterprise Hugging Face PRO Enterprise Support Inference Providers Inference Endpoints Storage Buckets --[0--> --]--> Back to Articles a]:hidden"> How Specializing the Vocabulary of SLMs Can Cut Inference Cost by 77% in Real Applications Team Article Published October 21, 2026 Upvote 23 +17 Bernardo França de Aguiar bernardoaguiar Follow Dharma-AI Gabriel Pimenta de Freitas Cardoso GabrielPimenta99 Follow Dharma-AI Bruno Duarte brunoduarte01 Follow Dharma-AI Francisco de Almeida Rocha Alves falves9101 Follow Dharma-AI :last-child]:mb-0"> A Vocabulary Built for Everyone Reading Is Cheap, Writing Is Expensive Why Existing Solutions Fell Short How We Solved It What Changed in the Grader Beyond the Grader Paying Only for the Tokens That Matter Sources Further Reading Models specialized for a domain or a task by fine-tuning general open-source models, such as Qwen and Gemma, inherit a vocabulary built for many domains and general applications. In an essay grader we run in production, that vocabulary split the text the model writes most into many tokens.

What changed

Price and bundle value

In an essay grader we run in production, that vocabulary split the text the model writes most into many tokens. This article previews some results from the VocabX paper, which we will publish soon: how we specialized the vocabulary around the cost of each token rather than its frequency, and cut measured inference cost by 77% without changing the base model.

AI features that change the value

The model was already small, at 4 billion parameters, and had already been trained for the task. Its tokenizer, however, was still the original Qwen3-4B one, built for many languages and many kinds of text. In the grader, it was used for a single language, Portuguese, and a single response format. That had two costs: the text the grader writes on every request took more tokens than it needed, and at every generated token the model scored a vocabulary of 151,669 entries, including many this task almost never uses.

Who should pay attention

A Vocabulary Built for Everyone The leading language models read and write in tokens, and the tokenizer decides how text becomes tokens. The tokenizers of the leading general-purpose models are trained on large collections of text from many domains and languages, from source code to contracts to Chinese. For a general-purpose model, that choice makes sense.

Patrick Tech Media take

Patrick Tech Media reads this kind of move as a real utility race: the package that removes steps, bundles tools, and lowers hidden cost usually wins longer than the launch buzz. Patrick Tech Media will keep checking Hugging Face Blog to see whether the value holds once the rollout broadens.

Source notes

Hugging Face Blog