I tried to determine why LLMs don’t write well. What is the problem? I dove immediately into training then needed to backtrack. It was difficult to reward good writing. In my following tests, I found that often what is labelled semantics in NLP is not actually semantics.

What does semantic mean?

From Cambridge Dictionary, semantic is “the literal or underlying meaning of words, phrases, and symbols, rather than their grammatical structure or surface form”. The grammatical or surface form is referred to as lexical. I’m going to use these terms throughout my writing. For most LLMs, it is important we know the level of semantic understanding they represent.

There is an MTEB benchmark specifically for this determination. STS is supposed to measure how well a model performs in clustering semantically similar texts. It is scored 0 - 100 and is based on human labels. From the English version of MTEB I found a 600m parameter, pareto frontier model: infgrad/Jasper-Token-Compression-600M. Not only pareto frontier but at the time it was significantly smaller than #1 on the benchmark but was still ranked #2!

Lexical Embedding Models

My environment to train LLMs to write better started simple. Teach a model to write like a specific author (or collection of authors). If you do this, then you can start to soup models to create new and unique writing styles. You can see similar work from early Elevenlabs where they cloned voices then began to mix their own. I began with the naive approach of using a centroid from the author's embeddings. The model was rewarded when it produced text which was similar to the author’s embeddings. To keep the task simple, I started this as a humanizer. Take text and transform it into author-like text. Initially, I used the Jasper embedding model. The writing would eventually degenerate during training.

I did then try a few other methods to correct for this degeneration but I kept coming back to the idea that the judge model was not strong enough. So I began to design some tests. An initial test generated a 2x2 grid of paragraphs. The goal was to determine if a paragraph was semantically similar to another paragraph. I used high/low lexical overlap and high/low semantic overlap as grid square categories. For example, a lexically similar paragraph which was semantically dissimilar would have many of the same words but little of the same meaning.

To perform this test, the model embedded an initial paragraph. We would then compare cosine distance between two grid square rewrites. The most revealing test was the low semantic but high lexical overlap compared to the high semantic but low lexical overlap test. The Jasper model only chose 1 of 20 pairs from the high semantic but low lexical overlap as more similar. It had a 5% rate of success. Jasper was predominately lexical. It was picking rewrites preferentially on word overlap during training. This was leading to degeneration as it was weak enough semantically that it would reward mambo jumbo writing in the name of lexical similarity.

The Failure of STS

If the Jasper model is pareto frontier on MTEB English and has a high STS score, then what is the STS benchmark measuring? Said in a different way, how lexical is the STS benchmark?

A character based 3–5-gram TF-IDF achieved 0.695 macro Spearman across ten English STS tasks. That’s sort of crazy! A relatively naive lexical setup will score 70/100 on something which is supposed to be a semantic benchmark. That leaves at most 30 points for semantics.

The jasper model scores nearly 89/100 and still appears to be mostly a lexically driven model. From the 2x2 test, we can estimate the models semantic/lexical split. What I found was that Jasper is estimated to be between 9 - 23% semantic with an average of 15%. Jasper is 85% a lexical model according to its outputs. One of the largest semantic benchmarks used across industry don’t reliably measure semantic understanding in LLMs. At best, it is a mix which is primarily lexical.

Given the above, you can’t trust the STS to be a fair determination of semantic quality. The issue here may be that the model is simply too small. What if I needed a 100B embedding model? I can’t test that directly but I can get close with a different model type.

Decoder LLMs to the Rescue

Lucky for me, there are a number of LLMs which have model size classes. I used the Smollm2 and Qwen3 family of models from 135m parameters to 32B parameters. Instead of using the embedding from an intermediate layer, I asked the models which of the two rewritten paragraphs preserved material meaning from the original. Thinking was disabled for all models during this test.

The scores improved across the Smollm2 models from chance at 135m to 60% accuracy at 1.7B. Qwen3 immediately saturated the benchmark at 97.5% from 4B to 14B then fully 100% at 32B. However, when I used the 4B model as a judge for training, we again had degeneration. This implied that the 2x2 test was not sufficient.

To fix this problem, I mined rewrites from training. I personally reviewed some then had an LLM grade the others with my examples as context. I manually reviewed every single grade. The test was to determine if one paragraph is semantically similar to another rewritten paragraph. This resulted in a small dataset of real paragraphs pairs from training. I had each model perform the test again for a total of 32 decisions per model from the new dataset.

Our second test (paragraph native), showed a more gradual rise in grading accuracy to model size. We did not see improvement from chance until the model size was 1.7B. We did not get close to 90% accuracy until the 14B model. This benchmark was not saturated even at our largest model size tested. I provide a chart and graph below of these tests.

Fig 1. Accuracy across model sizes on my 2x2 grid and the paragraph native test
Fig 2. The chart but in table form

The above seems to imply that semantic understanding requires a certain amount of scale. The paragraph native test is not easy. I had multiple rounds of reviewing labels because the decision was close. The point is that it is a demanding test. What is interesting is that easy semantic understanding emerges early (4B or earlier) while harder semantic understanding does not emerge until later in the model families (14B or later).

Anecdotally, it has been considered that 7-8B models or larger were a step size change in intelligence. I believe this anecdote comes from the semantic understanding. In terms of difficult understanding (paragraph native), we reach a threshold where it is correct at a rate humans would notice as trustworthy (70-80%) while also able to saturate very simple semantic understanding tests. I could be convinced this is the size required for a step change in understanding.

Going MoE

The next question is “what would be the best model in my price range?” My price range is what can fit and run quickly on a 80gb GPU. I ran Gemma 12B, Gemma 31B, Trinity Mini 26B3A and GLM-4.7 Flash 30B3A. The results follow:

Fig 3. MoE and Dense

What was surprising is how poorly the MoE performed on the semantic test. Trinity performed worse than the Qwen 4B model despite 22B more parameters total and only 1B fewer active. Equally as surprising was the Gemma model which had the 12B model surpassing the Qwen3 32B model on the paragraph rewrites. My takeaway is that Gemma sees much more training data (which anecdotally agrees with how Google rumors to train). This implies the scaling is both compute and parameter based for semantic understanding.

It also implies something about MoE architectures. I have not dwelt on this long enough to have a final thought. The easy answer is that I should compare models that serve at similar per token rates. The active parameters are not enough to enable broad semantic understanding compared to dense models at similar total parameter counts. That is fine. Maybe MoE’s do acquire more semantic understanding per active size relative to dense models. This is straightforward enough to test at a later date.

Big Model Smell

Semantic understanding is a simple framework to understand big model smell. It also explains why that hasn’t been a meme used recently. This essay is supposed to be an introduction to the collection so I will keep this section short. I will have a longer section in an upcoming essay. Big model smell is derived from the semantic understanding of the model. It is hard to quantify. When people use a model, one which has greater semantic understanding will come across more capable. It doesn’t matter how you word something as long as the vibe is correct. The model will understand.

This tracks well with what I have anecdotally seen. Models which pass the vibe test tend to either have extensive pretraining or are semantically strong. This tended to be a good proxy for size of the model. Reasoning models break the smell paradigm. The long and short is that reasoning models are trading horizon for capacity. This has currently failed to improve their lexical/semantic understanding to the extent a larger mode gets naturally. Their written speech thus comes across small model smelling despite benchmark capabilities increasing.

Closing

The main point I wanted to make in this essay is the difference between lexical and semantic understanding. Despite benchmarks to measure semantics, there is a lack of differentiation between lexical and semantics. There appear to be clear scaling gains for semantic understanding and a lexical regime inhabited by small models. This idea explains a decent amount of anecdotal understanding from ML researchers and the general public.

I don’t expect this essay to be wrapping anything up. It raises many more questions than it answers. However, I have found the lexical vs semantic regime to be useful as a lens to look at LLMs and their development. I will continue to use this framework for the essays in this collection.