As incomes have risen, it's important for Americans to find new ways to spend ever-increasing amounts of money. I propose that we spend some of it traveling to pick and eat fresh fruit that doesn't travel well.
Fresh Fruit Tourism
LLMs Can't See Pixels or Characters
Current LLMs almost always process groups of characters, called tokens, instead of processing individual characters. They do this for performance reasons: Grouping 4 characters (on average) into a token reduces your effective context length by 4x.
So, when you see the question "How many R's are in strawberry?", you can zoom in on [s, t, r, a, w, b, e, r, r, y], count the r's and answer 3. But when GPT-4o looks at the same question, it sees [5299 ("How"), 1991 (" many"), 460 (" R"), 885 ("'s"), 553 (" are"), 306 (" in"), 101830 (" strawberry"), 30 ("?")].
Make Data Pipelines Debuggable by Storing All Source References
A few jobs ago, I worked at company that collected data from disparate sources, then processed and deduplicated it into spreadsheets for ingestion by the data science and customer support teams. Some common questions the engineering team got were:
- Why is the data in some input CSV missing in the output?
- Why is data in the output CSV not matching what we expect?
To debug these problems, the process was to try to reverse engineer where the data came from, then try to guess which path that data took through the monolithic data processor.
This is the story of how we stopped doing that, and started storing references to all source data for every piece of output data.
Can Reasoning Models Avoid the Most Forbidden Technique?
In a recent post, Zvi described what he calls "The Most Forbidden Technique":
An AI produces a final output [X] via some method [M]. You can analyze [M] using technique [T], to learn what the AI is up to. You could train on that. Never do that.
You train on [X]. Only [X]. Never [M], never [T].
The article specifically discusses this in relation to reasoning models and Chain of Thought (CoT): if we train a model not to admit to lying in its CoT, it might still lie in the CoT and just not tell us.
This seems like a good idea, but something I read recently makes me wonder if we can really avoid this.
Tax-Optimized Risk in Portfolio Allocation
When you're subject to capital gains taxation, the government shares in some of the upside, but when you have capital losses, the government shares in the downside too. Because of this, the actual risk (and reward) of any given portfolio is lower than it seems. To counteract this, you should consider shifting your allocation toward riskier assets.