bookmarks

• June, 2025: the Apple reasoning paper confirmed that even with “reasoning”, LLMs still can’t solve distribution shift, the core Achille’s heel in neural networks that I have been writing about for nearly 30 years.

Luxury is a marker that we can afford to do something others might consider wasteful.

One of the most common mistakes I’ve seen is developers using SELECT DISTINCT as a quick way to eliminate duplicates that appear after a bad join. It’s an easy fix, but it hides a deeper problem. Usually, the duplicates exist because the join condition is incomplete or the relationship between tables isn’t truly one-to-one

But with conventional software, someone with inside knowledge can usually deduce what’s going on,

worked for a dozen years — will have a good idea why. “Here’s what really terrifies me” about the current breed of artificial intelligence (AI), he says: “there is no such understanding”, even among the people building it.

Martin Wattenberg, a computer scientist at Harvard University in Cambridge, Massachusetts, says that understanding the behaviour of LLMs could even help us to grasp what goes on inside our own heads.

stochastic parrots

some say more is going on, including reasoning and other startlingly human-like abilities

The researchers described the model’s behaviour as role-playing — doing more than parroting but less than planning.

When they asked their LLM whether it consented to being shut down, they found it drew on several source materials with the theme of survival to compose a compelling response (see ‘Lust for life’).

trained an LLM from scratch to play the board game Othello,

The team successfully trained a smaller model to interpret the internal activations of the AI, and discovered that it had constructed an internal map of the discs based on the text descriptions of the gameplay2

Because chatbots can chat, some researchers interrogate their workings by simply asking the models to explain themselves. This approach resembles those used in human psychology. “

The researchers first intentionally biased their study models by, say, giving them a series of multiple-choice questions for which the answer was always option A. The team then asked a final test question. The models usually answered A — whether correct or not — but almost never said that they chose this response because the answer is usually A

“It’s a little weird to study [LLMs] the way we study humans,” Bau says. But although there are limits to the comparison, the behaviour of the two overlaps in surprising ways.

“It is nonsensical to say that an LLM has feelings,” Hagendorff says. “It is nonsensical to say that it is self-aware or that it has intentions. But I don’t think it is nonsensical to say that these machines are able to learn or to deceive.”

Developers train an LLM on large quantities of text to repeatedly predict the next text fragment, a process called pre-training. Then, when the LLM is given a text prompt, it generates a continuation. Offered a question, it predicts a plausible answer. Most LLMs are then fine-tuned to align with developers’ goals

the interface might append a ‘system prompt’ to each user prompt

external documents

Librephone is a new initiative by the FSF with the goal of bringing full freedom to the mobile computing environment. The vast majority of software users around the world use a mobile phone as their primary computing device. After forty years of advocacy for computing freedom, the FSF will now work to bring the right to study, change, share, and modify the programs users depend on in their daily lives to mobile phones.

I've written at length about how I manage my digital consumption, from turning off notifications to forgoing social media entirely. The underlying premise here is that if you're trying to lose weight, you shouldn't carry cookies around in your pockets. And my phone is the bag of cookies in this metaphor.

One of the most consequential moments in California’s drive to beat back climate change will take place next month. The state will stop receiving electricity from the Intermountain Power Plant in Central Utah, meaning our reliance on coal as a source of power will essentially be over.

LLMs don’t copy-paste (or cut and paste) code. For instance, when you ask them to refactor a big file into smaller ones, they’ll "remember" a block or slice of code, use a delete tool on the old file, and then a write tool to spit out the extracted code from memory. There are no real cut or paste tools.

many people don’t realize how fast and aggressively the Trump administration is moving to end rule of law and convert America into a full-fledged autocracy

U.S. Border Patrol agents deployed to Chicago led a late-night raid on an apartment building this week, rappelling from helicopters onto rooftops and breaking down doors in an operation authorities said targeted gang members but which swept up U.S. citizens and families.

As part of the raid, some U.S. citizens were temporarily detained and children pulled from their beds, according to interviews with residents and news reports. Building hallways were still littered with debris two days later.

This October, the Internet Archive will celebrate an extraordinary milestone: 1 trillion web pages preserved and available for access via the Wayback Machine.

Is there a seahorse emoji? Let's ask GPT-5 Instant:

Laptops create. Phones consume. The split determines who commands and who complies—a power dynamic hiding in plain sight across every interface we touch.

My high school teacher in 2004 accused me of plagiarizing from Wikipedia because my research paper looked "too polished" for something typed on a keyboard instead of handwritten. Twenty years later, HN commenters see clean prose and assume LLM slop. Same discomfort, different decade, identical pattern: people resist leverage they haven't internalized yet.

use AI tools the way I used spell-check, grammar tools, and search engines before them—as cognitive leverage, not cognitive replacement.

I prefer to communicate with them via email rather than chat messaging. There are many benefits in such a choice.

the ability to sit patiently with a group of barely connected or disconnected ideas that allows a thinker to braid them together into something that is combinatorially new

The problem of the next 18 months isn’t AI disemploying all workers, or students losing competition after competition to nonhuman agents. The problem is whether we will degrade our own capabilities in the presence of new machines. We are so fixated on how technology will outskill us that we miss the many ways that we can deskill ourselves.

If you feel you are being watched, you change your behavior. Big Data is supercharging this effect.

A fire at the National Information Resources Service (NIRS)'s Daejeon headquarters destroyed the government’s G-Drive cloud storage system, erasing work files saved individually by some 750,000 civil servants, the Ministry of the Interior and Safety said Wednesday.