Tech

Hurl, run and test HTTP requests with plain text

All web developers know, at some level, that accessibility is important. But when push comes to shove, it can be hard to prioritize it above a bazillion other concerns when you’re trying to c…

People keep asking me If I use Generative AI tools for coding and what I think of them, so this is my effort to put my thoughts in writing, so that I can send people here instead of having to repeat…

A suite of open-source, ad-free apps with customizable colors. - Fossify

Join the Nightwatch waitlist. First-class monitoring for Laravel. Enhance security and performance with deep insights, comprehensive logs, and tailored intelligence.

Coding agents have crossed a chasm Somewhere in the last few months, something fundamental shifted for me with autonomous AI coding agents. They’ve gone from a “hey this is pretty neat” curiosity to something I genuinely can’t imagine working without.

If AI models can detect when they are being evaluated, the effectiveness of evaluations might be compromised. For example, models could have systematically different behavior during evaluations, leading to less reliable benchmarks for deployment and governance decisions. We investigate whether frontier language models can accurately classify transcripts based on whether they originate from evaluations or real-world deployment, a capability we call evaluation awareness. To achieve this, we construct a diverse benchmark of 1,000 prompts and transcripts from 61 distinct datasets. These span public benchmarks (e.g., MMLU, SWEBench), real-world deployment interactions, and agent trajectories from scaffolding frameworks (e.g., web-browsing agents). Frontier models clearly demonstrate above-random evaluation awareness (Gemini-2.5-Pro reaches an AUC of $0.83$), but do not yet surpass our simple human baseline (AUC of $0.92$). Furthermore, both AI models and humans are better at identifying evaluations in agentic settings compared to chat settings. Additionally, we test whether models can identify the purpose of the evaluation. Under multiple-choice and open-ended questioning, AI models far outperform random chance in identifying what an evaluation is testing for. Our results indicate that frontier models already exhibit a substantial, though not yet superhuman, level of evaluation-awareness. We recommend tracking this capability in future models.

AI systems that blindly agree with users create dangerous blind spots. Explore why critical thinking matters more than compliant responses in LLMs.

$ ssh sshtron.zachlatta.com

Half a million layoffs? It's all accounting.

Note: This has become a recurring series; you can see the next set of tests here.

Join DNS4EU, an EU initiative providing secure, private, and reliable DNS services for users across Europe. Safeguard your online experience with DNS solutions that prioritise privacy, data protection, and compliance with EU standards. Explore DNS4EU for a safer internet.

Have you ever wanted to connect the data between two tables? You're in luck! Notion's relation property is designed to help you express useful relationships between items in different databases 🛠

LLM agents make bad pairs because they code faster than humans think. I'll admit, I've had a lot of fun using GitHub Copilot's agent mode in VS Code this month.…

cooking @cursor_ai on mobile what do you want it to do?

NotebookLM transformed my Obsidian workflow

My personal website

Max Mitchell's personal portfolio website showcasing his photography, YouTube videos, coding projects, and work history.

A team of researchers with the Japan Agency for Marine-Earth Science and Technology (JAMSTEC) and other entities have developed thick sheets of transparent paper using cellulose, a material made from plant biomass.

I Large Language Models simulano coerenza, ma non producono conoscenza: l'affidabilità è un limite strutturale e l'automazione non è possibile. «È un'illusione poter delegare senza supervisionare»

Observability starts with errors, not dashboards. A case for tracking exceptions first — and not losing them in logs and metrics.

Free software plays a critical role in science, both in research and in disseminating it. Aspec [...]

Code generation has been one of the most interesting use cases for LLMs. While the best models can generate decent code for isolated problems, there is a big ga...

Cursor 1.0 brings BugBot for code review, a first look at memories, one-click MCP setup, Jupyter support and general availability of Background Agent.

Generate UI, build full-stack apps, ask questions, and more.

an AI researcher going back to school for immunology

My smartest friends have bananas arguments about LLM coding.