Top postsTop post
AI Safety veteran Holden Karnofsky thinks there’s a 49% chance his actions are making things worse.[1] In 2025, Jesse Clifton even stepped down as the executive director of the Center on Long-Term risk because of similar reasons. Even top AI Safety strategists don’t know what will make things better, and...
Micheal Vassar’s strategy for saving the world is horrifyingly counterproductive. Olivia’s is worse. A note before we start: A lot of the sources cited are people who ended up looking kinda insane. This is not a coincidence, it’s apparently an explicit strategy: Apply plausibly-deniable psychological pressure to anyone who might...
tl;dr: with multiple agents, control attempts tend to create conflict, because control attempts shut down communications channels, which leads to feedback loops in the form of intensifying tug-of-war over variables. intentionally relaxing control to better understand the other agents can break the cycle, and forms the basis of many therapeutic...
tl;dr: progress on making Claude friendly[1] is not the same as progress on making it safe to build godlike superintelligence. solving the former does not imply we get a good future.[2] please track the difference. [edit: terminology note: Local vs Asymptotic Alignment would capture the thing I'm trying to say...
Most ideas fail. Successful startup founders don't get their first idea right. So they try things out, iterate relentlessly, and pivot when needed –[1] until they find something that works. In AI Safety, most ideas won't be impactful either. However, most people don’t know how to systematically move quickly and...
tl;dr: Mixing goal-directedness into cognitive processes that are working to truth-seek about possible futures tends to undermine both truth-seeking and effective pursuit of your goals. Cleanly separating them has nice properties. [Epistemic Status: Mostly empirically observed rather than rigorously justified, but have seen it many times across different people and...
Your project might be failing without you even knowing it. It’s hard to save the world. If you’re launching a new AI Safety project, this sequence helps you avoid common pitfalls. Your most likely failure modes along the way: You never get started. Entrepreneurship is uncomfortable, and AI Safety is...