~www_lesswrong_com | Bookmarks (720)

Published on March 13, 2025 7:18 PM GMTWe study alignment audits—systematic investigations into whether an AI is pursuing hidden objectives—by training a model with a hidden misaligned objective and asking teams of blinded researchers to investigate it.This paper was a collaboration between the Anthropic Alignment Science and Interpretability teams.AbstractWe study the feasibility of conducting alignment audits: investigations into whether models have undesired objectives. As...

Published on March 13, 2025 6:29 PM GMTWe (SaferAI) propose a risk management framework which we think should improve substantially upon existing Frontier Safety Frameworks if followed. It introduces and borrows a range of practice and concepts from other areas of risk management to introduce conceptual clarity and generalize some early intuitions that the field of AI safety independently came up with.To maintain readability...

Published on March 13, 2025 6:17 PM GMTWhy do adults pursue long-term and complex goals? People don’t come into the world wanting to amass followers on Instagram, get a Ph.D., or become rich, yet these and a myriad of other goals are what seem to motivate most people. I have been interested in how these complex and long-term goals develop, and I think it...

Published on March 13, 2025 6:16 PM GMTThis post is a distillation of a recent work in AI-assisted human coordination from Google DeepMind. The paper has received some press attention, and anecdotally, it has become the de-facto example that people bring up of AI used to improve group discussions. Since this work represents a particular perspective/bet on how advanced AI could help improve human coordination, the following explainer...

Published on March 13, 2025 10:24 AM GMTI scrutinise the so-called "reversal curse", wherein LLMs seem not consier inverse relationships between conceptual nodes.I show that, far from being a proof of a lack of logical skills, it is a normal artefact of saliency, known in humans as associative recall asymmetry, and propose a conceptual-network model of the causes which works independently of substrate.Discuss

Published on March 13, 2025 2:40 PM GMTThe most hyped event of the week, by far, was the Manus Marketing Madness. Manus wasn’t entirely hype, but there was very little there there in that Claude wrapper. Whereas here in America, OpenAI dropped an entire suite of tools for making AI agents, and previewed a new internal model making advances in creative writing. Also they...

Published on March 13, 2025 12:58 PM GMTIf there is an international project to build artificial general intelligence (“AGI”), how should it be designed? Existing scholarship has looked to historical models for inspiration, often suggesting the Manhattan Project or CERN as the closest analogues. But AGI is a fundamentally general-purpose technology, and is likely to be used primarily for commercial purposes rather than military...

Published on March 13, 2025 12:08 PM GMTIntroSo far we have identified lock-in risk, defined lock-in, and established threat models for particularly undesirable lock-ins. Now we present this evaluation benchmark for large language models (LLMs) so we can measure (or at least, get a proxy measure for) the risk level of LLMs.AI is the key technology in the manifestation of lock-in risks; AI systems...

Published on March 13, 2025 10:52 AM GMTThis started life as a reaction to a post by Raemon, in which he calls on Anthropic to be more willing to halt its research if it enters a danger zone, and for them to take the relevance of technical philosophy much more seriously. My personal opinion is that at this point, even slowing things down (I mean...

Published on March 13, 2025 9:40 AM GMTCrossposted on the EA Forum.Displacement of other Space-Faring Civilizations (SFCs) may have macrostrategic impact for longtermists as hinted in a previous post. For example, our causal impact depends on how much value we produce relative to how much value would have been produced without us. In this post, we clarify the concept of Civ-Saturation, the metrics to...

Published on March 11, 2025 5:45 PM GMTEpistemic status: Speculative pattern-matching based on public information. In 2023, Gwern published an excellent analysis suggesting Elon Musk exhibits behavioral patterns consistent with bipolar II disorder. The evidence was compelling: cycles of intense productivity followed by periods of withdrawal, risk-taking behavior (like crashing an uninsured McLaren), reduced sleep requirements during "up" phases, and self-reported "great highs, terrible lows."Gwern's analysis...

Published on March 11, 2025 3:57 PM GMTTL;DR Large language models have demonstrated an emergent ability to write code, but this ability requires an internal representation of program semantics that is little understood. Recent interpretability work has demonstrated that it is possible to extract internal representations of natural language concepts, raising the possibility that similar techniques could be used to extract program semantics concepts....

Published on March 11, 2025 3:39 PM GMTForethought[1] is a new AI macrostrategy research group cofounded by Max Dalton, Will MacAskill, Tom Davidson, and Amrit Sidhu-Brar.We are trying to figure out how to navigate the (potentially rapid) transition to a world with superintelligent AI systems. We aim to tackle the most important questions we can find, unrestricted by the current Overton window.More details on our website.Why...

Published on March 11, 2025 3:38 PM GMTThis is a linkpost for a new paper called Preparing for the Intelligence Explosion, by Will MacAskill and Fin Moorhouse. It sets the high-level agenda for the sort of work that Forethought is likely to focus on.Some of the areas in the paper that we expect to be of most interest to EA Forum or LessWrong readers are:Section 3...

Published on March 11, 2025 2:30 PM GMTEpistemic status: The following isn't an airtight argument, but mostly a guess how things play out.Consider two broad possibilities:I. In worlds where we are doing reasonably well on alignment, AI control agenda does not have much impact.II. In worlds where we are failing at alignment, AI control may primarily shift probability mass away from "moderately large warning...

Published on March 11, 2025 1:35 PM GMTThis is a response to Matt's earlier post. If you see "a large mixture of alignment proxies" when you look at a standard loss function, my post might save you from drawing silly conclusions from the earlier post. If you parse the world into non-overlapping magisteria of "validation losses" and "training losses", then you should skim this...

Published on March 11, 2025 11:52 AM GMTTL;DR: We provide some evidence that Claude 3.7 Sonnet doesn’t encode hidden reasoning in its scratchpad by showing that training it to use paraphrased versions of the scratchpads does not degrade performance.The scratchpads from reasoning models look human understandable: when reasoning about a math problem, reasoning models consider intermediate steps similar to the ones I would use,...

Published on March 11, 2025 5:50 AM GMTThose engaged with questions of how to make the world a better place are probably all too familiar with the infighting that occurs among people who are on the same side of a political issue because of differences in methodology.  Among those concerned about AI x-risk, for example, PauseAI activists sometimes criticize LessWrong and EA participants for...

Published on March 11, 2025 4:56 AM GMTCognitive reframing is a powerful psychological technique that encourages subtle shifts in perspective, which can result in more positive and empowering interpretations to help challenge everyday negative thoughts and behaviors.First I’ll tell the story of how I accidentally discovered cognitive reframing on my own, then I’ll define what the technique is, state why it’s useful (particularly for...

Published on March 11, 2025 3:14 AM GMTYou learn the rules as soon as you’re old enough to speak. Don’t talk to jabberjays. You recite them as soon as you wake up every morning. Keep your eyes off screensnakes. Your mother chooses a dozen to quiz you on each day before you’re allowed lunch. Glitchers aren’t human any more; if you see one, run....

Published on March 10, 2025 6:06 PM GMTI just saw on twitter someone claiming that we "don't know how to raise the birth rate". No need to link them, it's a fairly common sentiment.Have you actually tried raising the birth rate? For real, as in actually put in any effort?Let's run the numbers on the stupidest possible thing that could work. Offer anyone who...

Published on March 10, 2025 4:07 PM GMTProduced as part of the ML Alignment & Theory Scholars Program -  Winter 2024-25 Cohort**Epistemic status**: I received positive feedback when I presented this idea to people at EAG, and my MATS mentor Evan Hubinger suggested I make a more detailed write-up to get feedback from the community. This proposal is at the theoretical stage with planned...

Published on March 10, 2025 3:35 PM GMTNote: This post is intended to be the first in a broader series of posts about the difficult tradeoffs inherent in public access to powerful open source models. While this post highlights some dangers of open models and discusses the possibility of global regulation, I am not, in general, against open source AI, or supportive of regulation...