~www_lesswrong_com | Bookmarks (692)

Published on January 24, 2025 6:01 PM GMTI summarize my learnings and thoughts on Liron Shapira's discussion with Ken Stanley on the Doom Debates podcast. I refer to them as LS and KS respectively.High level summaryKey beliefs of KS:Future superintelligence will be 'open-ended'. Hence, thinking of them as optimizers will lead to incomplete thinking and risk mitigations.P(doom) is non-zero, but no fixed number. Changes...

Published on January 24, 2025 5:12 PM GMTThis is something I’ve been thinking about since my synthesizability article.Let’s assume, given the base twenty amino acids that are naturally present in the human body, we have every possible permutation of them for up to 100 amino acids, stored in a box with pH 7.4 water and normal pressures and temperature and isolated from one another....

Published on January 24, 2025 3:20 PM GMTThere was a comedy routine a few years ago. I believe it was by Hannah Gadsby. She brought up a painting, and looked at some details. The details weren’t important in and of themselves. If an AI had randomly put them there, we wouldn’t care. Except an AI didn’t put them there. And they weren’t there at...

Published on January 24, 2025 1:54 PM GMTPreviously in the series: The laws of large numbers and Basics of Bayesian learning.Reminders: formalizing learning in ML and Bayesian learningLearning and inference in neural nets and Bayesian modelsAs a very basic sketch, in order to specify an ML algorithm one needs five pieces of data. An architecture: i.e., a parametrized space of functions that associates to each weight...

Published on January 24, 2025 10:39 AM GMTSay an LLM agent behaves innocuously in some context A, but in some sense “knows” that there is some related context B such that it would have behaved maliciously (inserted a backdoor in code, ignored a security bug, lied, etc.). For example, in the recent alignment faking paper Claude Opus chooses to say harmful things so that on...

Published on January 24, 2025 5:18 AM GMTPhysiology seemed like a grab-bag of random processes which no one really understands. If you understand a physiological process—congratulations, that idea probably doesn’t transfer much to other domains. You just know how humans—and maybe closely related animals—do the thing. At least, that’s how I felt. (These sentiments tend to feel sillier when spelled out.)I haven't totally changed...

Published on January 24, 2025 4:57 AM GMTA lot of my recent research work focusses on:1. building the case for why perfect surveillance is becoming increasingly hard to avoid in the future2. thinking through the implications of this, if it happenedWhen I say perfect surveillance, imagine everything your eyes see and your ears hear is being broadcast 24x7x365 to youtube (and its equivalents in...

Published on January 24, 2025 4:30 AM GMTI recently read Darren McKee's book "Uncontrollable: The Threat of Artificial Superintelligence and the Race to Save the World". I recommend this book as the best current introduction to AI risk for people with limited AI background. It prompted me to update my thinking about Asimov's Laws and related risks in light of recent evidence about AI...

Published on January 23, 2025 11:30 PM GMT I think of a standard contra dance as running 8pm-11pm: three hours is a nice amount of time for dancing, and 8pm is late enough that dinner isn't rushed. Looking over the 136 regular Free Raisins dances from 2010 to 2019 matches my impression: 85% were 3hr, 62% started at 8pm, and 51% did both. I...

Published on January 23, 2025 10:16 PM GMTCross posted from SubstackContinuing the Stanford CS120 Introduction to AI Safety course readings (Week 2, Lecture 1)This is likely too elementary for those who follow AI Safety research - my writing this is an aid to thinking through these ideas and building up higher-level concepts rather than just passively doing the readings. Recommendation: skim if familiar with...

Published on January 22, 2025 6:36 PM GMT(and as a completely speculative hypothesis for the minimum requirements for sentience in both organic and synthetic systems)Factual and Highly PlausibleModel latent space self-organizes during training. We know this. You could even say it's what makes models work at all.Models learn any patterns there are to be learned. They do not discriminate between intentionally engineered patterns or...

Published on January 22, 2025 5:36 PM GMTThis post is about ‘ut’, a gender-neutral pronoun I am proposing and plan to use experimentally in the future on my blog and here on lesswrong.(The u of ‘ut’ sounds like the u of uber, not the u of utter.)The pronoun is used to refer to a single individual (singular) whose gender is unspecified. It follows the...

Published on January 22, 2025 4:54 PM GMTCross-posted from Telescopic TurnipAs we all know, humans are terrible at building butterflies. We can make a lot of objectively cool things like nuclear reactors and microchips, but we still can't create a proper artificial insect that flies, feeds, and lays eggs that turn into more butterflies. That seems like evidence that butterflies are incredibly complex machines...

Published on January 22, 2025 3:41 PM GMTNote: This report was conducted in June 2024 and is based on research originally commissioned by the Future of Life Foundation (FLF). The views and opinions expressed in this document are those of the authors and do not represent the positions of FLF.This report investigates Training Data Attribution (TDA) and its potential importance to and tractability for...

Published on January 22, 2025 3:40 PM GMTNote: This report was conducted in June 2024 and is based on research originally commissioned by the Future of Life Foundation (FLF). The views and opinions expressed in this document are those of the authors and do not represent the positions of FLF.This report investigates Training Data Attribution (TDA) and its potential importance to and tractability for...

Published on January 22, 2025 11:48 AM GMTtl;dr: If a copy is not identical to the original, MWI predicts that I will always observe myself surviving failed Mars teleportations rather than becoming the copy on Mars. BackgroundThe classic teleportation thought-experiment asks whether a perfect copy is "you". This normally presents as a pure decision problem – do you step into the teleporter? But I suggest...

Published on January 22, 2025 10:45 AM GMTThis is a linkpost for an article I've written for my blog. Readers of LessWrong may want to skip the intro about Bayesian Reasoning, but might find the application to the Peter Miller vs Rootclaim debate quite interesting.I’ve been a fan of Bayesian Reasoning since the time I’ve read Harry Potter and the Methods of Rationality. In...

Published on January 22, 2025 9:46 AM GMTOn blanket criticism and refutationIn his long post on the subject, Charbel-Raphaël argues against theories of impacts of interpretability. I think it's a largely a good, well-argued post, and if the only thing you get out of it is reading that post, I'll be contributing to improving the discourse. There is other material with similar claims that...

Published on January 22, 2025 7:06 AM GMTSolution concepts in game theory—like the Nash equilibrium and its refinements—are used in two key ways. Normatively, they proscribe how rational agents ought to behave. Descriptively, they propose how agents actually behave when interactions settle into equilibrium. The Nash equilibrium[1] underpins much of modern game theory and its applications in economics, political science, and evolutionary biology. Here, we focus on the descriptive use...

Published on January 22, 2025 4:06 AM GMTIf there was a truly confirmed sentient AI, nothing it said could ever convince me, because AI cannot be sentient. Nothing to See HereI suspect at least some will be nodding in agreement with the above sentiment, before realizing the intentional circular absurdity. There is entrenched resistance to even trying to examine the self-report of sentience as a...

Published on January 21, 2025 6:09 PM GMTLike this piece? It's cross-posted from by blog: https://collisteru.net/writing/This is a work of fiction and parody. I have done my best to get the scientific details right as far as they are known today, but my real goal is social commentary, not scientific accuracy. NOAA DISCOVERS INSCRIBED METEOR ARTIFACT UNDERNEATH ATLANTICDec 8, 2018Scientists from NOAA claim to have...

Published on January 21, 2025 5:34 PM GMTGetting personalised advice from a real human can help newcomers to AI safety figure out how to contribute most effectively. For example, I (Bryce) ended up in my current role largely thanks to a call with 80,000 Hours.There are a number of organisations and individuals offering advisory calls, but many people who want to work on AI...

Published on January 21, 2025 5:27 PM GMTCrossposted on The Field Building Blog and the EA forum.Discuss