Bookmarks (679)

Published on May 3, 2024 6:26 PM GMTI've recently read some cool posts on rationality through history (e.g., see here), and want to see if there are more examples! So: do you know of any "ancient" individual figures or institutions that you would consider rational? Or at least from several centuries ago.Discuss

Published on May 3, 2024 6:10 PM GMTMany thanks to Spencer Greenberg, Lucius Caviola, Josh Lewis, John Bargh, Ben Pace, Diogo de Lucena, and Philip Gubbins for their valuable ideas and feedback at each stage of this project—as well as the ~375 EAs + alignment researchers who provided the data that made this project possible.BackgroundLast month, AE Studio launched two surveys: one for alignment researchers, and another...

Published on May 3, 2024 6:10 PM GMTNicky Case, of "The Evolution of Trust" and "We Become What We Behold" fame (two quite popular online explainers/mini-games) has written an intro explainer to AI Safety! It looks pretty good to me, though just the first part is out, which isn't super in-depth. I particularly appreciate Nicky clearly thinking about the topic himself, and I kind...

Published on May 3, 2024 5:48 PM GMTNicky Case makes applets for explaining concepts. You may have seen their explainer on game theory. They have written an AI safety explainer with comics, working with a group called Hack Club. Discuss

Published on May 3, 2024 5:43 PM GMTThis is an accessible introduction to AI Safety, written by Nicky Case and the teens at Hack Club. So far, part 1/3 is completed, which covers a rough timeline of AI advancement up to this point, and what might come next.If you've got feedback as to how this can be made more understandable, that'd be appreciated! Reach...

Published on May 3, 2024 8:40 AM GMTEvery LLM in existence is a blackbox, and alignment relying on tuning the blackbox never succeeds - that is evident by that fact that even models like ChatGPT get jailbroken constantly. Moreover, blackbox tuning has no reason to transfer to bigger models.A new architecture is required. I propose using an LLM to parse environment into planner format...

Published on May 3, 2024 1:54 PM GMTThis is cross-posted from our website: https://www.convergenceanalysis.org/publications/ai-clarity-an-initial-research-agendaExecutive Summary Transformative AI (TAI) has the potential to solve many of humanity's most pressing problems, but it may also pose an existential threat to our future. This significant potential of TAI warrants careful study of the AI’s possible trajectories and their corresponding consequences. Scenarios in which TAI emerges within the next...

Published on May 3, 2024 12:36 PM GMTTLDR – Apply now to ESPR and PAIR. ESPR welcomes students between 16-19 years. PAIR is for students between 16-21 years.The FABRIC team is running two immersive summer workshops for mathematically talented students this year.The Program on AI and Reasoning (PAIR) is for students with an interest in artificial intelligence, cognition, and minds in general. We will study how current AI systems...

Published on May 3, 2024 9:41 AM GMTMeta: I'm writing this in the spirit of sharing negative results, even if they are uninteresting. I'll be brief. Thanks to Aaron Scher for lots of conversations on the topic.SummaryProblem statementYou are given a sequence of 100 random digits. Your aim is to come up with a short prompt that causes an LLM to output this string...

Published on May 3, 2024 1:18 AM GMTAnnouncing the first academic Mechanistic Interpretability workshop, held at ICML 2024! I think this is an exciting development that's a lagging indicator of mech interp gaining legitimacy as an academic field, and a good chance for field building and sharing recent progress! We'd love to get papers submitted if any of you have relevant projects! Deadline May 29, max...

Published on May 1, 2024 6:12 PM GMTNate Silver tries to answer the question: "How do people formulate their political beliefs?"An important epistemological question that is, he says, under-discussed.He lays out his theory:I think political beliefs are primarily formulated by two major forces:Politics as self-interest. Some issues have legible, material stakes. Rich people have an interest in lower taxes. Sexually active women (and men!)...

Published on May 1, 2024 5:15 PM GMTI've been going through the FAR AI videos from the alignment workshop in December 2023. I'd like people to discuss their thoughts on Shane Legg's 'necessary properties' that every AGI safety plan needs to satisfy. The talk is only 5 minutes, give it a listen.Otherwise, here are some of the details:All AGI Safety plans must solve these...

Published on May 1, 2024 4:50 PM GMTAbstract:Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. We...

Published on May 1, 2024 8:02 AM GMTCoauthored by Dmitrii Volkov1.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0;...

Published on May 1, 2024 1:06 PM GMTACX recently posted about the Rootclaim Covid origins debate, coming out in favor of zoonosis. Did the post change the minds of those who read it, or not? Did it change their judgment in favor of zoonosis (as was probably the goal of the post), or conversely did it make them think Lab Leak was more likely...

Published on May 1, 2024 10:18 AM GMTMain event pageFriday 13th September- Monday 16th September 2024 is the 11th annual Less Wrong Community Weekend (LWCW) in Berlin. This is the world’s largest rationalist social gathering which brings together 250+ aspiring rationalists from across Europe and beyond for four days of intellectual exploration, socialising and fun.We’re expanding to 250+ participants and taking over the whole...

Published on May 1, 2024 2:50 AM GMTYouTube link Top labs use various forms of “safety training” on models before their release to make sure they don’t do nasty stuff - but how robust is that? How can we ensure that the weights of powerful AIs don’t get leaked or stolen? And what can AI even do these days? In this episode, I speak...

Published on May 1, 2024 12:46 AM GMTIf you're working on neurotechnology for safe AI, including brain-computer interfaces or whole-brain emulation approaches, consider joining this upcoming workshop: 2024 Neuro/BCI/WBE for Safe AI WorkshopMay 21 - 22, 9 am - 5 pmLighthaven, BerkeleyGoalsWhole Brain Emulation (WBE) represents a promising technology for creating human-aligned software intelligence directly derived from human brains. Recently, there has been a shift...

Published on May 1, 2024 12:42 AM GMTIf you're working at the intersection between cryptogrpahy, secuity and AI, consider joining this upcoming workshop:Foresight's AGI: Cryptography, Security & Multipolar Scenarios WorkshopMay 14-15, all-dayThe Institute, Salesforce Tower, San FranciscoGoalsTo help AI development benefit humanity, Foresight Institute has held various workshops over the past years and launched a Grants Program that funds work on AI security risks,...

Published on April 29, 2024 3:41 PM GMTIn “Freedom under naturalistic dualism” I have carefully argued that consciousness is radically noumenal, that is, it is the most real (perhaps the only real) thing in the Universe, but also totally impossible to be observed by others (non-phenomenal). In my view this strongly limits our knowledge on sentience, with important consequences for animalism, that I will...

Published on April 29, 2024 4:13 PM GMTAdversarial Examples: A Problem The apparent successes of the deep learning revolution conceal a dark underbelly. It may seem that we now know how to get computers to (say) check whether a photo is of a bird, but this façade of seemingly good performance is belied by the existence of adversarial examples—specially prepared data that looks ordinary...

Published on April 29, 2024 10:53 AM GMTThe 9th AI Safety Camp (AISC9) just ended, and as usual, it was a success! Follow this link to find project summaries, links to their outputs, recordings to the end of camp presentations and contact info to all our teams in case you want to engage more.AISC9 both had the largest number of participants (159) and the smallest...

Published on April 29, 2024 10:10 AM GMTCross-posted on the EA Forum. This article is part of a series of ~10 posts comprising a 2024 State of the AI Regulatory Landscape Review, conducted by the Governance Recommendations Research Program at Convergence Analysis. Each post will cover a specific domain of AI governance, such as incident reporting, safety evals, model registries, and more. We’ll provide an overview of existing regulations, focusing...

Published on April 29, 2024 2:30 AM GMTThis is a response to the post We Write Numbers Backward, in which lsusr argues that little-endian numerical notation is better than big-endian.[1] I believe this is wrong, and big-endian has a significant advantage not considered by lsusr.Lsusr describes reading the number "123" in little-endian, using the following algorithm:Read the first digit, multiply it by its order of...

Published on April 29, 2024 1:57 AM GMTDate: Saturday, May 4th, 2024Time: 1 pm – 3 pm PTAddress: Yerba Buena Gardens in San Francisco, just outside the Metreon food court, coordinates 37°47'04.4"N 122°24'11.1"W  Contact: 34251super@gmail.comCome join San Francisco’s First Saturday (or SFFS – easy to remember, right?) ACX meetup. Whether you're an avid reader, a first time reader, or just a curious soul, come meet! We will...

Published on April 29, 2024 12:48 AM GMTThis is a post on a novel cognitive architecture I have been thinking about for a while now, first as a conceptual playground to concretise some of my agent foundation ideas, and lately as an idea for a project that approaches the Alignment Problem directly by concretising a sort of AI-Seed approach for an inherently interpretable yet...

Published on April 29, 2024 12:22 AM GMTIn this post, I will provide some speculative reasoning about Simulators and Agents being entangled in certain ways.I have thought quite a bit about LLMs and the Simulator framing for them, and I am convinced that it is a good explanatory/predictive frame for behavior of current LLM (+multimodal) systems. It provides intuitive answers for why the difficulties around...

Published on April 29, 2024 12:18 AM GMTI find that an especially illustrative thought experiment regarding embodiment is to imagine a superintelligent Stone that can talk. Let’s say that this Stone can somehow perceive its environment but is, as you might expect, incapable of moving around. The Stone is not a very powerful optimiser of its environment as long as it is just lying...

Published on April 29, 2024 12:16 AM GMTThis is an idea I am toying around with for understanding resolutionally adjusted causal modeling - this is just a bunch of intuitions and pointing towards a somewhat clear framing of a fundamental thing. I am sure there are already plenty of accounts for how to approach this kind of task, but I like to figure stuff...

Published on April 27, 2024 6:34 PM GMTI just spent a couple of hours trying to understand the UN’s role in the governance of AI. The most important effort seems to be the Global Digital Compact (GDC). An initiative for member states to “outline shared principles for an open, free and secure digital future for all". The GDC has been developed by member states...

Published on April 27, 2024 4:36 PM GMTAbstract: First [1)], a suggested general method of determining, for AI operating under the human feedback reinforcement learning (HFRL) model, whether the AI is “thinking”; an elucidation of latent knowledge that is separate from a recapitulation of its training data. With independent concepts or cognitions, then, an early observation that AI or AGI may have a self-concept. Second...

Published on April 27, 2024 4:04 PM GMTCharbel-Raphaël Segerie and Épiphanie Gédéon contributed equally to this post. Many thanks to Davidad, Gabriel Alfour, Jérémy Andréoletti, Lucie Philippon, Vladimir Ivanov, Alexandre Variengien, Angélina Gentaz, Léo Dana and Diego Dorn for useful feedback.TLDR: We present a new method for a safer-by design AI development. We think using plainly coded AIs may be feasible in the near future...

Published on April 27, 2024 1:32 PM GMTThis is exploratory investigation of a new-ish hypothesis, it is not intended to be a comprehensive review of the field or even a a full investigation of the hypothesis.I've always been skeptical of the seed-oil theory of obesity. Perhaps this is bad rationality on my part, but I've tended to retreat to the sniff test on issues...

Published on April 27, 2024 1:22 PM GMTOne consideration that is pretty important for AI safety is understanding the extent to which a model's outputs are aligned with its chain of thought.This paper (Twitter thread linked) provides some relevant evidence. It demonstrates that it is possible for a model to achieve performance comparable to chain-of-thought with dots replacing the chain of thought under some...

Published on April 27, 2024 12:14 PM GMTVernor Vinge is a legendary and recently deceased sci-fi author. I’ve just finished listening to the first two books in the Zone of Thought trilogy. Both books are entertaining and culturally influential. The audio versions are high-quality.A Deepness in the Sky is about two spacefaring human civilizations with clashing societal structures converging on a mysterious solar system...

Published on April 27, 2024 11:13 AM GMTThis work was produced as part of Neel Nanda's stream in the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort, with co-supervision from Wes Gurnee.This post is a preview for our upcoming paper, which will provide more detail into our current understanding of refusal.We thank Nina Rimsky and Daniel Paleka for the helpful conversations and review.Executive...

Published on April 27, 2024 10:14 AM GMTNew tricks and tools can make cognitively demanding tasks less painful.As a college student two decades ago, Andrew Westbrook struggled to stay focused in class. He certainly had the capacity to concentrate; he could do it intensely when he really loved a topic. “When you’re totally engrossed in a good book, thinking can feel effortless, or even...

Published on April 27, 2024 8:02 AM GMTI don't know if this is a well known argument that has already been responded to. If it is, just delete the post.An implicit assumption most people make when discussion takeover risk, is that any misaligned agent will stave off enacting takeover plans with less than very high probability of success. If the Agent has an immediate...

Published on April 26, 2024 11:22 PM GMTTL;DR: In this post, I distinguish between two related concepts in neural network interpretability: polysemanticity and superposition. Neuron polysemanticity is the observed phenomena that many neurons seem to fire (have large, positive activations) on multiple unrelated concepts. Superposition is a specific explanation for neuron (or attention head) polysemanticity, where a neural network represents more sparse features than there...

Published on April 25, 2024 4:02 PM GMTPeople have been posting great essays so that they're "fed through the standard LessWrong algorithm." This essay is in the public domain in the UK but not the US.From a very early age, perhaps the age of five or six, I knew that when I grew up I should be a writer. Between the ages of about...

Published on April 25, 2024 2:51 PM GMTThis article is part of a series of ~10 posts comprising a 2024 State of the AI Regulatory Landscape Review, conducted by the Governance Recommendations Research Program at Convergence Analysis. Each post will cover a specific domain of AI governance (e.g. incident reporting, safety evals, model registries, etc.). We’ll provide an overview of existing regulations, focusing on the US, EU, and China...

Published on April 25, 2024 6:40 AM GMTIt seems to me worth trying to slow down AI development to steer successfully around the shoals of extinction and out to utopia. But I was thinking lately: even if I didn’t think there was any chance of extinction risk, it might still be worth prioritizing a lot of care over moving at maximal speed. Because there...

Published on April 25, 2024 5:43 AM GMTThe NIH has a page called Cancer Myths and Misconceptions that you come across if you end up looking into cancer for long enough, aimed at bio-illiterate patients and their families.Around half the things on that page are wrong at face value, and a solid percentage of those are contradicted by the pages and studies the NIH...

Published on April 25, 2024 2:18 AM GMT I refuse to join any club that would have me as a member. — Groucho Marx Alice and Carol are walking on the sidewalk in a large city, and end up together for a while. "Hi, I'm Alice! What's your name?" Carol thinks: If Alice is trying to meet people this way, that means she doesn't...

Published on April 24, 2024 11:50 PM GMTEpistemic status: party trick Why remove the prior One famed feature of Bayesian inference is that it involves prior probability distributions. Given an exhaustive collection of mutually exclusive ways the world could be (hereafter called ‘hypotheses’), one starts with a sense of how likely the world is to be described by each hypothesis, in the absence of...

Published on April 24, 2024 10:48 PM GMTNote: In @Nathan Young's words "It seems like great essays should go here and be fed through the standard LessWrong algorithm. There is possibly a copyright issue here, but we aren't making any money off it either." What follows is a full copy of the C. S. Lewis essay "The Inner Ring" the 1944 Memorial Lecture at King’s...

Published on April 24, 2024 9:21 PM GMTNote: It seems like great essays should go here and be fed through the standard LessWrong algorithm. There is possibly a copyright issue here, but we aren't making any money off it either. What follows is a full copy of "This is Water" by David Foster Wallace his 2005 commencement speech to the graduating class at Kenyon...

Published on April 24, 2024 8:04 PM GMTWarning: This post might be depressing to read for everyone except trans women. Gender identity and suicide is discussed. This is all highly speculative. I know near-zero about biology, chemistry, or physiology. I do not recommend anyone take hormones to try to increase their intelligence; mood & identity are more important.Why are trans women so intellectually successful?...

Published on April 24, 2024 5:50 PM GMTBefore we get started, this is your quarterly reminder that I have no medical credentials and my highest academic credential is a BA in a different part of biology (with a double major in computer science). In a world with a functional medical system no one would listen to me.  Tl;dr povidone iodine probably reduces viral load...