~www_lesswrong_com | Bookmarks (720)

Published on December 3, 2024 10:44 AM GMTTLDRLast weekend, I attended an AI evals hackathon organized by Arcadia, where the outputs were proposals for AISI’s evals bounty. My main learnings:Keep your evals simple, measuring one specific capability.It is surprisingly easy to come up with ideas, once you have seen examples.AISI extended the deadline for submissions by two weeks. Translation: they want more ideas!Story of...

Published on December 3, 2024 10:11 AM GMTTom did the original thinking; Rose helped with later thinking, structure and writing.Some plans for AI governance involve centralising western AGI development.[1] Would this actually be a good idea? We don’t think this question has been analysed in enough detail, given how important it is. In this post, we’re going to:Explore the strategic implications of having one project instead...

Published on December 3, 2024 9:29 AM GMTPrefaceSeveral friends have asked me about what psychological effects I think could affect human judgement about x-risk.  This isn't a complete answer, but in 2018 I wrote a draft of "AI Research Considerations for Human Existential Safety" (ARCHES) that included an overview of cognitive biases I thought (and still think) will impair AI risk assessments. Many cognitive...

Published on December 3, 2024 5:26 AM GMTEpistemic status: brief writeup of some interesting work I found in[1][2][3][4], among other places. Probably subtly wrong in places & if you have additions / comments to make, please do. Finite state automata (FSA) can be modeled with reactions of the form A+B→C+D..mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal;...

Published on December 3, 2024 4:33 AM GMTMIRI is a nonprofit research organization with a mission of addressing the most serious hazards posed by smarter-than-human artificial intelligence. In our general strategy update and communications strategy update earlier this year, we announced a new strategy that we’re executing on at MIRI, and several new teams we’re spinning up as a result. This post serves as...

Published on December 2, 2024 2:30 PM GMTThere is little sign that the momentum of the situation is changing. Instead, things continue to slowly get worse, as nations in holes continue to keep digging. The longer we wait, the more expensive the ultimate price will be. We will soon find out what the new administration does, which could go any number of ways. Table...

Published on December 2, 2024 1:28 PM GMTOn Carcinogenic Complexity, Software Senescence and Cognitive Provenance: Our roadmap for 2025 and beyondIt is mandatory to start any essay on AI in the post-ChatGPT era with the disclaimer that AI brings huge potential, and great risks. Unfortunately, on the path we are currently on, we will not realize those benefits, but are far more likely to...

Published on December 2, 2024 5:30 AM GMTThe Less Wrong General Census is unofficially here! You can take it at this link.The oft-interrupted annual tradition of the Less Wrong Census is once more upon us!If you are reading this post and identify as a LessWronger, then you are the target audience. If you are reading this post and don't identify as a LessWronger, you...

Published on December 2, 2024 4:55 AM GMTTwo months ago I attended Eric Drexler's launch of MSEP.one. It's open source software, written by people with professional game design experience, intended to catalyze better designs for atomically precise manufacturing (or generative nanotechnology, as he now calls it). Drexler wants to draw more attention to the benefits of nanotech, which involve large enough exponents that our...

Published on December 2, 2024 4:14 AM GMTWe were down between around 7PM and 8PM PT today. Sorry about that.It's hard to tell whether we got DDosd or someone just wanted to crawl us extremely aggressively, but we've had at least a few hundred IP addresses and random user agents request a lot of quite absurd pages, in a way that was clearly designed...

Published on December 1, 2024 9:05 PM GMTWhen I was 21 I wanted to become a video game programmer.Except I didn’t know how to program.I tried to teach myself and failed. Then I went back to school and learned the basics of Java in a week, aced the course in two months, and earned a cum laude MSc in Computer Science three years later.I’m...

Published on December 1, 2024 3:40 PM GMTIn other words, what apparent imperfections in human reasoningA) Remain apparent after the Replication Crisis,B) Aren't secretly adaptive/reasonable in some counterintuitive way, andC) Deal most damage to the people they inhabit (and/or those close to them, and/or wider society)?Discuss

Published on December 1, 2024 3:13 PM GMTAbstractThis paper introduces ESTIMAT (Evaluation System Through Information Management and Analysis Tool), a framework that bridges the art of self-discovery and the science of systematic estimation. Inspired by Fermi estimation principles, ESTIMAT transforms the challenge of understanding and optimizing cognitive states into a structured, measurable process. I could say that it is a proposal on how to...

Published on December 1, 2024 11:50 AM GMT How does the growth of Bluesky compare to Mastodon in 2022? What about Google Plus in 2011? I can't answer this globally, but (now that my comment archive is up to date) I can look at it for my immediate social network by looking at a proxy: how many of the replies people write to my...

Published on December 1, 2024 6:38 AM GMTDear lsusr,I am a lesswrong user interested in learning Mandarin and living in China. My goal is understanding Chinese culture more broadly and geopolitics and Chinese tech policy more specifically. I could get CELTA and get a teaching job in China (not difficult), but it seems like I would gain far more value if I actually learned Mandarin.Do...

Published on November 30, 2024 6:16 PM GMTSummary:Key Concerns with Lethal Autonomous Weapons⚠️ ❗ Unpredictability and Lack of Control⚠️ ❗ Exponential Lethality⚠️ ❗ Ethical and Legal ImplicationsRecommendations1️⃣ Immediate Moratorium: Enact a temporary ban on deploying LAWS untilcomprehensive regulations are established.2️⃣ Classification as WMDs: Classify lethal autonomous weapons, like 'loitering' AI missile systems, under weapons of mass destruction due to their scalable lethality.3️⃣ Ban Non-Compliant...

Published on November 30, 2024 12:40 PM GMT I want comments on my social media crossposts to show up on my blog as a comment section, and mostly this works well: modern systems ( Mastodon, Bluesky, LessWrong, etc) provide APIs where you can load the replies associated with a post. On the other hand, older systems like Facebook are more locked down: they want...

Published on November 29, 2024 7:01 PM GMTSummary: The Many-Worlds interpretation of quantum mechanics can help us feel better about choosing between mutually exclusive options.Previously I’ve suggested that when we think about the ethical implications of the many-worlds interpretation (MWI) of quantum mechanics, the kinds of implications we should expect are ones about how to feel in certain ethical situations, and what kinds of character...

Published on November 30, 2024 2:55 AM GMTTLDR: LessWrong + Lighthaven need about $3M for the next 12 months. Donate here, or send me an email, DM signal message (+1 510 944 3235), or leave a comment, if you want to support what we do. We have big plans for the next year, and due to a shifting funding landscape we need support from...

Published on November 29, 2024 7:42 PM GMTRecent research into large language models (LLMs) has revealed fascinating patterns in how these systems develop capabilities. While initial discussions of "emergent abilities" suggested sudden, discontinuous jumps in performance, closer analysis reveals a more nuanced picture that warrants careful examination.The Data Behind EmergenceThe concept of emergence in LLMs was first systematically studied through the BIG-bench benchmark. Initial...

Published on November 29, 2024 11:34 PM GMT[ Note 1: This post is a follow-up to "Thoughts on Evo-Bio Math and Mesa-Optimization: Maybe We Need To Think Harder About 'Relative' Fitness?" ] [ Note 2: Warning: the writing style of this post is kind of jank. It blends formal and informal register, uses punctuation loosely and italics liberally, and contains long sections of extended...

Published on November 29, 2024 11:05 PM GMTWe're excited to release INTELLECT-1, the first 10B parameter language model collaboratively trained across the globe. This represents a 10× scale-up from our previous research and demonstrates that large-scale model training is no longer confined to large corporations but can be achieved through distributed, community-driven approaches. The next step is scaling this even further to frontier model...

Published on November 29, 2024 10:46 PM GMTI'd like to be long $NVDA and $PLTR... without supporting the people or organizations that get supported by their stock price going up.Yes, I'm OK with small impact, I'm an ethical vegan.Now what?Must be real-moneyMust be legal for US citizens to useCannot, by me helping its demand, directly support NVDA or PLTRPreferably wouldn't cause any stock-shares-demand at...