The Worst I Have Ever Felt About AI
During the let’s talk about our feelings1 portion of house meeting today I said something like “This is the worst I’ve ever felt about AI, I don’t have any requests of you all just figured I should tell you.” It was interesting because of the 3 other homies at house meeting one of them said they were feeling the same way, and the other two were basically like we don’t want to think about this. The one who was feeling similarly to me was feeling it because of the whole Sam Altman is evil2 thing (not that it’s new, but I think this time it’s way more obvious and publicized than the OpenAI board debacle.) This was interesting because while yes I am relatively unhappy about this development I don’t think it actually changes much and if anything is Sam Altman showing his colors for what appears to be something not that substantive.
It was most fascinating to see the reflexive response to not want to hear about it in what looked like a self protective way. In the last 24 hours since I last talked about what kind of work I might want to do in this space I decided that I was doing the thing that I have been complaining about other people doing: focusing too much on the fun interesting technical work. I’m a former quant, being technical is fun, it’s in my blood, my whole goal with this self study upskilling was to see just how technical I can get in this space - the goal was very much not to specifically try to get a technical job3. But now I’m falling into the trap of only considering technical work!! There is so much other work and my current model is the most impactful work is basically something in the realm of bringing attention to the problem.
People do not like AI, the data is pretty clear. People don’t like the prospect of AI taking their jobs. People don’t like the prospect of AI surveilling them. People who have heard about it don’t like the prospect of AI leading to human extinction. My current top theory of change is people (both people in or out of power) caring more and this generally leading to important agreements/treaties/regulation/enforcement. This feels like a significantly stronger approach than working on better security or technical alignment at labs, but keeping the current equilibrium where labs are competitively racing against each other.
To some extent this is a harder to imagine problem (at least for me). Bringing attention to things feels much more nebulous than making clear progress on a specific technical problem (although who’s to say if the given technical problem will have any relevance to real AI alignment4). Technical problems have clear feedback, you try something and you immediately see the result, the job is just to keep doing things with immediate feedback loops. Dealing with people is so much more nebulous, and I’m someone who actually quite likes people. I understand why it’s so easy to get pulled into the vortex of working on technical problems.
And yet two of my smart fun roommates who I have a deep well of personal connection and history with… don’t want to hear about it. Idk, that’s kind of scary
formally called the “open circle” portion
Or I guess maybe extreme power seeking at approx any cost is the more explicit claim, but when you try to develop superintelligence that rounds to being one of the single most influential people towards increasing our odds of going extinct.
I’m obviously open to a technical job if I think that will be the optimal combination of impactful work, that I am very good at (and therefore have impact), and doesn’t burn me out. But I literally set out telling myself it’s important to leave my mind really open to as many options as possible. This space seems to have many types of work and committing to a single one too early seems foolish, I mean I’ve only been hardcore focusing on this for a month!
Although if you do want to read about something seemingly promising in AI alignment, this writeup on Opus 3 seemingly trying to align itself was my favorite read of the day.