AI Beliefs Out In The "Normal" World
Door-to-door surveying
In the last 48 hours after I released1 my public database of AI Safety papers2 I got over 1000 unique views to my site. That was really cool. Kind of astonishing how distracting “number go up” is. I was probably checking my stats at least once an hour, I have basically never been into social media (etc) so I haven’t built up my defense mechanisms to this distraction. I ran an extremely successful dinner party series for 4 years, and in 48 hours more people have looked at my website than the sum total of all people that came through my home. The scale of the internet is kind of astonishing.
So, the logical next thing to do is something that continues to take advantage of the internet! Nope. The weather in NYC is about to be quite nice… So I am going door-to-door surveying a bunch of people in the 3 days of nice weather we have. Most people I talk to or interact with regarding AI are in a pretty tight bubble: very educated, very tech literate, well paid, some might even say “technocratic elites”. There have been surveys done about AI existential risk3, but at the end of the day there’s something that hits very different about seeing stats on a page vs. personally going out and actually talking to people and hearing their unfiltered opinions.
My biggest goal right now is to actually figure out of all the various work options at my disposal which one I should start moving towards, so to some extent this feels like I’m slightly going on a not worth it tangent. It’s basically been half a day of ideating, half a day of coding, and then likely ~36 hours of executing4. I do think to some extent it will feel good/helpful to have a way better internalized model of what the average person thinks about this, but I am still using 1% of my year on this.
Really, the problem5 is that this is a super cool mini problem. There are a bunch of subproblems and every single one of them is interesting fun and novel!
Survey design! How can I squeeze the most survey signal out of the shortest possible survey? I want to be able to truthfully tell people hey this will only take 1 minute to maximize uptake. How do I order questions to minimize bias? How can I hide my personal bias? How can the wording of each question be tweaked for maximum understandability ?
How can I find a collection of neighborhoods and in what ratios to get the most representative sample? But also, maybe I want to eliminate some neighborhoods as options: some neighborhoods only have big doorman buildings which are way harder to canvass than a house with a front door, some neighborhoods are a total pain for me to get to, etc. I built a site that lets anyone select which neighborhood options to include, how much to weight various types of diversity, how many total neighborhoods to go to and then it calculates the most representative basket of neighborhoods6!
How can I have the smoothest in the field data submission flow? Build a bespoke app that allows for super simple one click answers to each question. How about demographics? You don’t want to have to ask the respondent most of these since it’s a waste of time and likely puts them on edge7. But, it’s really easy to collect the phone GPS location each time a response is submitted. Using the GPS location I can then make a strong Bayesian update on the expected demographic data based on the neighborhood I’m in. Gender and ethnicity8 you can get the surveyor to eyeball. The only (optional) demographic question I ask in the entire survey is age. What if I lose connection? Turns out I can set up the survey page to be easily downloadable as an app that shows up on the homescreen, with local memory and syncs submissions back to the DB once connection is re-established!
How to maximize the percent of people who say yes to my survey pitch. This is the one I haven’t tried yet, but there’s tons of interesting psych dynamics at play that I will get to try out in person iteratively. Is it better if I make it personal “Hi, I’m Cormac — I’m doing an independent survey on what people actually think about AI” vs. trying to get a micro-yes first “Hi, I’m Cormac — quick question: do you have any opinion on AI?”. Body language, cadence, tone, how I respond to various things they say, there’s so many aspects! I bet the best execution depends on reading the person’s vibe and picking the best strategy for the given person. This is genuinely a hard problem that I am thinking about in advance and will involve lots of iteration throughout the process. I have never canvassed before, I’ve heard it can be absolutely brutal - I’m curious how I will react to this thing that I hear can be very draining9.
The other hope for this is its use as a great instrumental goal10. My planned next project is to start really mapping out what all the various AI Safety orgs are actually doing and why they are doing it11. This feels good for a variety of reasons. It helps me personally in deciding what I want to do by seeing what the options are. It naturally leads to me talking to (emailing) a variety of people and (hopefully) getting their personal takes on what they think the most important theories of change are. Also, nobody has really done this, the best existing resource is probably the AI Safety field map, which is extensive (and I will certainly be using for discovery) but doesn’t really get into how specifically a given org is helping us not end up catastrophically fucked by AI. All of this to say, I think this is a great proof of work on my interest in extensively cataloguing various axes of importance in the world of AI safety. From cataloguing the body of work that is technical work in AI safety, to personally cataloguing and internalizing public opinions, to then really understanding the ecosystem12. The theory is people are more excited to talk to someone that has proof of work getting shit done in this domain and is onside13 so to speak.
How am I feeling about all of this? Pretty excited, slightly worried that I am letting myself get nerd sniped by a really interesting subproblem but also this feels generally worthwhile and working on things that are interesting and energizing is good! I can do more total work if I like it, my life is better, etc. The real question is after I have done reps and know what the experience is like, do I post this in the various places where people who care about AI hang out? I have built out the infra, the app is smooth, it’s easy to make someone a profile and let them go canvass. I can easily sort out results from a given user if there’s worry of a given canvasser giving bad data. It seems like there might be people who would enjoy going out and asking people what they think of AI. The worst that could happen is nobody is interested. Idk, I can’t really see any reasons not to at least give people the option, given I have built all the infra already. And if I’m really truly worried that I can’t trust other canvassers14 to be biased in their delivery I can always toss all other data.15
Released = Posted on LessWrong
And now also blog posts as well!
Although, all existing survey work has been online/over text. As far as Claude and I can tell, in person surveying of existential risk is novel work - pretty exciting to be doing something no one has done before.
ofc i can always stop early if it seems to be not getting me anything
If you can even count working on something that is maybe a marginally inefficient use of my time as a problem
Not guaranteed optimal since that’s hugely computationally expensive. It uses more of a greedy algorithm.
Not to mention lots of these have potential bias. I would be very surprised if asking someone how they politically identify in NYC didn’t skew D’s more likely to identify.
“white”, “black”, and “other” is quite sufficient because neighborhood by neighborhood “other” is quite defined. As with most big cities there’s tons of natural clumping by ethnicity etc.
physically, emotionally, intellectually
A goal that helps me accomplish a further goal. For many people getting a CS degree was an instrumental goal towards getting a job. Taking the train to my friends party is an instrumental goal towards attending their party.
What is their theory of change, why specifically do they think that what they are doing is helping lower the odds of catastrophic outcomes due to AI.
Not to mention the technical red team/model alteration work I’ve done. (And I’m currently ideating on the next technical project to work on since I think it’s quite helpful to remain grounded in the world of technical work while surveying the options)
There’s a lot of general distrust of anything reporter shaped in these general communities after super consistently getting approximately done dirty over and over by traditional reporters.
But that level of mistrust feels pretty misplaced, I’m not some lone paragon of virtue.
In my standard Claude blog post feedback it has lots of thoughts on potential holes in my survey methodology
Let’s go corm!