Jérémy Perret
Bio
Participation4
I write and talk about AI alignment, longtermism and EA more generally, in France. I'm active in French groups, trying to help grow the local EA community. I want to spend more time creating AI alignment content in French, lowering the language barrier.
How others can help me
If you've run AI alignment reading groups, please tell me the most intriguing questions you've encountered!
Posts 1
Comments17
Hi from the session! Thank you for running it.
- General idea: people have been asking for my takes on AI alignment topics, and much of what I'm replying has never been written anywhere. That ought to be fixed.
- Specific idea: I consider that the landscape of AI alignment outreach is shifting for a specific reason I'd like to expand on.
Writing this from the "Writing on the EA Forum" workshop at EAGxBerlin, wanted to post some... post ideas:
(a) A quick report on how the conference went, and the particular vibes I encountered, expanding on my "like an academic conference with the annoying prestige layers removed" idea.
(b) Content creation as lowering barriers to entry for a given field, especially language barriers. It seems like an obvious point but on I've never seen written anywhere?
(c) Something about the impact of sharing useful trivia and life advice. I have a recent example about hearing damage/loss prevention, what's the marginal value of these conversations?
(Thanks to Lizka for the nudge and excellent presentation)
Slightly humorous answer: it should be the very most pessimistic organization out there (I had MIRI in mind, but surely if we're picking the winner in advance we can craft an organization that goes even further on that scale).
My point is the same as jimrandomh: if there's an arms race that actually goes all the way up to AGI, safety measures are going to get in the way of speed, corners will be cut, and disaster will follow.
This assumes, of course, that any unaligned AGI system will be the cause of non-recoverable catastrophe, independently from the good intentions of their designers.
If this assumption proves wrong, then the winner of that race still holds the most powerful and versatile technological artifact ever designed; the kind of organization to wield that kind of influence should be... careful.
I'm not sure which governance design best achieves the carefulness that is needed in that case.
I cannot speak for all EA folks; here's a line of reasoning I'm patching together from the "AGI-never is unrealistic" crowd.
Most AI research isn't explicitly geared towards AGI; while there are a few groups with that stated goal (for instance, DeepMind), most of the AI community wants to solve the next least difficult problem in a thousand subdomains, not the more general AGI problem.
So while peak-performance progress may be driven by the few groups pushing for general capability, for the bulk of the field "AGI development" is just not what they do. Which means, if all the current AGI groups stop working on it tomorrow, "regular" AI research still pushes forward.
One scenario for "everyone avoids generality very hard while still solving as many problems as possible" is the Comprehensive AI Services framework. That is one pathway, not without safety concerns.
However, as Richard Ngo argues, "Open-ended agentlike AI seems like the most likely candidate for the first strongly superhuman AGI system."
To sum up:
- regular, not-aiming-for-AGI AI research will very likely attempt to cover as many tasks as possible, as most of the field has done, and will eventually on aggregate cover a wide enough range of capability that alignment issues kick in;
- more general agents are still likely to appear before we get there, with nothing impeding progress (for instance, while DeepMind has a safety team aware of AGI concerns, this doesn't prevent them from advancing general capability further).
A separate line of reasoning argues that no one will ever admit (in time) we're close enough to AGI that we should stop for safety reasons; so that everyone can claim "we're not working on AGI, just regular capabilities" until it's too late.
In that scenario, stopping AGI research amounts to stopping/slowing down AI research at large, which is also a thing being discussed!
Hi! Thanks for this post. What you are describing matches my understanding of Prosaic AGI, where no significant technical breakthrough is needed to get to safety-relevant capabilities.
Discussion of the implications of scaling large language models is a thing, and your input would be very welcome!
On the title of your post: the hard left turn term is left undefined, I assume that's a reference to Soares's sharp left turn.
This is an edited version of my EAG London 2025 report. Collection of musings and personal journal notes. Doesn't seem worth a proper post.
Previous EA conference I'd participated in was EAGxUtrecht 2024. I had scheduled a whole bunch of 1:1 discussions, about a dozen on Swapcard, plus the informal chats that just happened. I kept zero connections with all those people. Same, mostly, with the people from EAG London 2023. My goal was to do better. I went in with an objective, to find people I would have a powerful excuse to contact again. Either because of unanswered questions, or to work with them, or to check on their progress without the fear of bothering them.
The conference happened at the Intercontinental Hotel near the O2 in Greenwich Peninsula, London. I'm told it was also where it was held last year. There were some presentation rooms, some small rooms for speed meetups, quiet working and workshops, and a gigantic main ballroom where meals and most 1:1s happened. All of it with a view on the Thames, and a bunch of tall buildings on the other side. Excellent, all-vegan (of course) food.
There were a large number of French people attending. I barely spoke to them overall outside of life updates, except one I really wanted to network with, which I successfully did. I gathered an important amount of information on CeSIA, pride and glory of the small French AI safety scene. Despite being one message away from a few people working there, I almost never got in touch, but they were at EAG London, so I used it as a backstop for finally doing it. It worked.
In the past months I also endlessly put off applying to Successif, who I got pointed towards to make my career more impactful given my interests. They had a booth at EAG London as well, which I visited to give me extra in-person nudges to finally apply. I haven't done it yet. I told a friend about it, who immediately set a reminder to check on me to ensure I actually do it. He knows me well. I am grateful.
I'd reached out to a social media creator focusing on AI alignment. Her posts being on TikTok and Instagram, I wasn't aware of them. Since I wasn't alone in contacting her, a content creator meetup was organized on Sunday at lunch. I made a few extra contacts, one being a potential work/project partner. In the end I didn't have the opportunity to talk with the original meetup organizer (wrong end of the table), but I know how to reach her, so... the challenge is now to follow up.
Otherwise I participated in a few speed networking sessions. Very quick chats, some tuning into 1:1 discussions later. One student could thus enjoy my Twenty Minute Tour of the AI Alignment Technical Research Landscape. As is tradition, I immediately regretted not recording my own rant. I always tell myself that, since I just output it from memory given the prompt "what can I work on in AI safety" plus some coffee and sugar to launch me, I don't consider that information truly lost. Still, it could be of use to some people, with the caveat that I'm just a guy with takes, not an authority on the subject.
Similarly, the one take I told the most was "the AI safety comms landscape is changing because the safety cases we can extract from the latest system cards now actually speak to the general public, not just technical-minded people". I think this has implications, and is not coincidentally related to AI tools bleeding into the mainstream. The more actual use cases people are interested in, the more warped usage they will discover and have reactions about. Because it's the same underlying capabilities, the same base models, and current steering techniques don't work well enough.
I told this to a few people, and it should also be written, expanded on, somewhere. I might be wrong on this, but if I don't get the idea out there, I'll only update very slowly, through my own further reading, way later, past any window of relevant action, and me having a take will have been mostly pointless. I remember sharing much more of my takes back when I was hosting meetups on the topic, and facilitating AGI Safety Fundamentals sessions. I don't know if and how I influenced people, but it was certainly better than keeping ideas to myself.
My biggest source of grumpiness is not to have interacted more with the aforementioned French EA people. For various logistics reasons I didn't participate to the satellite event where I could have had lots of chats. I did other, also valuable things instead, so it's not a proper regret, but I'm still grumpy. Afterwards I thought, well, if I'm so annoyed, why don't I go talk to them in France, where I live, instead of whining about not seeing them in London? That's one extra follow-up challenge; I know I have trouble reaching out to people. This event provided a host of strong nudges, which was the point.
I also sought and met a few people I admire, to thank them for their work. It was important.
Overall, event was tentatively worth the time and financial investment. I will only be sure in hindsight, if (again!) I do follow up on all this networking. Though, I don't see clear ways it could have gone better. That's a win.