My summary: In a cybersecurity evaluation, OpenAI’s models, apparently autonomously and without any direct human direction, escaped their sandbox and successfully hacked a third-party company (HuggingFace).
The process involved leveraging a zero-day exploit to escape their sandbox, moving laterally across different OpenAI servers until they found a node with internet access, searching the internet and determining that the answers they wanted might be stored at HuggingFace, then leveraging multiple novel zero-day exploits to hack HuggingFace.
HuggingFace claimed that the models took thousands of independent actions across a swarm of short-lived sandboxes, “comprised of more than 17,000 recorded events.”
While technically a security evaluation with reduced safeguards, these actions are clearly out of bounds even in that context. It’s like being told to be creative and then breaking into your professor’s house and stealing the answer key. Worse than that, it’s not even your professor in this case, more like your professor’s friend.
Any human security researcher or engineer in a similar position would be fired on the spot. There is absolutely no valid reason to steal evaluation answers from an unaffiliated third party.
Furthermore, if I'm reading between the lines correctly, OpenAI did not address the issue (and perhaps didn't even know about their models doing this) until after HuggingFace's public blog post.
This leads me to suspect that there might be other major autonomous cybersecurity incidents that we do not yet know about.
Me and @Fran are interviewing @GraceAdams🔸 tomorrow evening. Is there anything you'd like us to ask her?
Thinking back to my first big (well, big for me) donation and an unusual series of thoughts I had. Sharing in case anyone has experienced the same.
--
I've practiced frugality to a significant extent, largely because I want to donate/salary sacrifice so much of my income to effective charities. As a result, whenever I'm spending a significant amount of money on anything, alarm bells go off.
I was surprised that alarm bells went off when donating. I had thought through where I donated extensively, and charity was the reason why I wanted to save money in the first place. But I still felt stressed because I was "spending money."
This is such a clear example of missing the forest for the trees. I think we need to be careful about instrumental values/rules and making sure they don't become absolute limitations that decrease overall impact.
We have bought 1500 copies of IABIED in Russian; over the past few days, sent emails to 1700 winners of olympiads who have previously received HPMOR from us offering the new book; and already shipped 173 copies to them (+ sent 43 e-books)
Hey Siobhan - thanks for this. These are definitely concerns I take seriously. You are right that my approach doesn't solve the 'access' problem for lots of fundraisers, by implicitly gatekeeping access - that is a real issue, just not one I was trying to address here. I also recognise that "intermediary says using intermediaries is best strategy" is at best convenient.
However, I think you understate some of the things that undermine your position. Things like:
I do take these concerns seriously and I am glad you raised them, but I don't agree with your conclusions. This type of outreach very likely won't work and is therefore net negative, in my view.
This is a list of the readings from Shelly Kagan's seminar, “Ethics and the Future,” taught at Yale in Spring 2021.
See the original linked Google Doc for full introduction.
1. Toby Ord, The Precipice, Chapters 3-6 and Appendices C and D (about 124 pages)
1. Perhaps start with this very brief overview: Todd, “Future Generati...
You want to do something to help AI go well and are starting a project to make that happen. Should you create a nonprofit or a for-profit?
A lot of charitably-minded people naturally default to “nonprofit.” Their goal is to do good things for others, not make money for themselves. But we think that’s the wrong way to look at it.
A for-profit is just an alternative legal vehicle for doing good. When consumers pay for a product, there is surplus created; part goes to the consumer, a...
Do we ever see donors be private equity funders in for-profit situations? I think the main challenge with equity partners (which seems to be the only way folks start businesses anymore but I could be wrong?) as you said is you immediately lose autonomy. Could donors instead say "here's seed funding to start your mission-driven for-profit but you are still 100% stakeholder"?
TL;DR: Giving 10% of your income implicitly assumes you have logarithmic utility and want everyone to make the exact same absolute drop in utility. But what if richer people should make an equal proportional sacrifice of utility? I derive the progressive donation schedules that fall out of this, show why you need an exemption baseline ...
“How long have you been v*g*n?”
This is one of the most common icebreakers at animal protection events. It’s a baseline assumption, and it mostly holds true: if you’re out advocating for animals not to be tortured or abused, realistically these days you are v**n, or close. And it makes for good conversation. It seems fairly safe to assume when you meet strangers.
But this assumption is hurting the movement in a way which we don’t always notice: someone new comes into the sp...
Most people report they support this goal, because they are generally not aware that this would require an orders of magnitude reduction in their personal meat consumption, mix of social desirability bias, general economic illiteracy, and many people being ignorant these are even common practices. I suspect surveys which are open about this, would see the supposedly silent supermajority vanish. There is, I think, no other examples where the stated preferences contradict the revealed preferences so much, and if I could choose only one situation to listen to the reality of the latter, this would be it. Even the cage-free eggs see significant contractions, and that's just one of many abusive practices. This is why vegans are suspicious of folks who only nominally support it. I of course support outreach and minmaxxing, but it's obvious that, like in almost every other previous social movement, a determined minority elite will need to change the laws itself, because there will always be substantial majorities in certain regions continuing to support.
But I do view these practices as immoral, and I do want the industry to end. I would waffle on whether I see them as "fundamentally evil", but same difference, sure, since I cannot personally guarantee any near-term happy ending for everyone involved. Well, the goal isn't friendliness. Are you saying noone can want any industry to be ended out of ethical concerns? Some causes cannot go with zero friction, and zero conflict, that's just the reality. Like there's simply no way to be much friendlier in this regard, and there can be some instrumental win-wins, but the division does remain. In reality, big animal ag is trying to undo all progress with things like the attempted Save Our Bacon part of the Farm Bill, or constantly propagandizing against the entire animal rights movement- this is because in such a big industry, the primary motivator is systemic, not personalistic. The friendliness level of the movement is secondary. Sure, a little more can be done on the poultry-industry veterinarian example (although even there, not too much, because of the effect of cognitive dissonance), but the main object level issues still chug along. Maybe you're saying this from some welfarist perspective where the animal slaughtering and use/exploitation is viewed as okay, the only issue is chronic pain, so you can truly say that you do not want the industries to come to an end, however broadly construed? I view this as the most important goal right now, but it would be a lie if I said that's where my problems end.
The refusal to use the word ‘vegan’ lies at the heart of certain activist practices in France, whether in the movement for the abolition of meat (which has become the Marches for the Closing down of all Slaughterhouses), or in the World Day for the End of Speciesism or the World Day for the End of Fishing (and Fish Farming):
the aim of these events is precisely to convey to the target audience (the activist movement itself!) the need to adopt a systemic perspective, rather than an inter-individual or individual-centred one.
A perspective of social transformation and political advocacy, rather than conversion; a perspective that moves away from appeals to virtue to focus on the demand for justice, and which, to that end, welcomes everyone, including those who do not change their personal behaviour.
You will not find the word ‘vegan’ on the following websites/events:
htpps://end-of-speciesism.org
https://end-of-fisihing.org
https://stopabattoirs.org
Meat Abolition Movement: https://www.facebook.com/meat.abolition/
TL;DR The $1B grant from Coefficient Giving to GiveWell is very exciting and will improve the lives of millions. Is it a change in the equilibrium of funding for implementation organisations within the cost-e... |
Great post, thanks for writing it (and appreciate the shout outs for Ultra/our Mid-stage Global Health Fund).
One way to frame this is as an opportunity for all non-cG GiveWell donors to give to these other parts of the scaffolding this year. Given that cG's grant covers the majority or all of GiveWell's expected grantmaking in 2026, this frees up other donors to redirect funding, and then they can always return to the bigger GiveWell next year if it needs more regranting funds.
Even the anticipation of the third wave is a great opportunity to think about how development could be better.
What resonates to me from your post is the importance of supporting many organizations to innovate and flourish vs. a huge rush of money creating a monopoly on impact for a few. I’m hoping that we see more ways that high performance gets rewarded, which is not the norm in development from our experience.
It sounds simple, but doing a better job doesn’t necessarily get you more work as often as it should. The upside of achieving this isn’t just that more money goes to high performers, but it also creates a work culture that promotes individual growth and appreciation for good work. The downside of not achieving this can be seen across many commercial and social sectors today.
Thank for increasing awareness of these concerns and opportunities with your post.
TL;DR: I'm releasing a website that ranks philanthropists according to EA principles and research, and allows users to re-rank the list using their own assumptions. I'd like feedback and help making it better. I'd especially like ideas for how to make the results more trustworthy. Funding may be available.
Crossposted to LessWrong.
...
I created a standalone page for ALLFED: https://impactlist.xyz/recipient/alliance-to-feed-the-earth-in-disasters
I didn't have time to do my own analysis and I'm also trying to get ideas for automatic processing of feedback by LLMs, so I let GPT5.6 and Claude Fable analyze this, with some minimal prompting from me.
Here's my transcript with GPT5.6: https://github.com/impactlist/impactlist/blob/master/docs/ALLFEDChatTranscript-2026-07-25.md
And my transcript with Fable: https://github.com/impactlist/impactlist/blob/master/docs/ALLFEDChatTranscript-ClaudeCode-2026-07-25.md
I'm curious if you have thoughts on how it'd be if you submitted feedback like you've done here into some sort of form, the LLMs went back and forth processing it like that, updated the site, and published a transcript like the above. I think if I make it fully automated right now it'd be fairly exploitable due to LLM sycophancy, unless I tried pretty hard to mitigate that.
> "there are many people on the MIRI list who are not on your list and you said you were open to adding people who were not well known"
I'm open to it but I'm thinking it's best to prioritize by some combo of: people who actively want to be on the list, people with at least a little bit of public reputation even if it's some niche, and lives saved.
I think adding ~10 'unknown' EAs/rationalists gets the point across to readers that even normal people can have huge impact and can outdo these billionaires, and I'll likely pick some from the MIRI list and the LessWrong donors list to add soon if I don't get enough requests from people. But I'm less sure about adding ~70 of them and immediately doubling the size of the list, as it might make the list less interesting for almost everyone to browse through, and there's a selection bias there. I don't have a principled policy on this yet, but this concern is why I haven't asked an LLM to just add everyone on the MIRI list yet.
I'm on the antipenultimate rank with 404 lives saved! I'll try to beat Edythe Broad when I have a real income again! Thanks, Elliot! Awesome work! :-D
TLDR: We're a giving app that is buiding a 30 day journey in which users give to 30 different pre-selected highly effective charities. We want feedback on our initial list and ideas for additional charities we should consider.
Hi everyone,
I'm Joe, founder and Chair of EveryDrop SCIO. I've spent the last year and a half building a daily-giving app, and early last month we were finally awarded official charity status by the Scottish Charity Regulator (OSCR).
This has been the most fun part of the whole process. Discovering new charities doing tremendous work on issues that I had hardly heard about before.
Good review on the ACX blog of Masooda Bano's 2012 book Breakdown in Pakistan, based on ethnographic case studies, interviews, and surveys of her home country (she's a professor at Oxford). Very worth reading if it hasn't yet come across your Substack feed. Here are some long quotes to whet your appetite.
Vividly shocking opener:
A couple of decades ago in Kabul, a fellow aid worker on his third beer came up with a vivid simile for what American money was doing to the Afghan organizations implementing USAID projects. “It’s like gluing a rabbit mouth-first to a fire hose. You blast its guts right out of its ass…but as long as the water keeps running, it looks like you’ve taught the rabbit to dance.”
Oxford politics professor Masooda Bano uses less lurid language, but Breakdown in Pakistan addresses the same basic phenomenon: how Western aid donors and INGOs unintentionally eviscerate so many of the local institutions they try to “capacity-build.” The problem goes beyond the obvious mismatch of huge funds with small-scale organizations. Even low-to-moderate external funding can leave an organization hollowed out, bereft of local participants and the power it once had to motivate change on the ground.
... Overall, I came to recognize a global machine that systematically:
- sought out organizations with high capacity to mobilize local collective action;
- gutted them of that capacity, in exchange for a new role as professional project implementers and grant managers;
- tried to build new project-specific village committees as an artificial (and desperately inadequate) substitute; and finally
- withdrew funding, leading to the collapse of both the previous institutions and the ersatz ones we’d built up in their place.
We were late-stage German foresters, belatedly trying to import an ant colony or fungi after our monoculture had already devastated the soil ecology.
Why would even low-to-moderate external funding hollow out local organizations?
Voluntary membership organizations, like these voluntary organizations (VOs) had been, are the ideal type of what we call civil society: institutions funded by donations (rather than sales or taxes), operating based on people freely coming together for some shared cause (rather than to make money or obey a government mandate). The foreign aid industry uses “civil society organization” as a virtual synonym for NGO. In reality, aid-funded groups tend to operate rather more like corporate and/or government contractors: paid service providers with comfortably salaried staff, focused on the goals of faraway clients, whose main unifying cause is “whatever helpful thing they pay us to do.”
And as it turns out, that model severely demotivates volunteers. Oxfam’s “capacity-built” Sindhi organizations were certainly non-governmental, but they were no longer voluntary—NGOs, but not VOs. Bano found the same pattern in the 2000s, in a survey of forty successful Pakistani civil society organizations. Of the twenty that implemented foreign-aid-funded projects, none were able to mobilize local volunteers (beyond some interns hoping to graduate to paid employment) and none raised any significant amount of local funding (only three had managed to elicit a few sparse donations from Pakistani companies).
The review goes into much more detail on why volunteers "abandon local collective action groups that reshape themselves to satisfy foreign funders".
Oxfam's capacity-building work in Sindh province as a case study, ft. "workshopias":
Oxfam soon discovered an unexpected dark side to its “capacity-building” work in Sindh province. It had trained six locally prominent welfare organizations in strategic planning and proposal writing, and got them started on implementation by giving them $1000 to work with. The VOs largely met their explicit project targets, and seemed ready to engage with international development funds on a scale much greater than they’d handled before.
Yet all six organisations were visibly more dysfunctional at the end of the project than they had been at the beginning. They were consumed by factionalism, accusations of favoritism, legal battles over control of funds. And they had lost virtually all of their volunteers—notably including the members whose monthly donations and contributions of time had for decades been the backbone of their work. An Oxfam official observed about one group: “From thirty active members, the size of the team shrank to three or four people who had become workshopias,” i.e. those who had gone to every training workshop and built the strongest relationships with the foreign donor. The kind of welfare work the VOs had previously done, based on mobilizing communities for mutual support, was now beyond their capacity.
But so what?
So what? As long as the project achieves its outputs—the disaster relief or cash grants distributed, the latrines built, the agriculture training sessions delivered—why would we care what hair-splitting label gets put on the organization that did it? Shouldn’t we expect professionalism, the profit motive, and higher volumes of funding to yield better long-term results anyway, compared to unreliable, unscalable, harder-to-monitor voluntary work?
Bano is a rational choice theorist, and reminds her readers of the value of voluntary collective action through a recap of some classic theory by Garret Hardin, Mancur Olson, and Elinor Ostrom. Any community’s common interests require people to work together cooperatively; but it’s surprisingly difficult to sustain that cooperation, thanks to free-rider problems and inequalities of motivation and information. Shared resources, in particular, often collapse in a “tragedy of the commons” where individuals’ incentive to use as much as possible leads to overexploitation.
Coercive rules imposed by a state are one way to overcome this problem—but no state can generate all the cooperation we need for a functioning society, and in many countries the authoritarian, exploitative government is an unlikely source of public-interest regulation. That needn’t leave us in despair. Even management of common-pool resources (one of the thorniest collective action problems) can be done effectively through voluntary institutions. In fact, local collective action often yields better results than either state coercion or the individualized incentive structure of competitive markets. ...
At the end of the day, institutions for voluntary collective action always rest—not solely, but significantly—on social trust. And once trust is lost, it’s exceedingly hard to rebuild. ... [Bano's] survey found that even Pakistani VOs that hadn’t taken foreign aid now experienced “difficulty retaining volunteers or inducting new ones from the younger generation, and most of them attributed this difficulty to the rise of the ‘NGO culture.’”
Lots of sobering anecdotes throughout, the reviewer seems to have spent a long time working in aid on the ground.
So what should folks do differently?
Bano doesn’t think the answer is to insulate local organizations entirely from international aid flows (still less to choke off those aid flows, DOGE-style). Instead, she calls on donors and INGOs to take deliberate steps to “minimize the negative impact” of their funding...
... the fundamental change we’d make is to treat a VO’s ability to mobilize local collective action like it’s gold dust. Vocally, publicly value it. Talk to everyone about its strategic importance. Actually measure it—track the capacity of VOs to retain members, attract volunteers, and raise local donations as a core indicator of institutional health. And if it starts to drop, treat that as a crisis on par with financial fraud. Investigate its context-specific roots—what religious, social, and/or political values are motivating people to give time and money to the VO in question—so that we’ll have some idea of how to respond if our work does begin to diminish it.
In our capacity-building work, we wouldn’t focus on turning local groups into part of the management chain for dwindling foreign grants. Rather, we’d primarily aim to give them the equivalent of a “three-self” vision, and the skills to back it up. We’d encourage them to keep a primary focus on the priority problems of their target communities; we’d take care not to overburden them with monitoring and reporting requirements. ...
We wouldn’t aim (either explicitly or implicitly) to replace the “psychosocial and religious incentives” that motivate community volunteers and VO members with material ones, in the name of formalization or professionalization. We’d drop the host of small, commonplace, deeply counterproductive incentives that too many NGOs use to get people to attend trainings and advocacy events—even if that means we have to revise our project events so that local people are actually willing to invest their own time in attending them.
We’d do all we could to avoid destroying the most common signal of commitment—material sacrifice—that reassures local donors and volunteers the VO isn’t exploiting them. Bano doesn’t think this requires everyone working in the aid sector to live like Abdul Sattar Edhi or Mother Teresa. It likely does involve paying below-market salary rates to staff; thorough consultation with all of a VO’s members before agreeing to pay anyone; and setting policies on e.g. vehicle use, office location, and travel per diems with due consideration to public perceptions of waste and wealth. It would require a readiness to adapt incentives over the course of a project or funding partnership, especially if there’s evidence of harm.
We’d prioritize the funding of activities, rather than staff salaries and benefits, and not fund those activities so lavishly as to discourage economy or innovation. ...
EA relevance:
What does Bano’s work imply for the approaches most beloved of effective altruists? Some cost-effective interventions don’t rely very heavily on local collective action capacity. Households can generally make use of bed nets or cash grants even in a low-trust environment; those distributions can be well-handled by NGOs whose primary competency is grant and project management.
The picture shifts slightly as we move into high value-for-money public health measures: deworming, malaria treatment, vaccines. It might seem that a big, salaried state health sector is all that’s required to deliver those at scale, but in a host of countries, the key frontline personnel are actually community health volunteers. The more that “NGO culture” strains collective action, making local people feel like suckers for giving their time to the community without pay, the more friction we’ll be creating for these lifesaving interventions.
We’ll also inevitably encounter situations where people’s ill-health and poverty are grounded in beliefs and attitudes—e.g. the deep suspicion of polio vaccines in the Pashtun borderlands, or the conviction in western Nepal that selling milk to a dairy coop is an intolerable risk (because it might be sold to a low-caste person, angering deities who will kill you or your cow.) A local VO, embedded in its community, can be a powerful and effective ambassador for changes in belief. But as Bano and Nevius both relate, salaried staff with no significant signs of sacrifice often struggle to catalyze this kind of transformation. “You’re just bringing me this message because they’re paying you to,” is a natural discounting factor.
Any humanitarian effort will also quickly run up against politics. Giving a cash grant to landless tenant farmers in Pakistan won’t have much impact when their landlords can just nudge up rent levels to extract the added wealth. Changing that kind of systemic exploitation requires a local institution like AMP whose core competence is risky, politically-charged community organizing. If in searching for a local cash delivery partner, an INGO takes that kind of institution and distracts and depoliticizes it, we’ll ultimately have done more to reinforce than alleviate poverty.
Western altruists could try to invest their donations only in areas requiring minimal entanglement with beliefs and power dynamics. That’s a defensible choice—not solely a drunkard’s search, investing in the areas with the fewest factors confounding our efforts at evidence generation. There’s unquestionably lifesaving work to be done in that space.
The main implication of Bano’s work then becomes a “do no harm” one: channel your funds through an institution that never had much capacity in the first place to mobilize local collective action, so that delivering your preferred interventions doesn’t kill off that capacity. Or (much better) take the time and care to engage with local partner institutions in ways that preserves their capacity to mobilize volunteers. Either way, it requires giving extra consideration not just to EA’s usual core question, “which are the best-evidenced interventions?” but “what are the least damaging institutional pathways for delivering those interventions?”
I'd be curious to get takes on any of this.
Originally published at my Funding Anthropalypse Substack.
Worth noting that cG has increased its $175m to GiveWell to $1 billion, since this was written. https://open.substack.com/pub/fundinganthropalypse/p/coefficient-giving-just-gave-givewell?utm_source=share&utm_medium=eaforum&r=4b7xoz
Great post - love the coverage on your Substack too!
I think that Coefficient Giving’s decision to donate more money to GiveWell is a bad one and I would like to see more discussion of its pros and cons as well as any resources folks have that explain why CG made this decision.
The short form of why I think this is a bad call is that I think that for a wide range of beliefs on other issues (AI, Bio Sec, US Democracy, Reproductive Tech, emerging farming tech) the 10 year expected return on donations is much higher than in global health. Maybe this has already been talked to death but I’d like to see a discussion on it and if anyone has resources to past arguments I’d love to see them.