I'll point out, you now are describing the actual engineering work needed to achieve privacy and reliability, which includes context limiting for privacy's sake, and a backup scheme. That might need to be "above the fold" stuff if you're trying to get people to use LLMs for administrivia.
You may need to consider badness v. likeliness of mishaps - for example, I think it can wipe your calendar, and while that's unlikely and it's hard to imagine the exact prompt that would cause it, it's not really unprecedented either. Anxiety about these things will go up as you move outward from tech enthusiasts, and I think it's unclear whose instinct is right here - maybe they're too cautious, maybe you're taking too much risk. If you're an entrepreneur/seller, you want empathy to your customer's risk tolerance. If it's higher, they care more about this type of problem. Maybe you would try to sell those customers on readonly workflows.
I really think the key concept in modern AI prompting / coworking / engineering is verifiability. Software engineers are mostly not stepping back from reading / reviewing AI code, and people sending out real emails really aren't going to want to step back from verifiability.
From a salesmanship point of view - you probably should stress the actual problems it can solve more, and not the fact that it's generally capable and can do lots of stuff if you just connect it to everything. That's a bit of a "draw the rest of the owl" move, and I guess some people who are have tech adopter / tinkerer mindsets will respond to your push, but if you're trying to work your way through skeptics with less time experiment, you really need to solve problems they actually have, not give them half-solutions to nonspecific problems. (Key book, aimed at entrepreneurs: "The Mom Test.")
Does either of these come off as slop to you?
Short answer: I think that was a good automation, because the "sent by Claude" transparency helps a lot.
Analysis:
In my mental model, "slop" is when the author thinks the AI substitute is human quality, and the reader can tell it's not. If you say "Sent by Claude" then that pretense isn't really there and you and reader are aligned this was an automated process not intended to be empathetic communication. There still might be quality loss, as is true for any automation, LLM or not.
Worth considering where you did make tradeoffs:
The reader isn't sure how much you pre-screened - they just got an AI summary and can screen for themselves.
You lost a chance to make a personal plea like "I was chatting about this with a few people, ..."
It may be better to describe the tradeoff explicitly when it comes up, rather than handwave like, this was easy to automate and there's no slop effect.
All of which seems very intuitive; these LLMs can do agentic work now, so why isn't everyone using them? I have no idea, but from first-hand experience I rarely such LLM use by EA community organizers.
Meta:
your top priority really should be understanding that. Worst case is they have a good reason to not use LLMs, and you might want to worry about that reason to.
Object:
Worst catastrophic error: privacy leak because of shared context.
"Read all my emails then tell my student Bob I'm late"=> "Hi Bob, I'm late, sorry, really hectic day dealing with all the changes going on in the department!" and Bob is thinking "?? what changes?"
Second worst catastrophic error: it deletes all your data.
Other problem: are your sure your AI writing isn't coming across as slop? Your example with forwarding the recommendation was lossy - the LLM did not forward your reason for thinking it was good for the group, it made up a new one. My bet is people can tell your slop is slop more than you realize and I really recommend assuming all writing you do is empathetic thinking involving your listener. It is not safe to outsource to LLM while being an organizer (or anyone who... thinks.)
Your prior that the problem is not anything other than low pay should be incredibly strong. If, as is prognosticated, a huge sum of money flows into AIS, then Google senior staff engineers and VPs will be in play to lead research projects, and I strongly suspect the prerequisite that the candidate "must be AIS-pilled" will quietly resolve itself. Pricey management consultants at McKinsey and BCG will be retained to speed all this up.
My contrarian stance: veganism is not EA. The impact of veganism on your time, money, health, budget, enjoyment of veganism is substantial. You're likely doing useful, impactful things with your time and energy when you have them, and veganism is a drain on your time and energy.
The only thing that tips the scale toward veganism in my judgment (I do not presently practice veganism), is the practice just interests you. Despite the above detriments, most people can go vegan if they want. If you love it, feel empathetic to animals, feel it refreshes your values, sure, do it.
But it's not the sort of practice I would make central to EA. It should be as taboo as playing guitar, basketball, or backpacking, which is to say not taboo, but not automatically on topic.
hate datacenters, would support just about any anti-dc regulation
if you believe anything about the HF incident or any AI capabilities, you are a shill falling for AI marketing
I think both views are propagandized. My mental model is, propaganda hardens and narrows political beliefs, and I think on both points, the social media consensus has gone beyond what widespread sincere belief looks like.
The distilled factual version is really not too hard for ordinary people to understand. "The models are very powerful and they can't control them." If people are just never really getting to that, it's not complexity, it's some other psychological or social force. (I concede the big picture conclusion which is like "oh and it can attempt take over," sounds way more absurd to an AIS-novice than the constituent parts.) Any propaganda researchers reading? I'd like to know WTF is going on with point #2 above.
Recently, I thought you could take something like anti-dc sentiment and attach wonkish AIS riders. I think I'm right that negative AI valence is very important, but I'm more concerned propaganda can interfere with "valence => desired action."
On the other hand -- a number of elites seem to have noticed what happened, e.g. D Ted Lieu and R Nathaniel Moran with AI Kill Switch Act.
I guess I should point out that Stoicism tells you don't stare into the abyss. It's explicit that inner circles matter more than outer circles. It's not compatible with utilitarianism. You are believing different things as a Stoic than you are as a utilitarian, and when you think about your beliefs you have different thoughts. I haven't seen any EA endorse not agreeing with utilitarianism. If you do, that opens a lot more options in mental health, and you can just directly do the Stoic thing of caring about yourself more than others because it's moral to.
A few years ago it was: stare into the abyss of tremendous amounts of suffering, on the order of trillions or quadrillions of animals, now and into the future. Literally Soares' "Guilt" series says you should do this as a virtue.
Now it's: stare into the abyss of imminent AI-caused extinction.
These tenets imply mental health ought to be a foremost priority, no? What I observe is something more like this essay -- after a few years of realizing poor EA mental health isn't correlative but it's something our philosophy specifically induces, we write a few self-help posts about it, probably achieving the same efficacy as reading the first few chapters out of Ryan Holiday's book. I might even cite the comments here as evidence we're not effective: they're mostly along the theme of, anxiety that doing more than the minimum of self-care sacrifices performance.
Politician as in "running for office" is a little distinct, I definitely agree/get that we want some EA/ people doing policy especially in AIS. That was possibly a weak example, but it's hard to think of specifically unexpected examples of something.
I appreciate this reading it as a critique, but I really have no way to tell if the claims add up to the conclusion.
For example: growth can be bad, EA strikes many as an ideology, fewer people self-ID as EA, EA's funnel seems in confict with truth seeking, the central/corporate pressure seems at odds with community building -- those are all worth keeping in mind. I just don't know if they add up to a damning outcome or not.
Which is not to say he'll definitely win: it's a competitive field and he's at 42% on Manifold. Still, I decided to donate,
You should specifically donate to candidates who won't "definitely" win. Closest to 50% is best, and 42% is pretty darn close to 50%. My heuristic is to try treating 25-75 as a competitive race.
For candidates <50%, you have util/fuzzy split: it's fun to win, but util cares about flipping a loss to a win, not just being on the winning side.
(Caveats, out of scope to this specific point: betting markets are sometimes doctor-able or can have multiple equilibria, early momentum for a candidate at near 0% is a special case)
I'll point out, you now are describing the actual engineering work needed to achieve privacy and reliability, which includes context limiting for privacy's sake, and a backup scheme. That might need to be "above the fold" stuff if you're trying to get people to use LLMs for administrivia.
You may need to consider badness v. likeliness of mishaps - for example, I think it can wipe your calendar, and while that's unlikely and it's hard to imagine the exact prompt that would cause it, it's not really unprecedented either. Anxiety about these things will go up as you move outward from tech enthusiasts, and I think it's unclear whose instinct is right here - maybe they're too cautious, maybe you're taking too much risk. If you're an entrepreneur/seller, you want empathy to your customer's risk tolerance. If it's higher, they care more about this type of problem. Maybe you would try to sell those customers on readonly workflows.
I really think the key concept in modern AI prompting / coworking / engineering is verifiability. Software engineers are mostly not stepping back from reading / reviewing AI code, and people sending out real emails really aren't going to want to step back from verifiability.
From a salesmanship point of view - you probably should stress the actual problems it can solve more, and not the fact that it's generally capable and can do lots of stuff if you just connect it to everything. That's a bit of a "draw the rest of the owl" move, and I guess some people who are have tech adopter / tinkerer mindsets will respond to your push, but if you're trying to work your way through skeptics with less time experiment, you really need to solve problems they actually have, not give them half-solutions to nonspecific problems. (Key book, aimed at entrepreneurs: "The Mom Test.")
Short answer: I think that was a good automation, because the "sent by Claude" transparency helps a lot.
Analysis:
In my mental model, "slop" is when the author thinks the AI substitute is human quality, and the reader can tell it's not. If you say "Sent by Claude" then that pretense isn't really there and you and reader are aligned this was an automated process not intended to be empathetic communication. There still might be quality loss, as is true for any automation, LLM or not.
Worth considering where you did make tradeoffs:
It may be better to describe the tradeoff explicitly when it comes up, rather than handwave like, this was easy to automate and there's no slop effect.
Meta:
your top priority really should be understanding that. Worst case is they have a good reason to not use LLMs, and you might want to worry about that reason to.
Object:
Worst catastrophic error: privacy leak because of shared context.
"Read all my emails then tell my student Bob I'm late"=> "Hi Bob, I'm late, sorry, really hectic day dealing with all the changes going on in the department!" and Bob is thinking "?? what changes?"
Second worst catastrophic error: it deletes all your data.
Other problem: are your sure your AI writing isn't coming across as slop? Your example with forwarding the recommendation was lossy - the LLM did not forward your reason for thinking it was good for the group, it made up a new one. My bet is people can tell your slop is slop more than you realize and I really recommend assuming all writing you do is empathetic thinking involving your listener. It is not safe to outsource to LLM while being an organizer (or anyone who... thinks.)
Your prior that the problem is not anything other than low pay should be incredibly strong. If, as is prognosticated, a huge sum of money flows into AIS, then Google senior staff engineers and VPs will be in play to lead research projects, and I strongly suspect the prerequisite that the candidate "must be AIS-pilled" will quietly resolve itself. Pricey management consultants at McKinsey and BCG will be retained to speed all this up.
My contrarian stance: veganism is not EA. The impact of veganism on your time, money, health, budget, enjoyment of veganism is substantial. You're likely doing useful, impactful things with your time and energy when you have them, and veganism is a drain on your time and energy.
The only thing that tips the scale toward veganism in my judgment (I do not presently practice veganism), is the practice just interests you. Despite the above detriments, most people can go vegan if they want. If you love it, feel empathetic to animals, feel it refreshes your values, sure, do it.
But it's not the sort of practice I would make central to EA. It should be as taboo as playing guitar, basketball, or backpacking, which is to say not taboo, but not automatically on topic.
Reddit chatter is like:
I think both views are propagandized. My mental model is, propaganda hardens and narrows political beliefs, and I think on both points, the social media consensus has gone beyond what widespread sincere belief looks like.
The distilled factual version is really not too hard for ordinary people to understand. "The models are very powerful and they can't control them." If people are just never really getting to that, it's not complexity, it's some other psychological or social force. (I concede the big picture conclusion which is like "oh and it can attempt take over," sounds way more absurd to an AIS-novice than the constituent parts.) Any propaganda researchers reading? I'd like to know WTF is going on with point #2 above.
Recently, I thought you could take something like anti-dc sentiment and attach wonkish AIS riders. I think I'm right that negative AI valence is very important, but I'm more concerned propaganda can interfere with "valence => desired action."
On the other hand -- a number of elites seem to have noticed what happened, e.g. D Ted Lieu and R Nathaniel Moran with AI Kill Switch Act.
I guess I should point out that Stoicism tells you don't stare into the abyss. It's explicit that inner circles matter more than outer circles. It's not compatible with utilitarianism. You are believing different things as a Stoic than you are as a utilitarian, and when you think about your beliefs you have different thoughts. I haven't seen any EA endorse not agreeing with utilitarianism. If you do, that opens a lot more options in mental health, and you can just directly do the Stoic thing of caring about yourself more than others because it's moral to.
Doesn't EA do the minimum though?
These tenets imply mental health ought to be a foremost priority, no? What I observe is something more like this essay -- after a few years of realizing poor EA mental health isn't correlative but it's something our philosophy specifically induces, we write a few self-help posts about it, probably achieving the same efficacy as reading the first few chapters out of Ryan Holiday's book. I might even cite the comments here as evidence we're not effective: they're mostly along the theme of, anxiety that doing more than the minimum of self-care sacrifices performance.
Politician as in "running for office" is a little distinct, I definitely agree/get that we want some EA/ people doing policy especially in AIS. That was possibly a weak example, but it's hard to think of specifically unexpected examples of something.
I appreciate this reading it as a critique, but I really have no way to tell if the claims add up to the conclusion.
For example: growth can be bad, EA strikes many as an ideology, fewer people self-ID as EA, EA's funnel seems in confict with truth seeking, the central/corporate pressure seems at odds with community building -- those are all worth keeping in mind. I just don't know if they add up to a damning outcome or not.
Pointing this out:
You should specifically donate to candidates who won't "definitely" win. Closest to 50% is best, and 42% is pretty darn close to 50%. My heuristic is to try treating 25-75 as a competitive race.
For candidates <50%, you have util/fuzzy split: it's fun to win, but util cares about flipping a loss to a win, not just being on the winning side.
(Caveats, out of scope to this specific point: betting markets are sometimes doctor-able or can have multiple equilibria, early momentum for a candidate at near 0% is a special case)