We organised an EA weekend in London for people who are already familiar with EA. A review of CZEA "Intense EA Weekend" retreat was helpful to us so I am sharing the lessons we learned as well.
Table of contents
Main lesson: 1 on 1s seem to be very useful
Event descriptions and lessons from running them
Productive disagreements workshop
Discussing various topics in small groups
"How to talk about EA" workshop
Main lesson: 1 on 1s seem to be very useful
We experimented with various event formats. In a feedback form we asked to rate how useful each event was (for having a greater positive impact) on a scale of 1 to 10. According to the feedback, the most valuable event was pairing people up to talk 1 on 1 for 30 minutes:
|
Activity |
Weighted and normalised* |
Unweighted |
Vote count |
|
1-1s |
7.3 |
6.8 |
24 |
|
Career Club |
6.6 |
5.9 |
16 |
|
Murphyjitsu (next steps): how useful? |
6.3 |
5.4 |
17 |
|
Hamming Circles |
6.2 |
5.6 |
20 |
|
Socialising after the events |
6.1 |
5.6 |
21 |
|
Decision Making workshop |
6 |
5.3 |
14 |
|
“How to talk about EA” workshop |
5.5 |
4.8 |
17 |
|
Lightning talks |
5.4 |
4.9 |
20 |
|
Productivity workshop |
5.4 |
4.6 |
8 |
|
Productive disagreements workshop |
5.3 |
4.8 |
19 |
|
1 minute introductions |
5.1 |
4.6 |
19 |
|
Discussion about where to donate |
5 |
4.7 |
7 |
|
Explaining Concepts |
5 |
4.7 |
20 |
|
Discussion about diversity within EA |
4.4 |
4.2 |
10 |
*see this comment for explanation
This is consistent with feedback from group leaders' retreat where 1 on 1s were voted to be the most valuable activity on most days. What is more, when asked what the most enjoyable part of the weekend was, six respondents answered that it was 1 on 1s (the most common answer).
We gave each of the participants a piece of paper with a list of questions to ask each other to jump-start the conversations. I think we told them that asking the questions was optional. The questions may have helped to avoid the fear of running out of topics to talk about, even if they weren't used directly. We tried to pair people who don’t know each other well and have something in common.
The rest of this post does not contain anything important but could be useful to people who are organising similar events.
Overall feedback
30 people attended on at least one of the days (excluding 4 organisers). 25 attendees filled the feedback form. One of the questions was:
Here are the results:
|
Answer |
Count |
|
About as valuable (roughly the same as counterfactual) |
1 |
|
Somewhat more valuable (1-3x the counterfactual) |
8 |
|
Much more valuable (3-10x the counterfactual) |
9 |
|
Far more valuable (10-30x the counterfactual) |
5 |
|
Vastly more valuable (>30x counterfactual) |
2 |
It’s unclear how good people are at answering such a difficult question but overall, the event seems to have been successful.
You can see a full summary of responses to the feedback form here. It includes answers to some free text questions that are not discussed in this post.
Schedule
Saturday
9:30 - 10:30 Arrivals & registration
10:30 - 10:45 Overview & schedule
10:45 - 11:45 One-minute introductions
11:50 - 13:00 Explaining concepts
13:00 - 14:00 Lunch
14:00 - 15:30 Productive disagreements workshop
15:40 - 16:40 Decision-making workshop or circling in a park
16:50 - 18:00 1-1s
18:00 - 23:00 Socialising, dinner
Sunday
9:30- 10:25 Arrivals
10:25 - 11:00 Group loving-kindness meditation or discussion in another room
11:00 - 11:15 Intro to the day
11:15 - 12:20 Hamming circles/Planning for the year
12:25 - 13:00 1-1
13:00 - 14:00 Lunch
14:00 - 14:30 1-1
14:30 - 14:40 Group photo
14:40 - 15:40 Career club or discussion about where to donate in another room
15:50 - 16:15 Lightning talks (Moral Circle Expansion and a short talk about tax-efficiency)
16:15 - 16:50 Discussing various topics in small groups
16:50 - 17:25 "How to talk about EA" workshop
17:30 - 18:15 Problem-solving circle: next steps
18:15 - 18:30 Feedback
18:30 - 22:00 Socialising, laughter yoga
Slides from the event can be seen here.
Event descriptions and lessons from running them
One-minute introductions
Everyone had 1 minute to introduce themselves. The purpose was for people who have something to talk about to know about each other. The following slide was shown to give people ideas on what to talk about:
Questions were ordered and picked in a way to make it less intimidating to talk. However, it caused many of the answers to be less informative. It would’ve probably been better to put questions like “What’s your plan for improving the world?” first. It was also difficult to hear what many people said partly due to bad acoustics. Two people wrote that they were uncomfortable introducing themselves. Overall, some online document or a sheet on the wall would have probably been better.
Explaining concepts
This is an idea we took from CZEA "Intense EA Weekend" retreat because this was rated as the most useful event at that retreat. The participants were asked to pick from a list of interesting concepts (e.g. scope insensitivity, optimiser’s curse) which ones they would like to have explained to them or explain to others. We used a different sheet from CZEA. Our sheet can be seen here. We asked people to divide into pairs or small groups and find different people when they hear a gong (every 15 minutes). We told that it is ok to skip topics if they don’t want to explain them or have them explain. According to the feedback, this was not a successful event.
Productive disagreements workshop
This idea was taken from an event that was ran in New York and adapted for EA purposes. First, I gave a short talk on how to disagree productively. I then asked everyone to rate whether they "agree", "disagree" or are "unsure" on various controversial EA topics and give their opinions on the importance of various causes. A sheet of questions can be seen here (note that there are two tabs). We then asked participants to pair up, compare sheets, find disagreements and discuss. Every 15 minutes we asked everyone to find a different pair.
One piece of feedback about the event was: “Hard to find disagreement. Many statements were too hard to have a confident belief on.” I think another issue was that many people didn’t know enough about some of the topics to have meaningful discussions.
One idea was that explaining concepts and productive disagreements workshops could help people to have more productive discussions during 1 on 1s because it sets a tone of talking about EA topics. What is more, people could continue one of these activities during the 1 on 1s. Some people told me that events were successful in this regard. However, it still seems like overall they weren’t very successful.
Discussing various topics in small groups
The idea was:
- People suggest many topics
- I read the list of topics and everyone votes on the ones they want to discuss (multiple votes allowed)
- People separate into groups based on which topic they want to discuss the most.
This is the way discussions are organised at a philosophy meetup I occasionally attend. One mistake I made is not giving people a minute to think of topics to discuss. The only topics suggested were
- Diversity within EA community
- Value drift
- Productivity and self-improvement (this wasn’t really a topic, it had prepared materials)
We separated into three groups to discuss these. Discussions did not work too well, possibly because the groups were too big.
"How to talk about EA" workshop
We separated into 4 groups based on which subtopic we wanted to discuss. Subtopics were:
- How to talk about EA with people interested in you
- How to talk about EA with people interested in the ideas
- How to talk about EA with people you want to be interested in the ideas
- How to talk about EA with people already on board
In the feedback form, two people mentioned that they’d rather this was a workshop (possibly a one that includes roleplay) rather than a discussion.
Application forms
We publicly advertised an expression of interest google form. Purposes were:
- See how many people are interested
- Filter out the people who are not yet familiar with EA (because the event was not aimed at newcomers). This required quite a lot of effort. I don't think it was worth it.
- See which events people are most interested in attending
Out of 57 people who filled the form, only 14 attended the event. This is partly because we publicized the form 2 months before the event but sent tickets only 2 weeks before the event because that’s when the availability of the venue was confirmed. Some people made other plans for the weekend because they weren’t sure if the event is going to happen. We had to send the tickets sooner with no venue specified. Many of the attendees came because we invited them personally 2 weeks before the weekend. Some of them also invited other people. There could have been many more attendees if we sent invitations sooner. However, it’s unclear whether it would’ve been easy to manage many more people.
Thanks for writing this up. Three questions
The numbers on how useful things are seem quite low to me. What did you write as the ends of the scale? I'm thinking in terms of net promoter scores where anything below a 9 or a 10 is considered neutral or bad.
Can you explain Hamming circles? I couldn't find out how they worked even after a quick google.
Did you ask people if there was anything they wanted to do on the weekend but didn't do? I'd be curious to see if people came up with anything.
3) Because of your question I added a link to the summary of the responses to the feedback form to the article. Here are responses I see that suggest different activities:
We also had some event ideas ourselves that we decided not to include in the weekend:
2) I will ask the person who ran Hamming circles to explain it, I only participated in half of the event myself. You can get some idea from the slides (slides 21- 28).
1) >"The numbers on how useful things are seem quite low to me..."
On the scale 1 was "Useless" and 10 was "Life-transforming". But just before asking for feedback, I made a change in the slides and added this meaning to the ratings of the events:
I explained it to people as well. This was... not smart. Because of this, some respondents gave low scores to all the events. E.g. someone said that the weekend was "Far more valuable (10-30x the counterfactual)" but did not gave any event a rating that is higher than 4. Others ignored the point and gave high ratings for all events.
That's why I weighted and normalised the ratings. If someone said that the weekend was "Vastly more valuable (>30x counterfactual)", I multiplied all their ratings by a constant so that their highest rating would be 10. If they rated the weekend as "Far more valuable (10-30x the counterfactual)", I multiplied all their ratings so that the highest rating would be 9. 8 for "Much more valuable", 7 for "Somewhat more valuable", and 6 for "About as valuable".
Yeah, I thought the ends of the scales might have been more extreme than we'd normally use. It's probably quite hard to get people to sensibly answer unfamiliar, tricky questions.
I guess this very much depends on how individual activities are executed. We had our 2.5 day retreat in Dortmund, Germany about a month ago, and while I didn't see the evaluation results, I got a strong impression that most people agreed on these points (still, take this with a grain of salt):
career discussion in small groups (~3-5) was quite useful; we had about 1 hour per group, and more would probably have been better.
double crux (I guess similar to productive disagreement?) was a cool concept, but a bit difficult to execute under the given circumstances (although it worked great for me), for similar reasons as mentioned by you
discussion about where to donate - this was, to some degree, what this weekend was primarily about for us, as we raised money on the first evening and then had to figure out where to send it. And while it started very slowly, we ended up spending many hours on Sunday on this (very open) discussion, and it was tremendously valuable. I really didn't expect this, but ultimately, judging from how engaged everybody was, how interesting our conversations were in the end, and how often each of us changed their mind over the course of the discussion, this was a great way to spend our time
Hi Saulius. CZEA retreat organizer here.
Thank you for this report.
We did "concept explaining somewhat differently:
It seems to me that Circling and Hamming circles need the people to feel close and trusting each other. We thought about doing Hamming circles on Saturday evening but decided to do something else because the atmosphere was not right. How well it worked for you?
I didn't attend, but someone told me the circling thing was awkward.
I see, that is very different.
I only know about the feedback I already presented and the one hamming circle I myself participated in. In my circle all 3 of us were quite connected already. It may have helped but I'm not sure it was necessary. Both of them didn't know what to do with their careers so it was like a mini career-coaching session. It felt productive.
oh, I didn't notice that there was a question about circling as well. There are circling meetups where I did circling with strangers and it was great. So it's not necessary to be close with people before, though the experience is quite different if you are. I imagine that many people will find circling awkward no matter what but in this case I think we simply did not have enough time to get into it. We only allocated 1 hour for it (and I'm not sure even that is enough) and then we decided to do it in the park but walking to the park took much more time than expected. It was the lowest rated activity (3.7) but I didn't include it in the table of the article because I believe that it wasn't given a fair chance.
Thanks for the write-up, I was wondering how concretely did you run the 1-1s. I saw the list of questions in the google slides, did you just ask people to discuss any of these? Also, did you pair people up or did they pair up themselves?
People were free to talk about anything they want. I'm pretty sure we told that using the questions was optional. I changed the post slightly to make it more clear. Personally, I didn't use the questions in any of my 1 on 1s. I know some people used the questions and they said that the first two questions ("When did you know that you wanted to be altruistic and why?" and "When did you realise you care about effectiveness and why?") resulted in long and not very productive conversations. So I moved them to the end in the questions document. I think that the list of questions has a lot of room for improvement. Maybe next time I will say that if they want to use the questions, they should look through the list and decide which question they are the most curious about.
We paired people up ourselves. We tried to pair up people who don't know each other well and have something in common (e.g. both are excited about the same cause area). We considered letting people pair themselves up but had reservations because some people might get upset if no one wants to talk with them. However, at the end of the first day we told people that if there is someone they'd like to be paired up with on the second day, they can write it on a piece of paper and put it in the box (so that only we could see their preference). But very few (2-4) people did that.
Btw, we will run an event that is all about 1 on 1s (https://www.facebook.com/events/330349444169839/) to see if it is a good stand-alone event (because it could be that it works only in the context of a weekend/conference/retreat). I'll write here about how it went, if I'll remember to do it :)
Those are really useful comments, thanks. I wonder how did the event go?
It went well, I asked ~8 people for feedback and it was all positive, almost everyone said that they liked it better than a regular social. One person said that it made them less socially anxious than regular socials. I think we'll organise these every other month or something like that (in addition to socials).
We first quickly presented what topics each of us would most want to talk about, then we paired people up, based on that (though a lot of pairings were random). Each person had three one on ones and then we all had a picnic. I'm not sure how well it would work if the weather was less nice and we couldn't do it in a park (or a building with many breakout rooms like at the EA weekend).
What did you think about the length of the retreat? Would people have liked to stay longer? We're planning to organize a retreat from Thursday to Sunday in The Netherlands in between Christmas and New Year's.
I don't know, we simply didn't talk about that at all. My guess is that 4 days is not too long. EA globals sometimes last 3 days, if you include the social on Friday. I believe that a recent group organisers' retreat lasted an entire week. An AI camp lasted 10 days. These latter two events are not quite the same, but I guess you could ask Remmelt Ellen whether they felt too long, I believe he was present in both of them. Hmm, the fact that your event is during winter could matter a bit though, because going outside is usually a refreshing change of atmoshpere during such things.
By the way, this was not a retreat, we did it in an office in London and people slept elsewhere.
Alright thanks! :) Remmelt is also organizing this retreat, so we have that info!