MLB: Intensive ML Bootcamp by Aalto EA

Contents

• Introduction
• Curriculum
• Flexibility Summary
• Selectiveness Summary
• Intensiveness Summary
• Peer Learning Summary
• Misc notes
• Feedback from Participants
• What were the best bits? (participant responses)
• What stuff could have been improved? (participant responses)
• Free comments: Have something else to say that might be interesting? please take a minute to pause here before moving on. (This is probably the most important section in this form for me) (participant responses)
• Appendix

Introduction

Aalto EA organized an ML Bootcamp from the 16th-26th of August 2022. It was meant as an environment that people could use to ramp up their ML skills. It turned out to be an interesting success. Here, I lay down what happened in more detail, in case you’re curious or want to replicate it (or some aspects of it) in your group.

Initially, it started off as a selfish idea where I wanted to attend MLAB by Redwood Research but couldn’t due to some visa reasons. Then I figured that I could just replicate the whole environment of the bootcamp in Finland instead of doing something just by myself in my room. This is how it started. Then I had some discussions with the EAs in my group and in Finland, and this seemed like a very good opportunity to try something we’ve never done before, and also upskill AI-interested EAs in necessary skills - which is a visible bottleneck in our group. We have many people interested in AI Safety, but most of us seem to not have the skills necessary to get hired by the super-competitive AI-safety orgs right now.

We started with the conservative metric that if this bootcamp leads to any one of the participants starting to work in AI Safety in the next 1-1.5 years, we would consider this a success. Some ways in which the bootcamp was (partially) AI-safety focused were:

• AI-safety interested people had an edge in the selection process.
• Quite a few of our informal discussions (lunch, breaks, etc) were centered around AI Safety and Risk
• Day 4 was an AI Safety day with Joel Pyykkö (an independent AI Safety researcher), where everyone had a primer on AI Risk and a discussion afterwards. The whole thing lasted almost 4 hours and was super cool!

Curriculum

MLB Info Guide

Flexibility Summary

Flexibility in curriculum is good in the sense that people can pursue their own learning journeys and do what’s best for them. Especially works for self-directed individuals. It’s bad in the sense that in a peer-learning environment if people diverge too much in what they’re doing it negatively affects the group and reduces interaction. I’m not sure how to find the optimal balance here.

Selectiveness Summary

We used a selective and elite framing for our bootcamp to attract applicants. We advertised that we would only accept 4-6 participants, but received 16 applications and ultimately accepted 9. The approach seemed to work, as some participants were excited to be accepted.

Intensiveness Summary

Our program was designed to be intensive and ambitious^[1], with participants attending regularly for 9 days and working for roughly 7 hours each day. This approach worked well for some people, providing the certainty that they would be focused on the topic for the duration of the program. However, it was difficult to sustain this level of intensity for 9 days, and in the future, we would still aim for high intensity but over a shorter period of time.

Peer Learning Summary

We initially chose peer-learning due to resource constraints, but it turned out to be a good choice. The participants had very different perspectives which led to very interesting discussions, which was one of the highlights of the program. I will definitely optimize more for this in the future, for example by cultivating a more friendly, open, and taking confusions and ideas seriously vibe. The flexibility of the curriculum and the variance in skills within the participants hindered this quite a bit, and I would take care to pay more attention to these two factors more to induce a better peer-learning environment.

Misc notes

Going outside during pomodoro breaks and doing some physical activity is really fun and useful. We had a tennis ball we would throw around, but anything works.

Feedback from Participants

What were the best bits? (participant responses)

• having a dedicated time and space to work on a project (here increasing ML skills)
• The best conversations, where everyone contributed and we were going hard at a problem and then emerged victorious with an answer (omg this could be facilitated with something like daily post-lunch hard random question, just nerdsnipe the group and watch the discussion unfold)
• Conversations, throwing the tennis ball
• (1) Setting public goals and reflecting on progress, (2) useful materials I hadn't noticed like the mathematical framework for transformers, (3) discussions often forced me to discover holes in my knowledge and patch them.
• opportunity to learn in a social setting
• Finishing the day feeling like you had done some work
• Dedicated place and timing. Having breaks in between and physical activities, intellectual discussion in the breaks, guidance and motivation to work in ML. Working in our own pace without feeling any pressure.

What stuff could have been improved? (participant responses)

• grouping up a little more, but this depends totaly on the attending people and their level of knowledge, so nothing the organizer could change
• The classroom vibe felt a bit tense for me for some reason - also this place is far from my home - and a lot of the free convos, esp toward the end of the bootcamp, were too advanced for me to understand, so the convos got less useful for me.
• Maybe giving a specific time to teach/give information about some tools.
  I think the length and intensity targets of the bootcamp were at cross-purposes. It's hard to sustain a high level of intensity for 9 days. Either a shorter, more intense period like a hackathon weekend or a longer, less intense bootcamp might be easier to get buy-in for from participants. I didn't really find this a huge problem, though.
• the 3 negative points above: 1) vegan snacks if possible. 2) a room with tables and bigger whiteboard. 3) although the flexibility in curriculum might be good, it also felt like people were doing different things most of the time so it wasn't much different from just learning by myself. Perhaps some structure could have made it more collaborative
• I feel there was a bit too much freedom and distraction. It's an inherent problem in putting people at different stages in their learning journey in the same place, so we couldn't build enough momentum.

Free comments: Have something else to say that might be interesting? please take a minute to pause here before moving on. (This is probably the most important section in this form for me)

• I was able to achieve my goal set for this bootcamp, which I am happy about. Although I am not totally satisfied with the algorithms I made, this is mainly because of the limited time I had. I really enjoyed the atmosphere in here, it helped me work and stay focused. Also it was nice to get to know all of you guys :)
• Good job, great success :) Next time I think it's safe to have a bit simpler snacks. To have lunches covered is really nice. I hope you keep doing "bootcamps" or whatever cool projects you come up with that have positive overflow effects to other people. This is great. Also you should write a post about this experience, detailing how and why you did it, what kinds of results were there, and how other people can replicate.
• Organization and the idea behind this bootcamp was good. Thanks for your efforts and friendly nature.
• thank you for the bootcamp, it was great!
• Could implement some Cal Newport’s Deep Work ideas in the future.

Appendix

Personal

Personally, I was super skeptical of this being a success. I had mentally prepared myself for maybe getting 1-2 more participants, but everyone dropping out except me by the end. And then I was organizing this on the motivation that I would have to be the 1 person to starts working in AI Safety soon, for the bootcamp to be a success. This is a very weird rationalization - I get it - but it helped me go ahead with it.

Selectiveness Babble

Numbers

• Applicants
  • Google form: 13
  • Direct Contact: 3
• Accepted: 9 total
• Attended: 8
• Finished: 5

It generally gets quite hard to get people to sign up for an event, more so if you require a lot of time and effort from them. In this bootcamp, I was basically asking people to commit two weeks of full-time work to it. So the potential applicant set was already pretty small. We tried out some strategies to help us get more people. I think they worked well, but maybe the counterfactual would’ve been the same. I can’t think of a low-cost way to judge. We projected the bootcamp as a highly selective and elite program, asking a lot from the participants (instead of the opposite framing of us being the one in need which we generally follow), where you would feel hyped if you were accepted. I picked this approach from the Ivy league in general, and Slush in finland for example. It seems to have worked well. We advertised we would accept4-6 individuals, got 16 applications, and ended up getting 9 participants in total. 
I went with 9 participants, even though I’d mentioned I would take 4-6 because I realized that I could handle more, and the 4-6 was a more conservative number I came up with because I wasn’t sure we would even get that many people. Some participants, upon receiving the acceptance, were super hyped - which felt kinda surprising and nice.

Peer Learning Babble

The initial reason for choosing peer-learning instead of “teaching” was resource constraints, but it turned out to be a good choice. Especially during the early half, when most participants were working on similar stuff. Many participants were approaching the concepts from very different angles, and the discussions we had would sometimes turn super intense, lead even the more experienced participants to notice flaws or gaps in their understanding, have multiple people figuring out together what exactly goes on in a neural network, and overall were one of the best parts of the bootcamp. I personally learned a lot more than I would have in a more passive or personal environment.

I am happy with the peer-learning outcomes, and would spend much more effort in the future on how to facilitate the environment better to further cultivate this. 
Trying to cultivate a friendly and welcoming atmosphere, and explicitly reducing the barrier to ask and answer questions, for example by personally doing it or encouraging people when they do it, are some things to focus on. It also helps to have people thinking about roughly similar concepts at the same time.

I think having a couple more experienced people in the group to help think through things is very useful. It also strongly matters how similar the skills are of the people in the group, and the similarly of the projects they’re working on during the bootcamp. Regarding skills, I noticed it’s useful to have some kind of a lower bound (= you need to be at least this good), but having an uncapped upper bound is ideal and leads to interesting discussions. Regarding similarity of projects, people did like flexibility in what they could work on but it did strongly affect the peer-learning aspect and there’s definitely a tradeoff here. 

Intensiveness Babble

The program, inspired from MLAB, was designed to be intensive in nature. The participants were expected to attend regularly for 9 days, with roughly 7 hours of co-working each day. The simple idea behind this was to create an ambitious environment for everyone to get shit done. Check 42’s piscine. Naturally, there were both pros and cons to this approach.

The full-time approach worked really well for some people, providing the certainty that this is what they’re gonna be doing for the next 9 days - helping them focus in on the topic. This mindset did feel hard to sustain though as time went by, and that was the biggest con. In the future, I would still keep the intensity high but constrain it to fewer days. Sprints are incredible, but hard to sustain for long.

Babbling daily reflection (1st half of bootcamp only)

Day 1 Tue
The bootcamp started off super badly, with only 2 people arriving on time. It was super awkward to start the program with just those two, and maintain a positive environment until the rest arrived (1-2 hours later). Apparently, I found out that 10am is still too early morning for most students.

Once a few more people did arrive though, the day ended up pretty well. We started working on the curriculum. Also had a really good lunch discussion which one person was fomoing about because he didn’t get to attend the discussion. We had multiple really good discussions during the day as well. Most notably on backprop.

The end-of-day ref
We decided to implement pomodoros, 
Also pair coding. But I’m very unsure how to go about this so I didn’t end up doing this.
Didn’t have rest in between working. And participants ended up overworking and feeling exhausted. Pomodoros will solve.
The general vibe was that it was a fun environment and productive environment to work in.

Day 2 Wed
Walking to the lunch place was a nice break.
Again got a rec for pair coding but didn’t implement it.
We had breaks but stayed indoors. Should go outside

Day 3 Thu
I feel like we have this organic dynamic where the morning starts super slow, and then we get more and more into it as every hour passes.

Day 5 Mon
Rest breaks so much more fun. Thanks to the ball! Everyone is super happy about them.
The diverging is starting to be noticeable.
Let’s start setting intensions before every pomodoro

Thanks a bunch to the participants, other organizers at Aalto EA and EA Finland for helping out and encouraging me to do this! Special thanks to Ada, Xylix, Aleksi, and Santeri for providing feedback on this post :)

1. ^{^}

   this framing is generally unusual in Finland. Check out the "Tone" subpart in Ada-Maaria's post for some explanation of this: https://forum.effectivealtruism.org/posts/uxnpir7zLgwf3whg7/cultural-ea-considerations-for-nordic-folks