It's been a while since, but from what I remember, my questions were generally in the same range as the framing highlighted by user seanrson's above! I've also heard objections from people who've felt that predictions about AGI from biological anchors don't understand the biology of a brain well enough to be making calculations. Ajeya herself even caveats "Technical advisor Paul Christiano originally proposed this way of thinking about brain computation; neither he nor I have a background in neuroscience and I have not attempted to talk to neuroscientists about this. To the extent that neuroscientists who talk about “brain computation” have a specific alternative definition of this in mind, this proposal may not line up well with their way of thinking about it; this might make it more hazardous to rely as much as I do on evidence Joe gathered from discussions with neuroscientists."
This is super cool research and it’s great that you all were able to conduct this survey!!
I think it’s great that you all renamed to survey so that it didn’t specifically attract people who looked for words like “effective” or “altruism” or “existential risk.”
I wasn’t anticipating releasing the list (in some part because people may try to pander to a certain judge’s background and in some part to allow myself and the judges more flexibility in adding people last second).
Sending some judge recommendations my way would be great! I think having a variety of readers would be helpful :) Thank you!
Great point! Early on, I had someone more connected than me make a list of potential judges. We have 15 names brainstormed and sectioned off by how much they know about alignment. I can say with pretty high certainty that I imagine we will at least have someone whose full-time job is alignment reading the submissions (likely a person with a CS doctorate), but hopefully, we could get even more expertise :)
I do think it's possible that we might award more prizes retroactively if we recognize that we receive a lot of valuable submissions! Maybe an "honorable mentions" category.
Ah, I think my worry is that it feels difficult for me to find a standard to rate that actually tracks quality. If I give a couple of examples, people may feel limited to having their work look like those examples. I might say "make your distillation 1,000 words and explain two papers and I'll give you a prize" but 1,500 words on one paper might have made an optimal submission and I would have limited people's abilities. I think I find it hard to quantify a bar on writing since everyone has such different approaches. I think the real bar is something more like "the judges who know more about AI Safety than me believe that you have communicated this idea really well" and because of that it feels wrong for me to try to say "and if you do x you will definitely win something."
Hi! I’ve been thinking about this a bit more and I do think I want graduate students to be able to submit! However, since the main audience is meant to be undergraduate students, I may have to be harsher in evaluation or, more excitingly, maybe I could create a new tier for graduate students? For now I’d say feel free to submit and I’ll work out more specifics on my end and make an edit (+ reply to this) if I make official changes!
Thank you (both)!
It's been a while since, but from what I remember, my questions were generally in the same range as the framing highlighted by user seanrson's above!
I've also heard objections from people who've felt that predictions about AGI from biological anchors don't understand the biology of a brain well enough to be making calculations. Ajeya herself even caveats "Technical advisor Paul Christiano originally proposed this way of thinking about brain computation; neither he nor I have a background in neuroscience and I have not attempted to talk to neuroscientists about this. To the extent that neuroscientists who talk about “brain computation” have a specific alternative definition of this in mind, this proposal may not line up well with their way of thinking about it; this might make it more hazardous to rely as much as I do on evidence Joe gathered from discussions with neuroscientists."
Hi! Yes, I created a 6-week reading guide with discussion questions, quotes, recommended readings, and a "take action" section with more resources.
Thank you!
This is super cool research and it’s great that you all were able to conduct this survey!!
I think it’s great that you all renamed to survey so that it didn’t specifically attract people who looked for words like “effective” or “altruism” or “existential risk.”
Sure :)
I wasn’t anticipating releasing the list (in some part because people may try to pander to a certain judge’s background and in some part to allow myself and the judges more flexibility in adding people last second).
Sending some judge recommendations my way would be great! I think having a variety of readers would be helpful :) Thank you!
Great point! Early on, I had someone more connected than me make a list of potential judges. We have 15 names brainstormed and sectioned off by how much they know about alignment. I can say with pretty high certainty that I imagine we will at least have someone whose full-time job is alignment reading the submissions (likely a person with a CS doctorate), but hopefully, we could get even more expertise :)
I do think it's possible that we might award more prizes retroactively if we recognize that we receive a lot of valuable submissions! Maybe an "honorable mentions" category.
Ah, I think my worry is that it feels difficult for me to find a standard to rate that actually tracks quality. If I give a couple of examples, people may feel limited to having their work look like those examples. I might say "make your distillation 1,000 words and explain two papers and I'll give you a prize" but 1,500 words on one paper might have made an optimal submission and I would have limited people's abilities. I think I find it hard to quantify a bar on writing since everyone has such different approaches. I think the real bar is something more like "the judges who know more about AI Safety than me believe that you have communicated this idea really well" and because of that it feels wrong for me to try to say "and if you do x you will definitely win something."
Hi! I’ve been thinking about this a bit more and I do think I want graduate students to be able to submit! However, since the main audience is meant to be undergraduate students, I may have to be harsher in evaluation or, more excitingly, maybe I could create a new tier for graduate students? For now I’d say feel free to submit and I’ll work out more specifics on my end and make an edit (+ reply to this) if I make official changes!