but I think it's such a misplay from people close to EA to act as if any non-orthodox view on eugenics/crime/immigration is identical to racism.
It's probably a misplay from people close to EA to act like an article like this can't be called racist because it contains several graphs, zero instances of the n word and the author denies ownership of the racist troll account linked to his name, home town and life story and the anecdote about how he burst into tears when his genetic tests revealed some level of black ancestry. You can be deeply committed to racism if your public profile doesn't make Hanania seem like a moderate too .
Whose orthodoxy are we talking about anyway? I mean, the view that the delta between IQ scores of black people and white people is entirely down to genetics and extremely important is pretty unorthodox amongst geneticists and biologists, but it and associated views on immigration, twentieth century racism as rational response to black crime, the great evils of DEI etc are extremely orthodox on the racist right. A genuine heterodox thinker might conclude that racial disparities are unfortunately real and argue for extra DEI to correct for it, especially if they were also into the whole utilitarian EA thing. But even someone like Razib Khan who's really not well-suited to being a white supremacist trots out the same old tired Charles Murray culture war tropes.
One area I do agree with you is that it would be unfortunate if someone's vague genetic determinism got confused with the sort of person whose interest in genetic determinism manifests itself in blogging about how mad liberals are not to realise that racism is mainly down to black people's IQ, simply because they happened to end up on a panel with them at a prediction markets conference. But that's an argument in favour of generally avoiding platforming the sort of person whose research interests include proving the libs wrong about race and IQ, and especially against platforming mainly that sort of person on the topic of genetics. It's not like one needs to have spicy views on the impossibility of racial equality (and conformance with the ultraconservative right in most other views) to be interested in biohacking, markets or altruism ... if anything quite the opposite.
I think the same applies to individual views. Now yes, associating with people who are publicly racist or misogynist happens to be bad for "optics", which is likely to lead to objectively worse fundraising, recruitment or policy influence in most contexts. But also I think the views are bad and corrosive by themselves.
YC never had any sort of monopoly on the best startups though, its own terms are much better than they used to be, and its still definitely the most prestigious. The question isn't "why didn't Anthropic join YC", it's "why, if AI truly makes most developers 10x more productive and transforms the unit economics of customers, are the valuations of a cohort of prestigious "AI-enabled" startups round about their non-AI enabled 2019 peers?". There really aren't enough people founding research labs for that to be the issue
Of course, any early stage valuations graph is as much a graph of investor sentiment as anything else, but if AI was really making these companies significantly more productive, investors would have to be very bearish on AI or YC selection effect to have gone down the toilet for that not to show in the data. I presume median data looks similar?
The best counterargument is that AI also makes it easier for competitors leading to less defensible business models even if AI actually enables them to grow faster, but if that was the case investor sentiment should be much more bearish on incumbents...
Feels like most of these except the kidney are fairly easily faked or not that relevant. Community engagement and working for your own non-profit are about the first thing anyone seeking to grift will do, as well as people who are sincere (including as others have pointed out, the people who are sincerely wrong)
And Sam Bankman Fried was a vegan who apparently started off with a completely sincere desire to work for animal nonprofits. Treating these as signals and the people complaining about his behaviour and some of the things he said as noise was a bad move.
And apart from the modal return being probably ~0 (which might not be a problem if funding lots of useless stuff guarantees sufficiently useful projects get funded), it is also possible for longtermist work to be actively counterproductive, with significant uncertainty around the direction as well as the magnitude of the impact of many longtermist activities[1]
Indeed many longtermist arguments strongly imply that work of other longtermists is counterproductive (it is hard to see how Anthropic and Pause AI can both be right about frontier models)
And even you're certain you've picked the correct side of a particular risk-mitigation debate, political donations and campaigning are particularly prone to provoke responses from opponents. This is true of a lot of campaigning, but "AI is very dangerous" is more easily inadvertently motivating accelerationists to act to militarise it first than "this suffering could be stopped" is pivoted into an argument for more cages or fewer bednets
there are probably exceptions to this; it's difficult to imagine that an asteroid early warning system will hurt us, even if it turns out to be useless
As a non-vegan, if people tell me they're vegan I won't necessarily feel judged but can ensure they get a reasonable choice of food they will feel comfortable eating
Whereas if they describe themselves as a "pro-animal person" it sounds like someone who likes puppies (or if activism is implied, potentially more extreme than vegans, though of course that very much depends on how much they elaborate!). Alternatives like "I prefer plant-based food" sound like a taste preference or fad diet.
Individual people don't tend to brag about how as they've read everything that everyone else wrote, they've learned better than everybody else and can replace everybody else with better outputs at lower cost.
If they did this, I suspect they would not be popular
(Also, if we're humouring AI companies' claims that their products should be treated just like humans when it comes to "learning", we should probably question the double standard where both corporations and computer programs evade any accountability for AI generated outputs which would be considered unethical, malicious or negligent if they were the work by human employees...)
Interesting numbers (although with it being AI, I wonder if it's derived from substantive research others might have done or if they're purely arbitrary figures hallucinated to fill the gap).
A single factory making 2% of global burgers sounds like an implausibly large factory, but I don't see why it would actually need to be that big to achieve cost parity. There isn't a massive R&D cost (cf cultured meats and some alternative proteins) and most of the ingredients I'm aware of are available in bulk at relatively low cost. Of course there is also a wide disparity between burgers to achieve cost parity with and wholesale and retail prices. Being cheaper than the cheapest brand probably isn't necessary, being cheaper than the most expensive burgers brands which market themselves based on meat quality (which I think has already been achieved) probably isn't sufficient
I'm not sure naive total utility maximization [in a static framework] is the best framework to be thinking about dealing with existential risk over time.[1]
Assuming the number of risks and error bars are not trivially small, the universal outcome of concentrating all your risk mitigations on one is that most risks continue to be a high as they could possibly be. The modal outcome is that the risks ignored includes at least one risk greater than the one all efforts are concentrated on mitigating. Some reasonable assumptions in the article above show this can hold even where the actual biggest risk is orders of magnitude greater than the one targeted. In the diversified approach, less money are devoted to reducing the perceived biggest risk, but the rest is apportioned to reducing other risks. This seems more robust to conventional assumptions like uncertainty and some risks being easier to mitigate than others.
And tbh I'm not even seeing an average utility boost from concentrating on the single largest risk as opposed to mitigating lots of risks without ancillary assumptions like increasing returns to risk reduction expenditure or the actual value of many risks under consideration being 0.
It would be interested to see a more detailed and systematicreport on the activity and findings so far.
In some respects, it seems like a strange thing for GiveDirectly to be piloting. On the one hand, GiveDirectly has expertise in systematic studies of behavioural change in LDCs , and the chatbot possibly also performed programmatic functions in a cost effective manner. On the other hand it involves a charity known for its "let local people decide how to use money spent on their behalf, Western aid agencies doing it can be disempowering and often wrong" ethos asking "which parameters should we use to fine tune this [adaptation of a commercial] product we've designed to give them the most suitable answers before scaling up its deployment"... which seems like a very different ethos and approach.[1]
The conclusions highlighted from the research so far - both that if you give poor Rwandans access to ChatGPT they have a similar range of interaction to other humans[2] and that responses generated by an LLM with no meaningful local training dataset were often inadequate - seem unsurprising. I am sympathetic to arguments that people make better decisions with access to information, but I am also sympathetic to arguments a ChatGPT derivative is not the most valuable information Rwandans could receive (and may have minimal or even negative value)
I'm not actually sure what the costs of acquiring relevant local data and training a chatbot to achieve greater fluency in spoken Kinyarwada dialects and safeguarding against advice that is very bad in a local context are,[3] but they seem like a pretty relevant benchmark, since they might actually be considerable on a per user basis and the alternative for critical information like "what is the nearest health centre" might be something like signing people up to email lists, or a small number of human agents in Kigali costing surprisingly little.[4] I guess there's also the "who's paying?" question, especially when the current implementation appears to involve providing training data for one of the world's most valuable companies (and obscure languages may or may not add value to their model).
I feel one relevant benchmark for GiveDirectly specifically might be "what is the estimated cost per per person reached to improve it: would locals rather have a better chatbot or the cash?". It's possible the insights they're getting are extremely valuable particularly in the context of limited/no of web access, but it's possible they're not...
the relevant comparator might be the One Laptop Per Child project. Well intentioned, theory of change centred on the idea that people in LEDCs can be empowered by interacting with modern technology and better information too, but perhaps actual educational benefits didn't really stack up with the costs and the participants would have chosen to have something other than a computer
I must admit, I am curious about the extent to which Rwandans engaged in "witty banter" or attempts to manipulate the chatbot into saying something silly...
I don't know how bad the speaking and dataset is, and whether an adequate "solution" looks like a finetuning prompt with some info or developing a corpus of services data and synthetic idiosyncratic Kinyarwada to fix the model, but the latter option could be very expensive compared with the people it would actually reach...
It's probably a misplay from people close to EA to act like an article like this can't be called racist because it contains several graphs, zero instances of the n word and the author denies ownership of the racist troll account linked to his name, home town and life story and the anecdote about how he burst into tears when his genetic tests revealed some level of black ancestry. You can be deeply committed to racism if your public profile doesn't make Hanania seem like a moderate too .
Whose orthodoxy are we talking about anyway? I mean, the view that the delta between IQ scores of black people and white people is entirely down to genetics and extremely important is pretty unorthodox amongst geneticists and biologists, but it and associated views on immigration, twentieth century racism as rational response to black crime, the great evils of DEI etc are extremely orthodox on the racist right. A genuine heterodox thinker might conclude that racial disparities are unfortunately real and argue for extra DEI to correct for it, especially if they were also into the whole utilitarian EA thing. But even someone like Razib Khan who's really not well-suited to being a white supremacist trots out the same old tired Charles Murray culture war tropes.
One area I do agree with you is that it would be unfortunate if someone's vague genetic determinism got confused with the sort of person whose interest in genetic determinism manifests itself in blogging about how mad liberals are not to realise that racism is mainly down to black people's IQ, simply because they happened to end up on a panel with them at a prediction markets conference. But that's an argument in favour of generally avoiding platforming the sort of person whose research interests include proving the libs wrong about race and IQ, and especially against platforming mainly that sort of person on the topic of genetics. It's not like one needs to have spicy views on the impossibility of racial equality (and conformance with the ultraconservative right in most other views) to be interested in biohacking, markets or altruism ... if anything quite the opposite.
I think the same applies to individual views. Now yes, associating with people who are publicly racist or misogynist happens to be bad for "optics", which is likely to lead to objectively worse fundraising, recruitment or policy influence in most contexts. But also I think the views are bad and corrosive by themselves.
YC never had any sort of monopoly on the best startups though, its own terms are much better than they used to be, and its still definitely the most prestigious. The question isn't "why didn't Anthropic join YC", it's "why, if AI truly makes most developers 10x more productive and transforms the unit economics of customers, are the valuations of a cohort of prestigious "AI-enabled" startups round about their non-AI enabled 2019 peers?". There really aren't enough people founding research labs for that to be the issue
Of course, any early stage valuations graph is as much a graph of investor sentiment as anything else, but if AI was really making these companies significantly more productive, investors would have to be very bearish on AI or YC selection effect to have gone down the toilet for that not to show in the data. I presume median data looks similar?
The best counterargument is that AI also makes it easier for competitors leading to less defensible business models even if AI actually enables them to grow faster, but if that was the case investor sentiment should be much more bearish on incumbents...
Feels like most of these except the kidney are fairly easily faked or not that relevant. Community engagement and working for your own non-profit are about the first thing anyone seeking to grift will do, as well as people who are sincere (including as others have pointed out, the people who are sincerely wrong)
And Sam Bankman Fried was a vegan who apparently started off with a completely sincere desire to work for animal nonprofits. Treating these as signals and the people complaining about his behaviour and some of the things he said as noise was a bad move.
And apart from the modal return being probably ~0 (which might not be a problem if funding lots of useless stuff guarantees sufficiently useful projects get funded), it is also possible for longtermist work to be actively counterproductive, with significant uncertainty around the direction as well as the magnitude of the impact of many longtermist activities[1]
Indeed many longtermist arguments strongly imply that work of other longtermists is counterproductive (it is hard to see how Anthropic and Pause AI can both be right about frontier models)
And even you're certain you've picked the correct side of a particular risk-mitigation debate, political donations and campaigning are particularly prone to provoke responses from opponents. This is true of a lot of campaigning, but "AI is very dangerous" is more easily inadvertently motivating accelerationists to act to militarise it first than "this suffering could be stopped" is pivoted into an argument for more cages or fewer bednets
there are probably exceptions to this; it's difficult to imagine that an asteroid early warning system will hurt us, even if it turns out to be useless
As a non-vegan, if people tell me they're vegan I won't necessarily feel judged but can ensure they get a reasonable choice of food they will feel comfortable eating
Whereas if they describe themselves as a "pro-animal person" it sounds like someone who likes puppies (or if activism is implied, potentially more extreme than vegans, though of course that very much depends on how much they elaborate!). Alternatives like "I prefer plant-based food" sound like a taste preference or fad diet.
Individual people don't tend to brag about how as they've read everything that everyone else wrote, they've learned better than everybody else and can replace everybody else with better outputs at lower cost.
If they did this, I suspect they would not be popular
(Also, if we're humouring AI companies' claims that their products should be treated just like humans when it comes to "learning", we should probably question the double standard where both corporations and computer programs evade any accountability for AI generated outputs which would be considered unethical, malicious or negligent if they were the work by human employees...)
Interesting numbers (although with it being AI, I wonder if it's derived from substantive research others might have done or if they're purely arbitrary figures hallucinated to fill the gap).
A single factory making 2% of global burgers sounds like an implausibly large factory, but I don't see why it would actually need to be that big to achieve cost parity. There isn't a massive R&D cost (cf cultured meats and some alternative proteins) and most of the ingredients I'm aware of are available in bulk at relatively low cost. Of course there is also a wide disparity between burgers to achieve cost parity with and wholesale and retail prices. Being cheaper than the cheapest brand probably isn't necessary, being cheaper than the most expensive burgers brands which market themselves based on meat quality (which I think has already been achieved) probably isn't sufficient
I'm not sure naive total utility maximization [in a static framework] is the best framework to be thinking about dealing with existential risk over time.[1]
Assuming the number of risks and error bars are not trivially small, the universal outcome of concentrating all your risk mitigations on one is that most risks continue to be a high as they could possibly be. The modal outcome is that the risks ignored includes at least one risk greater than the one all efforts are concentrated on mitigating. Some reasonable assumptions in the article above show this can hold even where the actual biggest risk is orders of magnitude greater than the one targeted. In the diversified approach, less money are devoted to reducing the perceived biggest risk, but the rest is apportioned to reducing other risks. This seems more robust to conventional assumptions like uncertainty and some risks being easier to mitigate than others.
And tbh I'm not even seeing an average utility boost from concentrating on the single largest risk as opposed to mitigating lots of risks without ancillary assumptions like increasing returns to risk reduction expenditure or the actual value of many risks under consideration being 0.
It would be interested to see a more detailed and systematic report on the activity and findings so far.
In some respects, it seems like a strange thing for GiveDirectly to be piloting. On the one hand, GiveDirectly has expertise in systematic studies of behavioural change in LDCs , and the chatbot possibly also performed programmatic functions in a cost effective manner. On the other hand it involves a charity known for its "let local people decide how to use money spent on their behalf, Western aid agencies doing it can be disempowering and often wrong" ethos asking "which parameters should we use to fine tune this [adaptation of a commercial] product we've designed to give them the most suitable answers before scaling up its deployment"... which seems like a very different ethos and approach.[1]
The conclusions highlighted from the research so far - both that if you give poor Rwandans access to ChatGPT they have a similar range of interaction to other humans[2] and that responses generated by an LLM with no meaningful local training dataset were often inadequate - seem unsurprising. I am sympathetic to arguments that people make better decisions with access to information, but I am also sympathetic to arguments a ChatGPT derivative is not the most valuable information Rwandans could receive (and may have minimal or even negative value)
I'm not actually sure what the costs of acquiring relevant local data and training a chatbot to achieve greater fluency in spoken Kinyarwada dialects and safeguarding against advice that is very bad in a local context are,[3] but they seem like a pretty relevant benchmark, since they might actually be considerable on a per user basis and the alternative for critical information like "what is the nearest health centre" might be something like signing people up to email lists, or a small number of human agents in Kigali costing surprisingly little.[4] I guess there's also the "who's paying?" question, especially when the current implementation appears to involve providing training data for one of the world's most valuable companies (and obscure languages may or may not add value to their model).
I feel one relevant benchmark for GiveDirectly specifically might be "what is the estimated cost per per person reached to improve it: would locals rather have a better chatbot or the cash?". It's possible the insights they're getting are extremely valuable particularly in the context of limited/no of web access, but it's possible they're not...
the relevant comparator might be the One Laptop Per Child project. Well intentioned, theory of change centred on the idea that people in LEDCs can be empowered by interacting with modern technology and better information too, but perhaps actual educational benefits didn't really stack up with the costs and the participants would have chosen to have something other than a computer
I must admit, I am curious about the extent to which Rwandans engaged in "witty banter" or attempts to manipulate the chatbot into saying something silly...
I don't know how bad the speaking and dataset is, and whether an adequate "solution" looks like a finetuning prompt with some info or developing a corpus of services data and synthetic idiosyncratic Kinyarwada to fix the model, but the latter option could be very expensive compared with the people it would actually reach...
I suspect you get many person years of Rwandan human call centre time for a month or two of a mid-level AI engineer's time...