Thanks for commenting. I have just taken a look at your essay and associated entities as well as the unfairly condescending but informative comments it received.
It looks like our ideas are aligned. Some initial thoughts from me:
To make an scalable impact and safeguard or contribute favourably to the reputation of effective charities, I think our product offering should be at least as good as the current market. Drawing on success stories like Patagonia, and the rationale that consumers of premium products are more interested in brand differentiation - I think we should focus on offering premium products only, whatever the category of good.
From the sample I saw, the problem I see with CPI associated entities is they look frankly like white labelled low cost products sold online on less than slick websites - that likely will not be found unless people are really looking for them in order to donate - and those people likely already donate to effective charities - so contractually it might be marginally harmful but would be very skeptical about dollars moved to date.
I think there's a point to be made that the comparative advantage of people in our movement is effective charity rather than effective business, so I see the case for CPI. Perhaps a good change in strategy for CPI is to focus on partnering with existing companies in need of differentiation, rather than spawning businesses from within the EA ecosystem?
That being said, our movement doesn't necessarily lack the capacity for or to develop in house profit for good entrepreneurship - just as charity entrepreneurship capabilities have been built. However, from the lack of hands up to cofound in this comments section, it looks like further 'field building' is required to construct a coherent direction with this strategy. Otherwise, I expect my idea here will languish the way the other entities in your ecosystem appear to. Please do correct me if I'm mistaken - if funding has been forthcoming, committed volunteers coming forward or revenues healthy for instance
Thanks for the vote of confidence. My third last paragraph on the secondary path to impact begins talking to this.
I think further investigation is required to inform whether to ask, or to pursue other tactics to shape the behaviour of the existing market.
It's not one or the other - the existing market can be shaped as well as a competitor product introduced.
Some of the value of introducing a competitor over only shaping the behaviour of the existing market is:
that we exert continuous pressure on the current market to give effectively
we can be assured that donations will be directed to effective causes - we are not be beholden to the whims and changing pressures on those businesses
we are insulated from non-EA reputation risks and the reputations of effective charities are safeguarded by participants in the effective altruism movement more directly.
To your first para - yes I wonder how unionised countries and relevant sectors are in bottlenecks in the compute supply chain - Netherlands, Japan and Taiwan. I don't know enough about the efficacy of boycotts to comment on the union led boycotts idea.
I've raised this in response to another comment but I want to also address here the concern that workers who join a union would organise to accelerate the development of AI. I think that is very unlikely - the history of unions is a strong tradition of safety, slowing down or stopping work. I do not know an example of a union that has instead prioritised acceleration but there's probably some and it would get grey as you move into the workers self-management space.
I had explicitly considered this in drafting and whether to state that crux. If so, it could be an empirical question of whether there is greater support from the workers or management, or receptiveness to change.
I did not because I now think the question is not whether AI workers are more cautious than AI shareholders, but whether AI firms where unionised AI workers negotiate with AI shareholders would be more cautious. To answer that question, I think so
Edit: to summarise, the question is not whether unions (in isolation) would be more cautious, but whether an system of management (and policymakers) bargaining with a union would be more cautious - and yes it probably would
As you have said there are examples of individuals have left firms because they feel their company is too cautious. Conversely there are individuals who have left for companies that priorities AI safety.
If we zoom out and take the outside view, it is common for those individuals who form a union to take action to slow down or stop their work or take action to improve safety. I do not know an example of a union that has instead prioritised acceleration.
I don't think this is predicated on those assumptions.
My assumptions are:
AI workers who join a union are more likely to care about safety than AI workers who do not join a union. That is because the history of unions suggests that unions promote a culture of safety
Unionised AI workers will be more organised in influencing their workplace than non unionised AI workers. That is because of their ability to co-ordinate collectively
Therefore:
Unionisation of AI workers would encourage a culture of safety
Furthermore, these unions could be in a position to implement AI safety policies.
The pace of relevant research would slow (allowing for regulation to catch up)
Arguments against
Malevolent actor could publish or retrieve said information elsewhere than arxiv/bioRxiv/medRxiv/participating publisher. Members of the International Gene Synthesis Consortium (IGSC) voluntarily apply screening standards to assess gene sequence orders and customers, and these companies today only represent approximately 80% of global commercial gene synthesis capacity so compulsory filtering, or better coordination of the industry (which stricter self-regulation at this stage may deter) is required.
Benevolent actors would be face higher barriers to access useful information
For further investigation
Would Industry lead censorship could crowd-out or crowd-in government intervention that could enforce vetting across publishers and in other initiatives?
Would insulating the riskiest dual use publications from exposure to a broader pool of researchers accessing conventional publishers have unintended consequences?
Implementation
How would vetters, whether a regulatory agency or an independent initiative, screen papers? In the case of DNA synthesis, which does not account for all biosecurity relevant dual use research, a minimalistic approach is for vetters to utilise an encrypted database of riskier sequences of DNA , as proposed by MIT's Prof Kevin Esvelt.
However, dual use control at the publisher level would presumably not be restricted to DNA synthesis, it would include such things as studies of remote areas at the human-animal boundary.
Next steps
Esvelt is in dialogue with the Nuclear Threat Initiative who are coordinating higher level conversation in this area. If the publishers you mentioned aren't already part of that dialogue, the best next steps may be to connect Nuclear Threat Initiative folks with those academic publishers. But, I don't think that should mean this initiative shouldn't proceed in parallel. I think there is merit in taking some action now in this space because the conversation that the Nuclear Threat Initiative and co are kindling is a slow, multilateral process - screening DNA synthesis orders is not legally required by any national government at this stage.
Time sensitivity
The cost of DNA synthesis is declining and the fixed costs of filtering could grow as fraction of the cost, therefore the viability of a voluntary screening model could its highest right now.
Participation
I'm interested in being involved, but don't know that much about academic publishing or technical genomics stuff so probably not a fit to be a (solo) project lead. Do know about management, health policy, public administration, stakeholder engagement, communications etc
Thanks for commenting. I have just taken a look at your essay and associated entities as well as the unfairly condescending but informative comments it received.
It looks like our ideas are aligned. Some initial thoughts from me:
To make an scalable impact and safeguard or contribute favourably to the reputation of effective charities, I think our product offering should be at least as good as the current market. Drawing on success stories like Patagonia, and the rationale that consumers of premium products are more interested in brand differentiation - I think we should focus on offering premium products only, whatever the category of good.
From the sample I saw, the problem I see with CPI associated entities is they look frankly like white labelled low cost products sold online on less than slick websites - that likely will not be found unless people are really looking for them in order to donate - and those people likely already donate to effective charities - so contractually it might be marginally harmful but would be very skeptical about dollars moved to date.
I think there's a point to be made that the comparative advantage of people in our movement is effective charity rather than effective business, so I see the case for CPI. Perhaps a good change in strategy for CPI is to focus on partnering with existing companies in need of differentiation, rather than spawning businesses from within the EA ecosystem?
That being said, our movement doesn't necessarily lack the capacity for or to develop in house profit for good entrepreneurship - just as charity entrepreneurship capabilities have been built. However, from the lack of hands up to cofound in this comments section, it looks like further 'field building' is required to construct a coherent direction with this strategy. Otherwise, I expect my idea here will languish the way the other entities in your ecosystem appear to. Please do correct me if I'm mistaken - if funding has been forthcoming, committed volunteers coming forward or revenues healthy for instance
Pinging @Daniel_Wyrzykowski for comment
Thanks for the vote of confidence. My third last paragraph on the secondary path to impact begins talking to this.
I think further investigation is required to inform whether to ask, or to pursue other tactics to shape the behaviour of the existing market.
It's not one or the other - the existing market can be shaped as well as a competitor product introduced.
Some of the value of introducing a competitor over only shaping the behaviour of the existing market is:
Thanks, the replicated posts is a mistake and I don't know why that happened. I think they're deleted now.
To your first para - yes I wonder how unionised countries and relevant sectors are in bottlenecks in the compute supply chain - Netherlands, Japan and Taiwan. I don't know enough about the efficacy of boycotts to comment on the union led boycotts idea.
I've raised this in response to another comment but I want to also address here the concern that workers who join a union would organise to accelerate the development of AI. I think that is very unlikely - the history of unions is a strong tradition of safety, slowing down or stopping work. I do not know an example of a union that has instead prioritised acceleration but there's probably some and it would get grey as you move into the workers self-management space.
I had explicitly considered this in drafting and whether to state that crux. If so, it could be an empirical question of whether there is greater support from the workers or management, or receptiveness to change.
I did not because I now think the question is not whether AI workers are more cautious than AI shareholders, but whether AI firms where unionised AI workers negotiate with AI shareholders would be more cautious. To answer that question, I think so
Edit: to summarise, the question is not whether unions (in isolation) would be more cautious, but whether an system of management (and policymakers) bargaining with a union would be more cautious - and yes it probably would
As you have said there are examples of individuals have left firms because they feel their company is too cautious. Conversely there are individuals who have left for companies that priorities AI safety.
If we zoom out and take the outside view, it is common for those individuals who form a union to take action to slow down or stop their work or take action to improve safety. I do not know an example of a union that has instead prioritised acceleration.
I don't think this is predicated on those assumptions.
My assumptions are:
AI workers who join a union are more likely to care about safety than AI workers who do not join a union. That is because the history of unions suggests that unions promote a culture of safety
Unionised AI workers will be more organised in influencing their workplace than non unionised AI workers. That is because of their ability to co-ordinate collectively
Therefore:
Furthermore, these unions could be in a position to implement AI safety policies.
Arguments for
Arguments against
For further investigation
Implementation
How would vetters, whether a regulatory agency or an independent initiative, screen papers? In the case of DNA synthesis, which does not account for all biosecurity relevant dual use research, a minimalistic approach is for vetters to utilise an encrypted database of riskier sequences of DNA , as proposed by MIT's Prof Kevin Esvelt.
However, dual use control at the publisher level would presumably not be restricted to DNA synthesis, it would include such things as studies of remote areas at the human-animal boundary.
Next steps
Esvelt is in dialogue with the Nuclear Threat Initiative who are coordinating higher level conversation in this area. If the publishers you mentioned aren't already part of that dialogue, the best next steps may be to connect Nuclear Threat Initiative folks with those academic publishers. But, I don't think that should mean this initiative shouldn't proceed in parallel. I think there is merit in taking some action now in this space because the conversation that the Nuclear Threat Initiative and co are kindling is a slow, multilateral process - screening DNA synthesis orders is not legally required by any national government at this stage.
Time sensitivity
The cost of DNA synthesis is declining and the fixed costs of filtering could grow as fraction of the cost, therefore the viability of a voluntary screening model could its highest right now.
Participation
I'm interested in being involved, but don't know that much about academic publishing or technical genomics stuff so probably not a fit to be a (solo) project lead. Do know about management, health policy, public administration, stakeholder engagement, communications etc
Love it!