I second what John Maxwell says -- definitely one of the best things I think you could do is set up a blog and practice some Givewellian transparency where you publish as much as you can. It would be cool for us all to learn from it!
Especially if you speak candidly about learning experiences, mistakes, and failures and how you improve from them.
I currently have a google doc that I have been using to record hours, mistakes, lessons learned, and observations. I do think I should write it up as we make a blog.
Writing down problems has seemed to function in a similar way to rubber ducking though, trying to get certain problems into words can sometimes highlight a solution, and that has been useful.
For those who are interested, this is our current blog: https://eapolicy.wordpress.com/
We will try to keep it fairly updated.
This sounds awesome! Is the idea that policy comments would get regulators to: consider a new policy they weren't previously considering; change their mind about a proposed policy; help back them up politically for something they already want to do; or a combination of these? I'm not sure which type of impact policy comments are best for.
In fall 2014 the Effective Altruism Society of DC and Effective Altruism at UMD started an experimental policy project attempting to bring effective altruism ideas to public policy. We produced comments for regulations on the federal register proposed by the Department of Energy and Environmental Protection Agency. By offering independent cost-benefit analysis, we hoped to provide ammunition to any regulator trying to make policy more effective. While direct feedback from agencies is pending, the expected benefits were large enough to bring the project to Effective Altruism Ventures for assistance in conducting a full-time experiment.
On June 1, 2015, Effective Altruism Policy Analytics (EAPA) began operations, attempting to bring non-partisan, cause-neutral improvements to regulatory action in the United States. It works to determine the best practices for creating influential analysis and to turn that knowledge over to effective altruists. EAPA will offer its findings to the community and create guides and information for effective altruists interested in influencing policy. EAPA is staffed full-time by Matthew Gentzel and Emma Atlas, as well as part time by interns and a host of volunteer consultants. EAPA is still growing an even greater network of experts and volunteers interested in beneficially influencing policy. As the experiment proceeds, EAPA will gather feedback and create a foundation for the Effective Altruism movement to exert more political influence in the future.
Who we are:
Matthew Gentzel spent the first semester of his college education at the United States Air Force Academy, and then transferred to the University of Maryland to major in engineering. Matt first became interested in utilitarianism early on in college and worked on projects with Engineers Without Borders and small start-ups. Matt was introduced to Less Wrong in his junior year of college and helped found Effective Altruism at UMD in the summer before his senior year.
Emma Atlas graduated May 2015 with a Bachelor’s in Government & Politics from the University of Maryland. She worked as an intern in the Maryland General Assembly in Spring 2015 and grew an interest in creating non-partisan effective outcomes. She accepted an offer from Gentzel to help on the project, and began working on the project full-time in June.
Lawrence Roth is currently an undergraduate student majoring in Criminology/Criminal Justice with a minor in Geographic Information Systems at the University of Maryland. He has experience in programming, web design, and statistics. Roth was introduced to effective altruism by Gentzel and has attended events with the Effective Altruism Society of DC. He accepted an intern position at EAPA assisting with any research needs that Atlas and Gentzel require.
Matthew Dahlhausen is a graduate research assistant in the Mechanical Engineering Department at the University of Maryland and Cluster of Sustainability in the Built Environment (CITY). Matt completed an M.S. (2014) in architectural engineering from the Pennsylvania State University, and graduated cum laude from Dartmouth college with a B.S. (2011) in engineering sciences and B.E. (2011) in environmental engineering. Prior to starting his graduate studies, he served with AmeriCorps as a quality assurance manager for a residential energy retrofit program in Philadelphia. He is a member of the American Society of Heating, Refrigeration, and Air-Conditioning Engineers (ASHRAE).
Richard Bruns is an economist at the Food and Drug Administration. His job is to produce economic impact analyses of regulations. Richard has worked on the gluten-free labeling rule, the trans fat ban, and the rule on intentional adulteration of food supplies. Richard has extensive knowledge of the public comment process on regulations and how the FDA responds to these public comments.
Why EAPA?
Writing policy comments could be one of the most efficient ways you can create effective political influence. Federal agencies are often required to post new or proposed regulations online for the public to review and consider, and agencies must return a response to every comment posted by the public. By reviewing policies available for comment and offering analysis when positive influence can be made, you can have a greater impact than you would just by general voting.
As a group focused on facilitating the voice of effective altruists in the federal register, EAPA organizes efforts around targeting the places where we can make the biggest impact. By working and communicating with us, you could become part of the growing voice of effective altruists in government.
How you can help:
If you are an expert and interested in consulting with us, please send your contact information and area(s) of expertise to [email protected] with the subject line “EAPA Expert Contact.” If you know someone who may be interested, contact them for permission first, then add their contact information on this form.
We are currently seeking experts in:
Global Health
Foreign Aid
Urban Poverty
Housing
Criminal Justice
Policing
Juvenile Justice
Education
Immigration
Free Trade and Tariffs
Taxes
Economics
Impact of Biodiversity changes
Climate Change
Chemistry and Toxicity
We likely will be requesting more areas of expertise as we find more proposed regulations, so if you are interested but have a specialty outside those areas, feel free to send us your contact information anyway. Policy change is opportunistic, and likewise not all opportunities for cost effective change and feedback are in areas that have typically been investigated by EAs for high potential impact.
In the future, we are looking to offer more in-depth training in writing policy comments and building political influence for effectiveness. We would like to build a network of students and volunteers to examine the federal register and increase the probability that regulations are always efficient, cost-effective, and thoroughly researched.
Questions?
Go to or request to join the EA Policy Reform Facebook group and send us your questions or ideas. If you want to cooperate more closely and avoid duplication of effort, send an introduction to [email protected] and we will provide you access to our cooperative documents.
This sounds like a pretty cool project. Some thoughts:
Do we have evidence, whether quantitative or anecdotal, about the level of impact that policy comments have?
Reading about your spreadsheet reminded me of this post on sequence thinking vs cluster thinking. I think spreadsheets are inherently sequence-ish in nature; a change to a single field can completely alter the result.
As a software developer, I'm intimately familiar with the phenomenon of tiny subtle mistakes causing unexpected catastrophes... writing production software trains a certain paranoia when dealing with computers that's hard to get elsewhere. Sometimes I wonder if most scientific simulation code has subtle errors in it that invalidate the results because the researchers don't have enough programming experience to realize how easy it is to write code that's incorrect and simulations are black boxes where errors don't necessarily manifest themselves immediately & visibly.
One interesting idea might be for each person in your group to try to generate frameworks for evaluating a proposed regulation independently, and meet to compare spreadsheets later. Probably this would work even better with a group that had a diverse set of backgrounds (educationally, politically, etc.)
Because politics is apparently inherently interesting to almost everyone, you might be able to start blogging about whatever regulations you are looking in to and extract free cognitive labor from your readers to provide an additional perspective, critique your research, etc. In fact, maybe there is low-hanging fruit in just redirecting the attention of intelligent policy bloggers to policies that can actually be impacted in the near term instead of just whatever the fashionable discussion topics of the day are, then writing up any consensus they come to as a comment.
(This is a tougher one since there are also downsides to having politics discussions online, which I wrote about previously. Not sure where I currently stand, but leaning towards a blog promoting a high standard of discussion being the best option.)
Broadly my intuition is that some policy recommendations have the potential for 10x or 100x the impact relative to others, so you are better off aiming to produce a small number of high-quality, highly-targeted recommendations than a large number of lower-quality ones, at least in the long run. (Incidentally, are animal issues on your radar?)
I think we will start blogging in a limited capacity about regulations we are seriously considering working on and some that we considered and then dismissed. We probably aren't going to blog about every regulation we look at since there are so many. Some comments are likely to be far more impactful than others, however the comments that are likely to have the most impact are also likely to have slower feedback and no nearby certain deadlines for implementation.
Our current priority list seems to be: -Network early to get expert feedback and assistance -Produce lots of comments early to get feedback and learn how to make influential comments -Focus on high impact comments toward the end of our project trial
To some degree, these priorities can get jumbled by time sensitive opportunities, but as an overall aim, we think this is correct for moving forward.
Animal issues are on our radar, but I have yet to see anything lately relevant to factory farming of cows, pigs, or chickens. We have seen a lot of proposed rules about fisheries and species protection, but didn't have the expertise to go after them yet. If there are experts we could consult on animal issues, they would likely be helpful unless their way of sorting policies into "worth going after" and "not worth it" is the same as ours and nothing new comes up/is noticed.
Some of these questions require semi-detailed responses, so I will respond with a few different comments. Richard had some examples/anecdotes about the level of impact policy comments could have:
"A good recent example of FDA making major changes as a result of public comment is the Food Safety Modernization Act (FSMA) Produce Safety rule. This rule was extensively changed as a result of feedback from the public, mostly the affected farmers. Some of the comments were merely self-interested lobbying, but some pointed out where FDA's lack of understanding of specific situations would have led to an inefficient regulation. The changes were so extensive that FDA had to publish a reproposal of the rule.
This is a large and complex rule, so you can choose how much detail you want to look into. The overview is here:
http://www.fda.gov/Food/GuidanceRegulation/FSMA/ucm334114.htm
and it has links to further information. The full Federal Register notice is here:
https://www.federalregister.gov/articles/2014/09/29/2014-22447/standards-for-the-growing-harvesting-packing-and-holding-of-produce-for-human-consumption
You can do a text search in the Federal Register document for the phrase "Relevant Comments" to see the FDA responses to comments on various subjects and how those comments changed the regulation.
This is a link to the economic analysis of the reproposal, which has details on the costs and benefits of the changes in the reproposal:
http://www.fda.gov/downloads/AboutFDA/ReportsManualsForms/Reports/EconomicAnalyses/UCM442801.pdf
Another rule where public comments led to major changes and a repropsal is the FSMA Preventative Controls rule:
http://www.fda.gov/Food/GuidanceRegulation/FSMA/ucm334115.htm
However, understanding the issues in that rule requires a bit more specialized knowledge."
As for spreadsheets, we could go through a cluster thinking way of producing estimates, but I am under the impression this would take a lot more time per person, and then when comparing spreadsheets at the end, we'd be finding errors that would have been easier to handle earlier if we worked together earlier and got faster feedback.
There certainly is value to avoiding groupthink though. Overall I do think using multiple sequential techniques could be a rather rigorous way to evaluate something, and make a very good comment, but we are also trying to get useful feedback by making a lot of comments and we are already commenting much slower than we desired to because of outreach.
I would like to do this, but I think it would be useful to have more comment drafting workers if we want to do this a lot.