I think a week from today is a good amount of review time, but I think the key thing is to be clearer in setting expectations in the future.

(I'd also recommend, as I wrote above, apologizing to Sinergia for misunderstanding their earlier request and giving them a very short time to review.)

In the future the most reliable way to redact a screenshot is:

• Open the screenshot and add the black bars
• Save the edited screenshot as PNG

Since PNG doesn't support layers or history, the redacted information is reliably no longer present.

Uh oh! The emails you shared do include the full text, but with black bars added on top in Google Slides. This doesn't actually remove the information, and is a mistake people sometimes make when trying to redact things. Because I used an LLM to transcribe the emails, which used the underlying images, I ended up with the unredacted text. I've now gone through my comment above and put [redacted] anywhere I see a black bar, but you should be aware that the information is still in your slides for anyone to extract.

You're taking their wording too literally. You wrote asking them if 24hr was enough notice, and this response was Sinergia asking if they could instead tell you how long they'd need after reviewing the draft. You don't have to accept that — you could say you're not willing to hold off on publishing for more than a week or something — but when you didn't respond to it Sinergia was right to expect that you wouldn't drop a draft on them with 24hr notice before publication.

@VettedCauses DM'd me asking if I could look at this situation (I've been pushing a norm of running things by orgs before publishing while also sometimes talking about downsides).  I read the emails that VettedCauses shared below.

Overall, it seems to me that Sinergia is behaving pretty reasonably in this exchange, and Vetted Causes is not being clear enough in setting expectations about timelines and what opportunities Sinergia would have to correct things before publication.  I especially think the "After Charity Didn't Object" in the title isn't right.

Under the circumstances, if I were VettedCauses I would write something like:

After discussing our emails on the EA Forum I'd like to apologize for misunderstanding your requests around timing.  We overlooked your request on deadlines in your message of the 27th, and we're very sorry!  Because we missed that, we had thought you didn't have issues with a 24 hour review period.  While we can't in commit to unbounded review periods, we're happy to have some discussion on a case-by-case basis, considering how much we're asking for you to review.

With that in mind, would you like more time to prepare a public response before we publish the response we'd been targeting for today?  For example, would Tuesday the 15th work?

Emails, with thoughts interspersed:

From: The Vetted Causes Team
To: [redacted], carolina
Date: Wed, Mar 26, 3:25 PM

Hello,

We saw your responses to our Sinergia Review, and wanting to respect your wishes, have decided to follow up with you in private before issuing a response.

We think there are some misunderstandings regarding our intentions, and would like to clear them up.

Our goal is to help animals by directing donations towards effective causes. In the past, our team has contributed tens of thousands of dollars in donations to ACE's cause, and has explored ways to get more funding to ACE recommended charities. Given that some of these donations have been to ACE's recommended charity fund, we would guess some of our contributions have even gone to Sinergia.

The text message below was sent by Adam Hebert (founder of Vetted Causes) to a current Vetted Causes teammate in 2020:

[screenshot of 2020-12-01 text message conversation showing three attached images without thumbnails, a link to Facebook Social Impact's "Your Guide to Giving Season", and a message "3 of ACE's top recommended charities took part in this event. I wonder if we'd be able to find events like these and match everyone's donations"]

We hope this makes it clear that Vetted Causes is motivated by a genuine desire to help animals. We'd also like to note that later this year, we plan to publish positive reviews about some of the charities ACE has endorsed.

We do plan to eventually release full responses to ACE and Sinergia's responses. In order to respect ACE and Sinergia's wishes, we will show this response to both of you at least 24 hours before publishing (or more if you would prefer). Additionally, we will send clarifying questions to both Sinergia and ACE prior to writing the responses in order to ensure there are no misunderstandings.

If Sinergia or ACE would like, we'd also be happy to have a phone call to clear anything else up. Please let us know if you have any questions.

Best,

The Vetted Causes Team

Basically reasonable, though "We do plan to eventually release full responses to ACE and Sinergia's responses. In order to respect ACE and Sinergia's wishes, we will show this response to both of you at least 24 hours before publishing (or more if you would prefer)." shows less understanding of how organizations work than I'd like to see from a charity evaluator, even a volunteer-run group.  Better would be (a) default to an amount of time where the organization could plausibly review and respond without dropping everything, probably a week, (b) give more indication about when this might come than "eventually", and (c) clarify up front whether Vetted Causes is willing to engage in back-and-forth to improve the draft or whether they're just giving Sinergia the ability to prepare a public response.

From: Carolina Galvani
To: [redacted]
Date: Thu, Mar 27, 3:29 PM

Dear Adam and the Vetted Causes Team,

Thank you for reaching out and for your willingness to engage in further dialogue before publishing your posts and comments. We appreciate your consideration of our concerns, as well as those raised by the broader community regarding the importance of sharing the drafts of your posts in advance.

I am available to meet next week, and I may invite colleagues who have been involved in responding to your claims. You can schedule a meeting through the following link: https://calendly.com/carolina-galvani. We believe it is crucial to clarify your intentions regarding Sinergia and to speak about our request, according to the Forum's norms, that you refrain from implying we act in bad faith.

To ensure a transparent and constructive process, we ask that you share the drafts of your publications about Sinergia Animal with us. After reviewing them, we will inform you of the time required to provide our response and any additional information, considering our internal priorities and the necessary level of analysis. We hope you understand that the manner in which this process has unfolded has disrupted our work and may have affected our reputation. Furthermore, members of the EA Forum have expressed a preference for this approach too, which we believe would contribute to saving everyone's time and a more informed discussion. Our goal is to minimize unnecessary miscommunications, disruptions, and potential harm moving forward.

Additionally, we seek clarification regarding your stated motivation for creating Vetted Causes—to direct donations to more effective causes. In the context of effective altruism, "more effective causes" typically refers to broad focus areas such as AI, climate change, existential risks, pandemic preparedness, and global health. Initially, we understood this to mean that your aim was to divert funding from animal welfare to these other areas. Could you please clarify your position?

We look forward to an open and constructive conversation.

Best regards,

Carolina

Also basically reasonable.  In an ideal world Sinergia would be less prickly about whether VettedCauses is trying to redirect money between broad cause areas.  Note that Sinergia is giving a counterproposal, requesting that the process be that first Vetted Causes shares drafts and then Sinergia says how long they will need.  I don't think Vetted Causes has to agree to this, but unless if they didn't want to do it this way it would have been good to say something.

From: The Vetted Causes Team
To: adam, carolina, [redacted]
Date: Thu, Mar 27, 10:59 PM

Dear Carolina [redacted],

Thank you both for your kind replies. We hope there are no hard feelings, and look forward to resolving any misunderstandings.

I've scheduled a call with Carolina for next Friday via Calendly, and I look forward to speaking with you.

Carolina, regarding your question: Vetted Causes does not aim to divert funding from animal welfare to other cause areas. All of the charities we plan to recommend this year are animal-focused, and so far this year, we've donated over $1,000 to animal charities and $0 to the other areas you mentioned.

Best,
Adam

This would have been a good time to mention the desire to record the call.  This isn't a default in our society, and if Vetted Causes has a policy of requiring recordings that would be good to include up front.

Clarification on diversion is helpful.

From: Carolina Galvani
To: me, [redacted]
Date: Fri, Mar 28, 10:44 AM (13 days ago)

Hi Adam,

Perhaps instead of saying, "My sole motivation behind creating Vetted Causes is to direct donations to more effective causes," you could say, "My sole motivation behind creating Vetted Causes is to direct donations to more effective animal charities." Just a small suggestion for consideration.

Looking forward to our conversation on Friday.

Best wishes,

I guess it's fine for Sinergia to suggest this, though it would be better for the focus to stay on the review process.

From: Adam Hebert
To: Carolina, [redacted]
Date: Wed, Apr 2, 2:50 AM (8 days ago)

Hi Carolina,

I hope this message finds you well. My teammate [redacted] would like to be part of our meeting this Friday, as he worked on the Sinergia review. If every email address needs to be approved to join the meeting, please approve both [Adam's Email] and [[redacted]'s Email] to join the meeting.

As a standard policy, Vetted Causes records all meetings with charities. Please have everyone from your team who will participate in the meeting (including yourself) send [Adam's Email] an email stating:

I consent to being recorded during the meeting between Sinergia and Vetted Causes on April 4, 2025. I understand that Vetted Causes will record every moment of this meeting from the moment a member of Vetted Causes joins the meeting to the moment every member of Vetted Causes has left the meeting, and I consent to this recording.

Unfortunately, if anyone joins the meeting under a different name or without submitting this written consent in advance, we will have to end the meeting immediately.

We look forward to speaking with you and your team!

Best, Adam Hebert

This is a weird message.  It feels very aggressive and unprofessional, and it's not taking into consideration that it's asking for something unusual.  It think it would have worked a lot better as something like:

I'm sorry I forgot to mention this earlier, but Vetted Causes has a policy that all meetings we have with charities will be recorded.  We use these recordings to ensure our notes are accurate, and we never share them outside Vetted Causes without the permission of the charity.  We're also happy for you to record, as long as you similarly agree not to share outside your organization without our permission.

If this is ok with you, can you ensure everyone joining the meeting from your side is aware that we'll be recording?

And then instead of asking for statements over email, doing the conventional thing of (a) confirming that Vetted Causes has consent to record before turning on recording and then (b) repeating that recording is happening after turning it on, so that it's captured and it's clear that anyone remaining in the call is aware they're being recorded.

(I'm undecided on whether it's reasonable for VettedCauses to have an all-charity-meetings-recorded policy)

From: Carolina Galvani
To: me, [redacted]
Date: Wed, Apr 2, 4:07 PM (8 days ago)

Hi Adam and [redacted],

I hope you are well.

In this case, let's keep written communications only.

I will cancel the meeting.

Best wishes, Carolina

Very reasonable response!

From: Adam Hebert
To: Carolina, [redacted]
Date: Thu, Apr 3, 11:11 AM (7 days ago)

Hi Carolina,

No worries! Feel free to let me know if you change your mind. We also plan on starting a podcast soon, and you'd be welcome to come on as a guest.

I've attached a couple of questions we'd like to clarify for our response to Sinergia. We'll also be sure to send our response to you prior to publishing it.

Thanks for your help!

Best, Adam Hebert

If someone has just said they don't want to be recorded, it's pretty weird to immediately respond by inviting them on a podcast. 

More clarity on "send our response to you prior to publishing it" would be better.  Especially since Sinergia had requested a structure earlier ("after reviewing them, we will inform you of the time required to provide our response") and Vetted Causes hasn't indicated they have a problem with that.

From: Adam Hebert
To: Carolina, [redacted]
Date: Fri, Apr 4, 3:19 PM (6 days ago)

Hi Carolina,

We wanted to let you know that we plan to post our response next Thursday (April 10th). Once you and ACE have sent us your answers to the questions we sent (two different questions were also sent to ACE), we will complete our response and send it to you within 24 hours for your review.

We want to incorporate your clarifications into our response to avoid any misunderstandings. We also want to post our response soon since people are likely wondering why we haven't responded yet.

Thanks for your understanding!

Best, Adam Hebert

Giving Sinergia a timeline on publishing is good, but the information on when they'll be recieving the draft for comments is pretty messy.  For example, setting aside the timing of Sinergia's responses, it sounds like if ACE doesn't give a response until, say, Wednesday April 9th then Sinergia might not get to review before publication.  And if ACE's response comes in on the 8th (as Vetted Causes says it did) then Sinergia only has a few hours to review before publication.

Ideally Sinergia would have pushed back on this timing.

From: The Vetted Causes Team
To: adam, [redacted]
Date: Wed, Apr 9, 6:34 AM (1 day ago)

Hi everyone,

Attached [here] is a Google doc version of our response to Sinergia and ACE's responses, which we plan to publish tomorrow (April 10th).

Please note that this Google doc does not have most of our citations, since we add citations at the time we convert our articles to TypeScript to be posted on our website. However, most our citations will be to statements from ourselves/Sinergia/ACE and Sinergia/ACE's spreadsheets. Additionally, if you want to know what the citation is for any specific statement, you are welcome to send us an email and we will get back to you as soon as possible.

We would also like to note that we may make minor edits prior to publication, but we do not plan on making any major changes (we will be sure to notify you if we make one). Further, as noted in the introduction of our response, we will have a separate version of the article that doesn't use any summaries of what Sinergia said. This version will have close to the same text, but with exactly what Sinergia said in place of summaries (we are planning to just use pictures of Sinergia's response).

Please let us know if you have any questions or concerns. Thanks!

Best, The Vetted Causes Team

As above, I think 24hr is way too short to be practical.  Sinergia did push back on this originally, but I think in a way that VettedCauses didn't understand was pushback ("after reviewing them, we will inform you of the time required to provide our response").

While it's good that Vetted Causes is now making it clear that they're not planning on updating their article in response to issues Sinergia raises, it would have been much better (as I wrote above) to set these expectations from the beginning if that was indeed VettedCauses' plan.

Sending without citations makes this way more work for Sinergia, since I expect a major thing they want to check is that they're not being misquoted or having quotes taken out of context.  While this doesn't make sharing the draft with Sinergia useless, it does remove a lot of the value.  Additionally "we will get back to you as soon as possible" is pretty weak in a context of "we're publishing tomorrow".  While I think fixing the VettedCauses process to allow sharing cited drafts would be the way to go, saying something like "within 1hr if you request between 9am and 5pm" would help indicate that VettedCauses is committed to doing their part to make this short deadline possible (setting aside that I disagree the urgency is necessary or helpful).

From: Carolina Galvani
To: me, [redacted]
Date: Wed, Apr 9, 2:35 PM (16 hours ago)

Hi Adam,

We'll be responding by the end of today. That said, we do plan to share in the forum our disappointment that our request to check on deadlines was not accepted. We are likely not to respond to further requests if they are done in the same manner.

We hope for a more constructive and respectful interaction moving forward.

Best regards, Carolina

Pretty reasonable!

From: Adam Hebert
To: Carolina, [redacted]
Date: Wed, Apr 9, 6:40 PM (11 hours ago)

Hi Carolina,

Thank you for your message. We're sorry to hear that, but we don't believe we received a request to check deadlines.

For transparency to the public, would it be alright with you if Vetted Causes shared pictures of the emails that we've sent to each other so far? This way, there is no misrepresentation of what happened. If so, Sinergia is welcome to do the same.

Also, we'd like to let you know that we'll be making a post on the forum asking for guidance on how to handle this situation. We hope you understand this is our first time sending a charity an article prior to publication, and we are doing our best to navigate it thoughtfully.

Thanks!

Best, Adam Hebert

Yes, it does sound like VettedCauses missed the significance of the "after reviewing them, we will inform you of the time required to provide our response".

I agree 1% high, and I wish it were lower. On the other hand, we're specifically targeting stealth pathogens: ones where any distinctive symptoms come well after someone becomes contagious. Absent a monitoring system, you could be in a situation most people had been infected before anyone noticed there was something spreading. Flagging this sort of pathogen at 1% cumulative incidence still gives some time for rapid mitigations, though it's definitely too late to nip it in the bud.

It helps that I'm writing about stuff we've discussed internally a lot! Thanks for the good questions!

I still feel confused how big a sewershed is relative to a city

A sewershed can vary dramatically in size: it's the area that drains to some collection point (generally a treatment plant) and different cities are laid out differently. I'm most familiar with Boston (after refreshing the MWRA Biobot Tracker intently during COVID-19) and here the main plant serves ~2M people divided between the North and South systems:

Some other cities have much smaller plants (and so smaller sewersheds), a few have larger ones.

We're not sure yet about the effect of size.  It's possible that small ones are better because the waste is 'fresher' and you spend fewer of your observations (sequencing reads) on bacteria that replicates in the sewer.  Or it's possible that larger ones are better because they can support more observations (deeper sequencing).

if I read footnote 2 right, the implication is that by end of 2025, you'd aim to be able to detect a pathogen that sheds like Influenza A in cities you monitor before 2% of the population is infected?

Yes, that's right. Though sensitivity in practice could be higher or lower:

• As we gather more data we'll get a better understanding of how easy or hard it is to detect Influenza A, along with other pathogens. Our influenza estimates are based on ~300 observations, but we now have the data to estimate for the 2024-2025 flu season with a lot more data. This is mostly a matter of someone taking the time to dig into it and put out an updated estimate.

• We're still trying to increase sensitivity:

  • Testing better wet lab methods
  • Getting pooled airplane lavatory samples again, which have a ~20x higher human contribution
  • Figuring out which municipal sewersheds have the highest human contribution and focusing there

• The projection is based on an assumption of 9d end to end time, and is relatively sensitive to timing: if your pathogen doubles every 3d then the difference between a 9d and 12d turnaround time is 2x sensitivity. We're currently well above 9d, but we're on a track to get to ~7d via agreements with sequencing machine operators to reserve capacity and streamlining our processes. And then there are more expensive ways to get down to ~4d with serious investment in logistics (buy your own sequencer, run it daily, use the 10B flow cell for faster turnarounds, lab runs around the clock).

Which cities are you monitoring again?

Chicago IL, Riverside CA, and several others we hope to be able to name publicly soon.

I assume one weakness of this approach is in the geographic restrictions. Although I've vaguely heard of wastewater monitoring in a network of airports / aircrafts as a way to get around this (I can't tell if that's just an idea right now or if it's already being implemented, though.)

Yes, that's a real issue. Cosmopolitan US cities are not terrible from this perspective, especially if you have a bunch with different international connections, but they're still not good enough. Airplane lavatory sampling would be much better, not just because of this issue but also because (as I mentioned briefly above) they're much higher quality samples. We're working on this, but it's much more difficult than bringing on municipal treatment plant partners.

Was the 2% threshold chosen for a particular reason?

No, it's that 3x 25B is about the most we're able to scale to at this stage. If we thought we could manage the scale 1% would have probably been our target, though 1% is still pretty arbitrary. Lower is better, since that means mitigations are more effective when deployed, but cost goes up dramatically as you lower your target.

Not a silly question, and not something where I think we've talked about plans publicly yet. Some sort of red-teaming is something I'd like to see us do in the second half of 2025. Most likely starting with fully computational spike-ins (much cheaper, faster to iterate on) and then real engineered viral particles.