MathiasKB's Quick takes

I've noticed a decrease in the quality and accuracy of communication among people and organizations advocating for pro-safety views in the AI policy space. More often than not, I'm seeing people go with the least charitable interpretations of various claims made by AI leaders.

Arguments are increasingly looking like soldiers to me.

Take the following twitter thread from Dr. Peter S. Clark describing his new paper co-authored with Max Tegmark.

The authors use game theory to justify a slew of normative claims that don't follow. The choice of language makes refutations difficult and pollutes epistemic commons. For example, they choose terms such as 'pro-human' and include parameters such as 'naivety'.

These are rhetorical sleights of hand. Arguing for the benefits from automation to the consumer is now anti-human! You wouldn't want to be naive and anti-human now, would you?

I don't want this to be an attack on those who are against further AI development. PauseAI is a great example of what open and honest advocacy can look like. Being a vocal advocate for a cause is fine, disguising opinion as fact is not!

Anyone here who is planning to donate £5000-10000 to registered UK charities before end of year?

If so, I would like to facilitate a donor swap to animal welfare charities:
- Asia Accountability Initiative
- Screwworm Free Future

(ie. I give to the UK registered charity you were planning on donating to, and you donate to my preferred charity instead).

For inspiration, both Givewell and Giving What We Can are UK tax deductible, so anyone making donations to those could in principle do a swap.

I can highly recommend following Sentinel's weekly minutes, a weekly update from superforecasters on the likelihood of any events which plausibly could cause worldwide catastrophe.

Perhaps the weekly newsletter I look the most forward to at this point. Read previous issues here: 

https://sentinel-team.org/blog/

Why you should buy a desk treadmill

I've wanted to do a 'things I recommend you buy' list for a while, but I think my purchase of a desk-treadmill has been so much higher value than any other purchase I've made that I instead will make the case for only this item.

I've never liked standing desks. When I stand my feet get restless to the point of it being so distracting that I have to sit back down if I want to focus. Half a year ago, I got to try a desk treadmill and I immediately bought one for myself.

Since February I've been walking on this desk treadmill while working. I went from getting an average of <3000 steps a day, to an average of 17,000 daily steps (my day record is 46,000). It's been the single easiest improvement to my health I've made, and I highly recommend you get one as well.

I was fortunate to try it out before buying, but my biggest initial worries were:

• Whether I would actually use it, or it would just sit and collect dust
• Whether it would affect my productivity

Since buying it my home office desk has been permanently raised and I haven't used my office chair since. If I'm working or playing video games, I'm walking. When my feet need to rest, I take my laptop to the kitchen.

I also found no negative effects on my productivity. Unlike desk exercise bikes where people tell me they forget to pedal effectively making it an uncomfortable chair, with a desk treadmill my brain just goes 'oh I'm walking now' and then I entirely stop thinking about it until I'm out of flow state and I notice I've walked 5000 steps and could use a quick break.

I pushed my brother into buying one, and he's been very happy with his as well.

tips and tricks:

• I put mine on a homemade tennis-ball platform, to avoid my downstairs neighbour hearing my footsteps.
• You can't hear the engine through walls or while wearing noise-canceling headphones, but I suspect it would be too loud for an open office.
• I would get a bit dizzy after stepping off the treadmill the first week, but this quickly went away
• If you walk all day, you will be really sore after a few days. This doesn't happen to me anymore, but i'm also 25 so your experience may differ.
• I switch from running shoes to sandals once a day to avoid my feet blistering.

Why are seitan products so expensive?

I strongly believe in the price, taste, convenience hypothesis. If/when non-animal foods are cheaper and tastier, I expect the west to undergo a moral cascade where factory farming in a very short timespan will go from being common place to illegal. I know that in the animal welfare space, this view point is often considered naive, but I remain convinced its true.

My mother buys the expensive vegan mayonnaise because it's much tastier than the regular mayonnaise. I still eat dairy and eggs because the vegan alternatives suck.

What I don't understand is why vegan alternatives have proven so difficult to make cheap and tasty. Are there any good write ups on this?

Like when I go to a supermarket in Copenhagen, every seitan product will charge a significant markup over the raw cost of the ingredients (Amazon will sell you kilos of seitan flour at very little cost).

Do consumers have sufficiently inelastic preferences that a small market high-markup is the most profitable strategy? Is the final market too small for producers to reach economies of scale for seitan, or is it just difficult to bootstrap?

I would love to better understand what the demand curves look like for various categories of vegan products, as I really can't wrap my mind around how the current equilibrium came about

Why is it that I must return from 100% of EAGs with either covid or a cold?

Perhaps my immune system just sucks or it's impossible to avoid due to asymptomatic cases, but in case it's not: If you get a cold before an EAG(x), stay home!

For those who do this already, thank you! 

Excerpt from the most recent update from the ALERT team:

Highly pathogenic avian influenza (HPAI) H5N1: What a week! The news, data, and analyses are coming in fast and furious.

Overall, ALERT team members feel that the risk of an H5N1 pandemic emerging over the coming decade is increasing. Team members estimate that the chance that the WHO will declare a Public Health Emergency of International Concern (PHEIC) within 1 year from now because of an H5N1 virus, in whole or in part, is 0.9% (range 0.5%-1.3%). The team sees the chance going up substantially over the next decade, with the 5-year chance at 13% (range 10%-15%) and the 10-year chance increasing to 25% (range 20%-30%).

their estimated 10 year risk is a lot higher than I would have anticipated.

There has been a tremendous amount of discussion and conflict in the past months over the state of Effective Altruism as a movement. For good reason too. SBF, someone I was once proud to highlight as a shining example of what EA had to bring, looks to have have committed one of history's largest instances of fraud. I would be concerned if we weren't in heated debates over what lessons to take from this!

After spending a few too many hours (this week has not been my most productive) reading through much of the debate, I noticed something: I'm still proud to call myself an effective altruist and my excitement is as high as ever.

If all of EA disappeared tomorrow, I would continue on my merry way trying to make things better. I would continue spending my free time trying to build a community in Denmark of people interested in spending their time and resources to do good as effectively as possible.

What brought me to EA was an intrinsic motivation to do as much good as possible, and nothing any other effective altruist can do is going to change this motivation.

I'm happy to consider anyone who shares that objective to be a friend, even if we don't on the specifics of how exactly one should go about trying to do the most good. "Doing the most good" is a pretty nebulous concept after all. I would find it pretty weird if we all agreed completely on what that implies.

Remember:

- We're all on the same team
- We're all just human beings try to do the best we can
- We're all acting on imperfect information

Spreadsheets are in many ways a force-multiplier of all other work that one does. For that reason I am very happy to have invested significant time into becoming good at utilizing spreadsheets in the work I do.

Over the past months, I’ve increasingly started using GPT in my workflow and am starting to see it as a tool that similarly to spreadsheets can make one better across a vide variety of tasks.

It wasn’t immediately useful however! It was only with continuous practice that it started generating actual value.

It took me a while to get good at noticing when some task I was doing could be sped up by involving GPT, but especially for brainstorming or listing things it does in seconds what would take me hours. I highly recommend investing time it takes to get it into your workflow. It takes time to build an intuition of what it can and cannot do well.

For example, my org spent some hours creating a list of organizations that currently attempt to influence aid spending in our target country. I asked GPT what organizations we had missed and in seconds was able to add an additional 15 organizations onto the list we had overlooked.

The amount of tasks we can outsource to AI will only increase going forward, and I think those who invest time into getting good at utilizing the new wave of AI tools will be able to multiply productivity significantly and will be at an advantage over those who don't.

Does Claude-3 push capabilities?

I think it can be a fun exercise is to just interpret CEOs statements literally and see what they imply.

If Dario Amodei claims they don't want to push capabilities, I think an interesting question to ask is in what sense releasing the world's best LLM isn't pushing capabilities.

One option that seems possible to me, could be that they no longer consider releasing improved LLMs to meaningfully push the frontier. If Claude-3 spurs OpenAI to push a quicker release of GPT-4.5, this would not be an issue as releasing ever more refined LLMs doesn't meaningfully increase any specific risk.

In that case they would still be true to their word, Anthropic just no longer considers LLMs on their own to be the frontier of capabilities. This seems reasonable to me?

EDIT: Someone on lesswrong linked a great report by Epoch which tries to answer exactly this.

With the release of openAI o1, I want to ask a question I've been wondering about for a few months.

Like the chinchilla paper, which estimated the optimal ratio of data to compute, are there any similar estimates for the optimal ratio of compute to spend on inference vs training?

In the release they show this chart:

The chart somewhat gets at what I want to know, but doesn't answer it completely. How much additional inference compute would I need a 1e25 o1-like model to perform as well as a one shotted 1e26?

Additionally, for some x number of queries, what is the optimal ratio of compute to spend on training versus inference? How does that change for different values of x?

Are there any public attempts at estimating this stuff? If so, where can I read about it?

I'm working on an article about gene drives to eradicate malaria, and am looking for biology experts who can help me understand certain areas I'm finding confusing and fact check claims I feel unsure about.

If you are a masters or grad student in biology and would be interested in helping, I would be incredibly grateful.

An example of a question I've been trying to answer today:

How likely is successful crossbreeding between subspecies of Anopheles Gambiae (such as anopheles gambiae s.s. and anopheles arabiensis), and how likely is successful crossbreeding between anopheles gambiae and other complexes?

If you know the answer to questions like these or would have an easy time finding it out, send me a dm! Happy to pay for your time.

Does anyone have advice on getting rid of material desire?

Unlike many I admire I seem to have a much larger desire to buy stuff I don't need. For example I currently feel an overpowering urge to spend $100 on a go board, despite the fact that I little need for one.

I'm not arguing that I have some duty to live frugally due to EA, I just would prefer to be a version of myself that doesn't feel the need to spend money on as much stupid stuff.

Merry Christmas and happy holidays :)

Has anyone managed to get any use out of gpt-4 integrations yet? I've tried to set up integrations into my private spreadsheets with Zapier, but the painfully slow speed at which gpt-4 writes and needing to click a link to confirm every action makes any small ask slower than just doing it myself.

So far I've been pretty disappointed, but maybe I'm just steering myself blind on tasks that it's currently not well suited for.

Does anyone know if there been any research into creating engaging television for factory farmed animals? Google scholar didn't get much outside of using TV to induce feeding in chickens. I know there have been evaluations of branched chains as a way to improve the conditions for pigs, but I haven't seen any evaluation of television.

There's 24/7 television made for house cats, why couldn't something similar exist for chickens?

I'm not going to find time to look into this myself, so if somebody finds the idea intriguing, don't hesitate with starting!

I really really (and I cannot emphasize this enough) really dislike writing applications. It gives me a feeling of despair and inadequacy about my career and life choices. Due to this I write much fewer applications than I should be, and spend too much time and energy on the few I do send.

I generally feel confident about myself but writing applications for some reason really messes me up.

Has anyone here dealt with anxiety when writing applications? If so, how did you overcome it?

I'm in San Francisco this weekend and next week, if you're in the bay area want to meet, don't hesitate to reach out :) (both happy to meet 1-1 and at social events)

Signal boosting my twitter poll, which I am very curious to have answered:

https://twitter.com/BondeKirk/status/1758884801954582990

Basically the question I'm trying to get at is whether having hands-on experience training LLMs (proxy for technical expertise) makes you more or less likely to take existential risks from AI seriously.

I believe AI will significantly decrease the cost of overregulation, and make many policies attractive that previously were too costly to administrate.

Read why I believe this in my new substack, which I'm trying to start so I have a place to write about non-EA stuff!

https://mkbaio.substack.com/p/ai-will-make-regulation-much-less

I wrote down a list of all the things I could spend one hour every day doing. Among high scorers was teaching myself Mandarin.

Has anyone looked into the value of learning Mandarin, for the average person disinterested in China?

Triplebyte is a company that interviews and vets software developers, identifying their strengths and weaknesses. Triplebyte can cut down the time spent on draining interviews significantly. More importantly it makes it easy for firms to find candidates and vice-versa.

Would it be useful to have similar service for EA organisations?

It seems to me the skills EA organisations look for, seem harder to generalize than software development skills. This means centralized interviews are much less valuable.

What does seem useful is reducing the friction that arises from matching companies with candidates.

Less well known orgs could more easily find the labor they need and persons interested in direct work at EA orgs can devote their full focus on their current occupation knowing they will be visible to potential employers.

It seems the 80k job-board is already accomplishing much of this, does anyone reckon there would be demand for an expanded version of this?

At what point do feel with ~90% certainty you would have done more good by donating to animal charities than you've harmed by consuming a regular meat-filled diet?

It would be nice to know the numbers I have in my head somewhat conform to what smart people think.