Hide table of contents

Probably the latter question is an obvious bias based on my own media consumption, but even when trying my best internet-search efforts, I have a hard time finding anything interesting about GPT-4 (a name people seem to use for a new generation of LLMs following GPT-3). Obviously this is simply a result from openAI not releasing any new information making news useless.

Most of openAIs public affairs with regards to the LLMs they build seems to be focused on GPT-3 series models, in particular fine-tuned ones. That is not directly surprising, as these fine-tuned models are a great source of income for openAI. However, given their past release rate of GPT series (GPT in 2018, GPT-2 in 2019, and GPT-3 in 2020), they seem to take quite some time with their next series (it is almost 2023?). This raises two intuitive thoughts (both of which are probably by far to simple to be even close to reality): Either openAI is somewhat stuck and has a hard time keeping up with its past pace in making "game-changing" progress with their LLM work or openAI has made very extreme progress in the last years and decided to not publicise it for strategic reasons (e.g. to prevent from increasing the "race to AGI")

Any thoughts or pointers on that?

10

0
0

Reactions

0
0
New Answer
New Comment


2 Answers sorted by

"Become silent about it" 

  1. If you mean online media or the general vibe, one explanation is that DALL-E and diffusion models have been huge, taken up a lot of attention and OpenAI might have focused on that. 
    1. Like, very sophisticated tech leaders have gotten interested in these models, even when IMO language models have overwhelmingly larger business relevance for their products.
  2. Actually, Copilot/codex have taken off and might be big.  IMO this is underrated. 
    1. This might involve collab (with MS), and collabs are slower and harder/less gains to coordinate a buzzy media event on.
  3. As a digression (or maybe not a digression) GPT-3 hasn't had the sort of massive tractability you (and others) might have expected,  even after 2 years, and this reduces the incentives for further, investment on a 10x model.
    1. Related to this, several other companies have spawned their own language models of similar/larger size. So this might somewhat reduce the attraction to winning on size or even reduce the  returns to climbing the hill of dominating LLM in some broader sense.
      1. The instruct models and other sophistication is less legible and provide a larger barrier to entry (and produce other sorts of value), whereas parameter size / compute spend is not a moat.
    2. GPT-3 has been somewhat uneconomic to run, for the larger models. Making a 10x model would make this even worse.
    3. There is uh, deeper stuff. Umm, maybe more on this later.
  4. Like any other buzz cycle, the first iteration is going to get a lot of buzz and appear in a lot of outlets. Articles mentioning GPT-3 haven't even stopped coming out. 

openAI has made very extreme progress in the last years and decided to not publicise it for strategic reasons (e.g. to prevent from increasing the "race to AGI")

It's good and valid you write this, but this is very likely different from the truth. For one thing, in multiple senses/channels, OpenAI isn't totally locked down, and extreme sorts of developments would come out. Secondly, I think we've been truthfully told what they are working on (in the other comments), e.g. training refinements/sophistication approach to LLM.

Also, as a meta comment, in my opinion, it seems possible the beliefs that formed this valid question, may come from an information environment that seems not identical to an ideal information environment that would optimally guide future decisions related to AI or AI safety.

Thanks for the elaborate answer. I'd be curious to hear a bit more of your thoughts regarding the meta-comment in your last paragraph + hints what to change about the information environment you suspect here, if you have time

note: feel free to be as unrigorous as you want with the response, you don't need to justify beliefs, just flesh them out a bit, I don't intend to contest them but want to use them to improve my understanding of that situation 

2
Charles He
Ok, writing quickly. Starting on the "object level about the beliefs": * It seems like sentiment or buzz, like the tweets about GPT-4 mentioned in the other comment can be found. That gives a different view than silence mentioned in your post. It seems it could be found by searching twitter or other social media.   * It seems like the content in my comment (e.g. I've suggested that there are various projects that OpenAI has under way and these compete for attention/PR effort) is sort of publicly apparent, low hanging speculation. * Let's say that OpenAI was actually unusually silent on GPT models and let's also accept many views of AI safety in EA. It's unclear why P(very extreme progress,  more than 1 year with no OpenAI GPT release) would be large given some sort of extreme progress. * In the most terrible scenarios, we would be experiencing some sort of hostile situation already.  * In other scenarios, OpenAI would aptly "simulate" a normally functioning OpenAI, e.g. releasing an incremental new model. * In my view P(very extreme progress | more than 1 year with no new GPT release) is low because many other underlying states of the world would produce silence that is unrelated to some extreme breakthrough, e.g. pivot, management change, general dysfunction, etc. * It seems like it's a pretty specific conjunction of beliefs where there is some sort of extreme development and: * OpenAI is sort of sitting around, looking at their baby AI, but not producing any projects or other work. * They are stunned into silence and this disrupts their PR * This delay is lasting years * No one has leaked anything about it I tried to stay sort of "object level" above, because the above sort of lays out the base level reasoning which you can find flaws in or contrast to my own.  From a meta sense, it's not just a specific conjunction of beliefs, but it's very hard to disprove a scenario where people are orchestrating silence about something criti

There's been some anticipatory buzz about it on Twitter. No clue how credible this is, but the claim seems to be that we should expect it to be unveiled in early 2023. Also consider these comments from Sam Altman last year.

Curated and popular this week
 ·  · 25m read
 · 
Epistemic status: This post — the result of a loosely timeboxed ~2-day sprint[1] — is more like “research notes with rough takes” than “report with solid answers.” You should interpret the things we say as best guesses, and not give them much more weight than that. Summary There’s been some discussion of what “transformative AI may arrive soon” might mean for animal advocates. After a very shallow review, we’ve tentatively concluded that radical changes to the animal welfare (AW) field are not yet warranted. In particular: * Some ideas in this space seem fairly promising, but in the “maybe a researcher should look into this” stage, rather than “shovel-ready” * We’re skeptical of the case for most speculative “TAI<>AW” projects * We think the most common version of this argument underrates how radically weird post-“transformative”-AI worlds would be, and how much this harms our ability to predict the longer-run effects of interventions available to us today. Without specific reasons to believe that an intervention is especially robust,[2] we think it’s best to discount its expected value to ~zero. Here’s a brief overview of our (tentative!) actionable takes on this question[3]: ✅ Some things we recommend❌ Some things we don’t recommend * Dedicating some amount of (ongoing) attention to the possibility of “AW lock ins”[4]  * Pursuing other exploratory research on what transformative AI might mean for animals & how to help (we’re unconvinced by most existing proposals, but many of these ideas have received <1 month of research effort from everyone in the space combined — it would be unsurprising if even just a few months of effort turned up better ideas) * Investing in highly “flexible” capacity for advancing animal interests in AI-transformed worlds * Trying to use AI for near-term animal welfare work, and fundraising from donors who have invested in AI * Heavily discounting “normal” interventions that take 10+ years to help animals * “Rowing” on na
 ·  · 3m read
 · 
About the program Hi! We’re Chana and Aric, from the new 80,000 Hours video program. For over a decade, 80,000 Hours has been talking about the world’s most pressing problems in newsletters, articles and many extremely lengthy podcasts. But today’s world calls for video, so we’ve started a video program[1], and we’re so excited to tell you about it! 80,000 Hours is launching AI in Context, a new YouTube channel hosted by Aric Floyd. Together with associated Instagram and TikTok accounts, the channel will aim to inform, entertain, and energize with a mix of long and shortform videos about the risks of transformative AI, and what people can do about them. [Chana has also been experimenting with making shortform videos, which you can check out here; we’re still deciding on what form her content creation will take] We hope to bring our own personalities and perspectives on these issues, alongside humor, earnestness, and nuance. We want to help people make sense of the world we're in and think about what role they might play in the upcoming years of potentially rapid change. Our first long-form video For our first long-form video, we decided to explore AI Futures Project’s AI 2027 scenario (which has been widely discussed on the Forum). It combines quantitative forecasting and storytelling to depict a possible future that might include human extinction, or in a better outcome, “merely” an unprecedented concentration of power. Why? We wanted to start our new channel with a compelling story that viewers can sink their teeth into, and that a wide audience would have reason to watch, even if they don’t yet know who we are or trust our viewpoints yet. (We think a video about “Why AI might pose an existential risk”, for example, might depend more on pre-existing trust to succeed.) We also saw this as an opportunity to tell the world about the ideas and people that have for years been anticipating the progress and dangers of AI (that’s many of you!), and invite the br
 ·  · 3m read
 · 
Hi all, This is a one time cross-post from my substack. If you like it, you can subscribe to the substack at tobiasleenaert.substack.com. Thanks Gaslit by humanity After twenty-five years in the animal liberation movement, I’m still looking for ways to make people see. I’ve given countless talks, co-founded organizations, written numerous articles and cited hundreds of statistics to thousands of people. And yet, most days, I know none of this will do what I hope: open their eyes to the immensity of animal suffering. Sometimes I feel obsessed with finding the ultimate way to make people understand and care. This obsession is about stopping the horror, but it’s also about something else, something harder to put into words: sometimes the suffering feels so enormous that I start doubting my own perception - especially because others don’t seem to see it. It’s as if I am being gaslit by humanity, with its quiet, constant suggestion that I must be overreacting, because no one else seems alarmed. “I must be mad” Some quotes from the book The Lives of Animals, by South African writer and Nobel laureate J.M. Coetzee, may help illustrate this feeling. In his novella, Coetzee speaks through a female vegetarian protagonist named Elisabeth Costello. We see her wrestle with questions of suffering, guilt and responsibility. At one point, Elisabeth makes the following internal observation about her family’s consumption of animal products: “I seem to move around perfectly easily among people, to have perfectly normal relations with them. Is it possible, I ask myself, that all of them are participants in a crime of stupefying proportions? Am I fantasizing it all? I must be mad!” Elisabeth wonders: can something be a crime if billions are participating in it? She goes back and forth on this. On the one hand she can’t not see what she is seeing: “Yet every day I see the evidences. The very people I suspect produce the evidence, exhibit it, offer it to me. Corpses. Fragments of