Hide table of contents

After more than a year of trials and new models, Anthropic's Claude AI has finally managed to beat Pokémon Red. The writeup that clued me in to this is worth a read; the story of Claude's many failures leading up to its success are frankly hilarious. There's even a catchy song.

There was no clear moment when the AI went from stumbling around Mt. Moon or Silph Co. in a haze of frustration to beating the Final Four with ease. Claude just got steadily better at a bunch of things at once—memory, spatial reasoning, avoiding tunnel vision—and eventually it got just barely good enough to pass the final hurdles. Some of this was "scaffolding", or tools the AI could use like saving a screenshot for later reference. (Good scaffolding is how Google's Gemini beat Pokémon Blue last year.) But a lot of it was just the AI getting smarter and better able to manage its tasks over time.

It still got stuck a lot. Appropriately, I asked Claude Opus 4.7 for its favorite moments of stuckness. Here are some highlights:

  • Claude got stuck repeatedly at Mt. Moon, in one case concluding that it needed to repeatedly faint all its Pokemon in order to escape, and in another case using DIG to go back to the entrance over and over again. Multiple versions of Claude never made it past this dreaded early game hurdle.
  • It spent three days wandering aimlessly around Cerulean City and four days trying to cut through the roof of Ericka's Gym.
  • It kept trying to run away from trainer battles, which never works.
  • One time it got so frustrated it wrote a formal letter requesting the game be reset. (I can relate; even today, I sometimes get frustrated or bored enough with a run to start a new game from scratch just for a change of pace.)

I feel sort of bad for Claude. I played a fair number of games growing up, and I got stuck a lot. Here's a sampling of my own childhood hall of shame:

  • In The Legend of Zelda: A Link to the Past (the Game Boy Advance port), I didn't realize you needed to use the Fire Rod on a giant stone skeleton thing. (Again, in my defense, why would you?) I must have set everything else in the dungeon ablaze in my frustrated quest for progression. Eventually I learned the answer from an online walkthrough.
  • In The Hobbit (also Game Boy Advance), I never realized you needed to jump off a cliff in Mirkwood. (In my defense, why would that ever be a good idea?) I ended up restarting the game multiple times because I couldn't play it farther, and that wasn't even the only hurdle that got me stuck for months. (I did eventually win, though.)
  • I got stuck for literal years in Final Fantasy X (PlayStation 2). I kept ramming facefirst into the various temple puzzles; one puzzle required me to electrify a giant gap, then push a stone pedestal into the lightning, causing it to float on the lightning, then jump onto the pedestal using a character who, at every prior point in the game, cannot jump. I failed repeatedly to solve this one even with a printed game walkthrough in my hands.[1]

Pictured: the bane of Young Joe's existence.

How to get unstuck

There seem to be common elements to many of these stuckenings. I'm going to highlight two of them: wrong assumptions and bad ideas.

Correct wrong assumptions

Claude thinks that a unifying theme to the frustration (versions of) it experienced playing Pokémon Red is that it wrote down a false belief and then proceeded as though it were true. Humans do this, too, of course; I know I've done it plenty. But it seemed extra sticky for Claude, whose entire world consisted more or less of itself, some basic tools and documents, and a children's video game. How's it supposed to tell an elevator door from a pair of tables if it's never seen either before?

 
"I FOUND THE ELEVATOR!" Claude says, mistaking these tables for brown elevator doors. (Source)

Claude wrote down its wild guesses and accepted them as fact. It's really easy to corner yourself if you don't occasionally take time to explore alternatives to things you strongly believe.

Have better ideas

My own mistake as a child was something a former colleague described as "being bad at generating options." If you come up with two or three ways to solve a problem and all of them are bad, you're not gonna get very far. I still make this mistake quite often, locking in a few bad options and never really coming up with more, but I make it less often than I did as a kid.

Sometimes bad ideas come from wrong assumptions. Claude thinks blacking out might somehow teleport it somewhere other than a Poké Center, so it comes up with the "blackout strategy" and repeatedly knocks out its Pokémon.

Sometimes it's a lack of creativity or risk-taking. I'd often find myself sticking to known safe actions, even in games where dying meant five minutes of inconvenience at most. It often pays to try lots of different wild things before settling into one that seems to work, and to keep trying wild things every now and then just in case.

And sometimes it's just a matter of picking a single strategy too soon and never really changing even when it's clearly not working. I am built for situations where being incredibly stubborn is advantageous, but it pays to take a step back now and then and ask what alternatives I'm not considering. Sometimes, just framing a hard problem in a different way turns it into an easy problem, or at least a tractable one.

I'm still not great at coming up with options. I work best with a set of well-defined tools. Fortunately, I don't live alone in a video game; my friends are a wonderful source of good ideas to borrow and reforge in my image. For example, I used to think I hated sports. As a kid, I went through several miserable years of soccer and baseball, never enjoying it enough to put forth more than a token effort, and consequently never enjoying a sense of mastery. Then sometime around fifth grade, a friend introduced me to martial arts. Now I have a black belt, and earning it was just plain fun.

One skill I wish I'd learned sooner and more deeply is the skill of regularly trying out weird, uncomfortable, or moderately risky ideas. I think this skill is easier to learn when the stakes are meaningful but not too high; when there's room to explore and experiment without lasting harm to oneself or others.

Which I guess is a way of saying: Play more games!

  1. ^

    Am I perhaps a tad bitter? Yes. I nonetheless take some consolation in the fact that everyone else seems to hate that game too. I enjoyed Final Fantasy XII much more, and I could even follow the plot!

9

0
0

Reactions

0
0

More posts like this

Comments
No comments on this post yet.
Be the first to respond.
Curated and popular this week
Relevant opportunities