I really like this approach & just created a Custom GPT that does the same with just a link:
https://chat.openai.com/g/g-c3Lxqwie6-ea-intro-gpt
(I am open to creating more such LLMs/chatbots - it has become quite simple to me. And happy to show someone how to integrate this with an API as well.)
Thank you! That's a kind offer and I may take you up on it some time.
(I had been meaning to edit this post with a note that ChatGPT has a new feature that's better set up for doing this with a little more effort than what I proposed here.)
Many thanks Jamie. This is very useful! How I wish though AI didn’t make things up 😂😂 it’s so keen to please us!
Thanks for running with the idea! This is a major thing within education these days (e.g., Khan academy). This seems reasonably successful although Peter's example and the tendency to hallucinate makes me a bit concerned.
I'd be keen on attempting to fine-tune available foundations models on the relevant data. E.g., gpt-3.5 and see how good a result one might get.
My intuition after playing around with many of these models is that GPT 3.5 is probably not good enough at general reasoning to produce consistent results. It seems likely to me that either GPT 4 or Claude 2 would be good enough. FWIW, in a recent video Nathan Labenz said that he originally suggested to use GPT 4 and then go from there when people asked him for recommendations. The analysis gets more complicated with Claude 2 (perhaps slightly worse at reasoning, longer context window).
Yeah, this could be the case. Just not sure that gpt4 can be given enough context for it to be a highly user friendly chatbot in the curriculum. But it might be the best of the two options.
Is there a paid version that has a higher message limit? I see the API pricing but nothing for chat.
Yeah, not sure. I expect this won't be a major bottleneck for most participants if they're just using it to bounce a few ideas around with.
[EDIT: Custom versions of ChatGPT now seem a better way to do this. AND it seems like Claude have reduced their token limits (and least on free accounts), so the method below isn't really workable anymore, sadly.]
Anthropic recently released Claude 2, which can use up to 100,000 tokens of input text, which is apparently about 75,000 words. So, you can feed it loads of info and ask it questions or have a chat!
It actually generates helpful responses when discussing impact-focused questions! E.g.
Me, role playing: “One concern I have with this kind of approach is that I don’t like comparing causes that are so different to each other because it’s just unfair to make those kinds of comparisons when I don’t think they can be put against each other. I feel like all the discussed causes are important and require attention, but you can’t say which is more important than others.”
Claude (when loaded with week 1 of the EA Handbook / intro fellowship): “I understand the concern about comparing very different causes. There are some good counterpoints to consider:
- We're already implicitly comparing causes when we decide where to donate or volunteer our time. Choosing to focus more on cause X means we focus less on cause Y.
- Perfect comparisons are impossible. But rough, imperfect comparisons can still point us in better directions than no comparisons.
- The alternatives to comparing impact are making decisions based on intuitions, emotions, or heuristics. These often lead us astray.
- While all causes may be important, resources are limited. Comparisons help us determine which issues are relatively more pressing or neglected.
- If we refuse to compare, we risk missing enormous differences in scale. For example, health interventions that save one life for $5,000 vs $1 million.
- There are thoughtful ways to account for uncertainties and make "apples to oranges" comparisons more fair. For example, using ranges and probability distributions instead of point estimates.
Overall, while no methodology is perfect, the evidence suggests comparing causes leads to directing resources in ways that help more people. We just need to make comparisons thoughtfully and humbly. But refusing to compare at all means we fail to use information that could guide us towards helping as effectively as possible.”
This seems useful for supporting online fellowships and courses.
Or, you can just watch this video where I do it.
Thanks to Sebastian Schmidt for the prompt to think about how to use LLMs to support learning and engagement in online programmes!
Back in March I asked GPT-4:
The answer (one shot):
I didn't give it any other text as context, just the prompt above.
Yeah, seems fair; asking LLMs to model specific orgs or people might achieve a similar effect without needing the contextual info, if there's much info about those orgs or people in the training data and you don't need it to represent specific ideas or info highlighted in a course's core materials.