Hide table of contents

Summary

  • I'm concerned about a lack of discussion on this forum about the use of AI to cheat on academic writing. I think this type of misuse is an emerging AI alignment problem. 
  • This is adding to the substantial challenges faced by teachers. At least in the US, they are already disrespected, poorly paid, and "leaving the profession in droves." 
  • I provide 7 tips for teachers to optimize their essay prompts to help them recognize ChatGPT output.
  • I suggest that AI companies should provide free tools to teachers to help them detect and counteract cheating.

Notes: Crossposted here. I've added an EA-specific summary for the forum. This is not an attempt to synthesize AI-enabled cheating as a cause area. I'd file this more under "some advice for dealing with this new problem and some suggestions for AI companies." 

Introduction

Like so many people, I’ve been surprised and baffled by ChatGPT. It generates coherent responses to a huge array of prompts. The technology threatens the written assignment as a means of developing and assessing the type of “21st century” skills that are (at least for now) essential in so many jobs. 

The thing is this: ChatGPT has some glaring weaknesses. It produces a lot of material that is just flat out wrong. It even (very helpfully) fabricates citations and quotes. These telltale errors and fabrications are easy to see because you, the teacher, are knowledgeable about the essay topic. The key is to make your job easier by getting ChatGPT to throw as many errors as possible. That way, if students do try to use this AI to write their essays, they need to spend so much time researching and fixing the errors to avoid getting caught that it’s not even worth the trouble.

ChatGPT readily admits to fabricating a citation…and rhapsodizes about academic integrity. Source: ChatGPT from OpenAI.

Seven ways to optimize for nonsense ChatGPT output

Here are seven strategies that I’ve found useful for getting the best kind of garbage out of ChatGPT. I recommend implementing all seven together. Rather than explain each in detail, I provide some examples here.

  1. Ask about recent developments – ChatGPT usually refuses to opine on anything that isn’t covered in its training data, which runs through 2021. If you phrase the question correctly, it may still answer but will refer to older developments. For example, when asked about medications recently approved by the FDA, it references approvals from the 2010s.
  2. Request multiple citations – Many (though not all) of the citations generated by ChatGPT are complete fabrications, even though they look real. If you search for the sources or follow the links, you’ll usually find nothing. This is telltale.
  3. Ask for quotes from the citations – Similarly, ChatGPT will generate what appear to be quotes from sources when the quotes actually don’t exist in the real world. Even if you ask for “direct quotes” it will do this! Another dead giveaway.
  4. Tell it to take a stand – ChatGPT will often refuse to take a stand, it will hedge its answers, or it will make very general statements. For example, with health-related information, it usually throws in a very Mayo Clinic-esque disclaimer about consulting a health professional. How many students do that?
  5. Request specific information – The more detailed the response from ChatGPT, the more likely it is to be rife with errors. You’ll recognize many of these errors right away.
  6. Ask it to compare and contrast – This is a version of getting specific, which reveals more factual errors. But it also gives ChatGPT the opportunity to make more general statements that betray a lack of real understanding of the question.
  7. Test your prompt in ChatGPT – I’m hesitant to recommend this, because it gives OpenAI more training data, but you want to be sure the responses you get back are awful. Use this as an opportunity to optimize your prompt to maximize the AI’s errors.

It’s not a coincidence that many of these tips for writing prompts seem like good practices in the pre-AI era. Remember that ChatGPT has mastered an illusion: we’re impressed by the legibility and coherence of its writing and we assume this means that it really understands what it is talking about, the way a human student can.

What OpenAI (and other AI behemoths) should do

I’m worried that companies will recognize these faults and try to “fix” them. Let’s face it: this technology takes the job of teachers – who are already disrespected, underpaid, and under enormous strain – and makes it worse. This is a clear and present AI alignment problem. I think these companies should be very hesitant about making AI an even better cheating machine without also giving teachers tools to address new problems caused by AI technologies. Here’s one suggestion:

  • create a tool that checks an essay response for similarity to historical ChatGPT output. 
  • ensure this tool is easy to use and commit to making it free to teachers as long as ChatGPT (or its successors) can conceivably be used for essay cheating.

I’m sure there are logistical and privacy concerns. I’m also sure that these companies can figure out how to surmount these concerns or develop alternative tools for teachers – they certainly have the resources.

Some notes and disclaimers

  • This is current as of mid-to-late December of 2022. It seems possible (even likely) that OpenAI will further improve ChatGPT to enable “higher quality” cheating, so be sure to follow step 7 and field test any essay prompt in the software itself.
  • This is most applicable to science-related prompts for advanced secondary and tertiary-level students. I am sure some of the same ChatGPT weaknesses can be exploited in other fields and for students at other levels. I would welcome additional tips and adaptations to these tips.
  • This should not be taken as a defense of thoughtless regurgitation of facts as a pedagogical approach. That said, facts are important in science, and the types of errors that ChatGPT makes betray its lack of understanding of how facts relate to each other in complex systems. We want students to be able to use careful research and writing to develop that understanding.

8

0
0

Reactions

0
0

More posts like this

Comments3


Sorted by Click to highlight new comments since:

I think this type of misuse is an emerging AI alignment problem.

Misuse can be important or interesting, but the word “alignment” should be reserved for problems like the problem of making systems try to do what their operators want, especially making very capable systems not kill everyone.

This is helpful at the margin

But my experience over 15 years of teaching at UK universities is that “marking student research/take home essays ” was already a highly problematic and inaccurate means of evaluation. Extremely time consuming to judge, especially with students who struggle with the English language. (And you are asked to not penalise them for this). Students can and do pay others to write it for them.

the availability of chat Gpt makes this substantially more difficult.

My advice (to the educational system) would be a combination of:

  1. Try to work with a smaller number of intrinsically motivated students. Focus less on evaluation. This is particularly true for research intensive subjects.

  2. If you need to do evaluation, arrange proctored/Invigilated exams

Curated and popular this week
 ·  · 7m read
 · 
This is a linkpost for a paper I wrote recently, “Endogenous Growth and Excess Variety”, along with a summary. Two schools in growth theory Roughly speaking: In Romer’s (1990) growth model, output per person is interpreted as an economy’s level of “technology”, and the economic growth rate—the growth rate of “real GDP” per person—is proportional to the amount of R&D being done. As Jones (1995) pointed out, populations have grown greatly over the last century, and the proportion of people doing research (and the proportion of GDP spent on research) has grown even more quickly, yet the economic growth rate has not risen. Growth theorists have mainly taken two approaches to reconciling [research] population growth with constant economic growth. “Semi-endogenous” growth models (introduced by Jones (1995)) posit that, as the technological frontier advances, further advances get more difficult. Growth in the number of researchers, and ultimately (if research is not automated) population growth, is therefore necessary to sustain economic growth. “Second-wave endogenous” (I’ll write “SWE”) growth models posit instead that technology grows exponentially with a constant or with a growing population. The idea is that process efficiency—the quantity of a given good producible with given labor and/or capital inputs—grows exponentially with constant research effort, as in a first-wave endogenous model; but when population grows, we develop more goods, leaving research effort per good fixed. (We do this, in the model, because each innovator needs a monopoly on his or her invention in order to compensate for the costs of developing it.) Improvements in process efficiency are called “vertical innovations” and increases in good variety are called “horizontal innovations”. Variety is desirable, so the one-off increase in variety produced by an increase to the population size increases real GDP, but it does not increase the growth rate. Likewise exponential population growth raise
 ·  · 4m read
 · 
TLDR When we look across all jobs globally, many of us in the EA community occupy positions that would rank in the 99.9th percentile or higher by our own preferences within jobs that we could plausibly get.[1] Whether you work at an EA-aligned organization, hold a high-impact role elsewhere, or have a well-compensated position which allows you to make significant high effectiveness donations, your job situation is likely extraordinarily fortunate and high impact by global standards. This career conversations week, it's worth reflecting on this and considering how we can make the most of these opportunities. Intro I think job choice is one of the great advantages of development. Before the industrial revolution, nearly everyone had to be a hunter-gatherer or a farmer, and they typically didn’t get a choice between those. Now there is typically some choice in low income countries, and typically a lot of choice in high income countries. This already suggests that having a job in your preferred field puts you in a high percentile of job choice. But for many in the EA community, the situation is even more fortunate. The Mathematics of Job Preference If you work at an EA-aligned organization and that is your top preference, you occupy an extraordinarily rare position. There are perhaps a few thousand such positions globally, out of the world's several billion jobs. Simple division suggests this puts you in roughly the 99.9999th percentile of job preference. Even if you don't work directly for an EA organization but have secured: * A job allowing significant donations * A position with direct positive impact aligned with your values * Work that combines your skills, interests, and preferred location You likely still occupy a position in the 99.9th percentile or higher of global job preference matching. Even without the impact perspective, if you are working in your preferred field and preferred country, that may put you in the 99.9th percentile of job preference
 ·  · 5m read
 · 
Summary Following our co-founder Joey's recent transition announcement we're actively searching for exceptional leadership to join our C-level team and guide AIM into its next phase. * Find the full job description here * To apply, please visit the following link * Recommend someone you think could be a great fit here * Location: London strongly preferred. Remote candidates willing to work from London at least 3 months a year and otherwise overlapping at least 6 hours with 9 am to 5 pm BST will be considered. We are happy to sponsor UK work visas. * Employment Type: Full-time (35 hours) * Application Deadline: rolling until August 10, 2025 * Start Date: as soon as possible (with some flexibility for the right candidate) * Compensation: £45,000–£90,000 (for details on our compensation policy see full job description) Leadership Transition On March 15th, Joey announced he's stepping away from his role as CEO of AIM, with his planned last day as December 1st. This follows our other co-founder Karolina's completed transition in 2024. Like Karolina, Joey will transition to a board member role while we bring in new leadership to guide AIM's next phase of growth. The Opportunity AIM is at a unique inflection point. We're seeking an exceptional leader to join Samantha and Devon on our C-level team and help shape the next era of one of the most impactful organizations in the EA ecosystem. With foundations established (including a strong leadership team and funding runway), we're ready to scale our influence dramatically and see many exciting pathways to do so. While the current leadership team has a default 2026 strategic plan, we are open to a new CEO proposing radical departures. This might include: * Proposing alternative ways to integrate or spin off existing or new programs * Deciding to spend more resources trialling more experimental programs, or double down on Charity Entrepreneurship * Expanding geographically or deepening impact in existing region