- We've felt it and now we have the evidence: using genAI to write essays yields shallow encoding. Here, the subjects write essays using one of 3 tools (LLM, Search Engine, brain🧠 only) for 3 sessions. In the 4th session, LLM group needs to rely on 🧠 , and 🧠 group gets LLM. Key consequences for me:
- 𝐍𝐨, 𝐲𝐨𝐮𝐫 𝐛𝐫𝐚𝐢𝐧 𝐝𝐨𝐞𝐬 𝐧𝐨𝐭 𝐩𝐞𝐫𝐟𝐨𝐫𝐦 𝐛𝐞𝐭𝐭𝐞𝐫 𝐚𝐟𝐭𝐞𝐫 𝐋𝐋𝐌 𝐨𝐫 𝐝𝐮𝐫𝐢𝐧𝐠 𝐋𝐋𝐌 𝐮𝐬𝐞. See our paper for more results: "Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task" : www.brainonllm.com
- 1️⃣ Blank page syndrome - LLMs are often seen as an antidote to it. But the paper finds that "the LLM group produced statistically homogeneous essays within each topic". Hence, if you want your piece to stand out, using LLMs to fight the syndrome might not be the best idea.
- 2️⃣ When integrating LLMs into teaching activities, start with cognitive effort, followed by the use of genAI - "Brain-to-LLM participants could leverage tools more strategically, resulting in stronger performance and more cohesive neural signatures".
- In my module, I do that by asking the students to complete independent research first and only then engage with the genAI task.
- 3️⃣AI detection - LLM group displayed significantly lower ability to quote one own's essay - AI use detection could leverage this by randomly sampling students for oral exams discussing the essay. Alternatively, only suspect cases could be subject to such an exam.Jun 18, 2025 13:17
