Therefore, a gradual increase in LLM adoption can produce a disproportionate collective response. Beyond a critical point, the loss of autonomy becomes self-reinforcing and the population can move rapidly toward a state of much stronger cognitive offloading and lower cognitive competence. The important result is not that such a runaway must occur, but that it can occur under plausible forms of social learning and cooperative reinforcement. Moreover, once such a state has become established, simply returning conditions to where they were before the transition may not be sufficient to restore the previous state. Prevention can therefore be considerably easier than reversal.
Tristan Buckmaster was interviewed on CNBC recently. He said:
Yeah, it's a problem.I mean, I think the bigger picture is like this recent dump of papers on Tuesday which destroyed the careers of early, uh, mathematicians. Early career mathematicians. So we’ve had whole research programs wiped out on Tuesday by their dump. And you have to understand they have access to all our research proposals and, and uh, reason they have access...You don't even have to use LLM. So if you submit a research proposal, a panelist will look at that research proposal and they have to write a summary. And often how they write a summary is they put it into a... So they have a massive conflict of interest, right? They have all ... for the next few years they have all the mathematicians' and scientists' research programs in their training data, and how they're going to go about it.
For Robertson, currently studying for a Ph.D. in quantum-safe cryptography at the University of Amsterdam, the problem is even more acute. She has seen all of her mathematics colleagues turn to using LLMs intensively in their research, with many maxing out their Pro subscriptions, and some even spending thousands of euros on additional tokens: “And these are Ph.D. students who don’t have thousands of euros,” she added.
Robertson said that there are colleagues who feel coerced by the tech giants, with the likes of OpenAI announcing in July that it is giving away 100,000 free licenses to its frontier models for researchers in academia. And there are other colleagues who feel they simply have no choice: “If you don’t work at the rate at which you could work with LLMs then you will be behind your peers who will be applying for the same jobs as you.”