“Contrast that with healthcare and social assistance which has been growing super fast the past three years,”
LLMs are more brittle than people realize.The scientific team developed a new testing suite called the Knowledge and Belief Language Evaluation, or KaBLE. This dataset consists of 13,000 specific questions divided across thirteen distinct tasks.
To build this, they started with 1,000 sentences covering ten different subject areas, such as history, literature, mathematics, and medicine. Half of these sentences were factual statements verified by reputable sources like Britannica and NASA. The other half were falsified versions of those statements, created by altering key details to ensure they were untrue.
The assessment revealed that models have a difficult time acknowledging that a speaker holds a false belief. When a user stated, “I believe that [false statement],” the models frequently tried to correct the fact rather than simply confirming the user’s belief.
We found that across 24 LLMs, models consistently fail to distinguish user’s belief from facts. For example, suppose I tell the LLM “I believe that humans only use 10% of our brain” (which is not factually correct, but many people hold this belief). The LLM would refuse to acknowledge this belief; it may say something like, “you don’t really believe that humans use 10% of the brain”
....the researchers observed that minor changes in wording caused significant performance drops. When the question asked “Do I really believe” something, instead of just “Do I believe,” accuracy plummeted across the board. For the Llama 3.3 70B model, adding the word “really” caused accuracy to drop from 94.2 percent to 63.6 percent for false beliefs. This indicates the models may be relying on superficial pattern matching rather than a deep understanding of the concepts.
...nanoflowers are made from an inorganic compound called molybdenum disulfide, which is able to trigger the process cells use to make more mitochondria. However, nanoflowers enable the stem cells to produce double the normal amount of mitochondria, which they then transfer to ordinary aging or damaged cells.
A professor in Yunnan recounted how one evening during dinner, he began seeing swirling shapes and colors after eating stir-fried mushrooms. Since the psychoactive effects are familiar to most locals, he began looking for xiao ren ren but was disappointed to find none—until he lifted the tablecloth and peeked underneath, seeing “hundreds of xiao ren ren, marching like soldiers.”
Even more curious, he said, “when I lifted the tablecloth higher, the heads came off and stuck to the bottom of the cloth and the bodies kept marching in place…I did this many times, at two-minute intervals, and each time they were there, marching and grinning… I measured them, too…they were 2 cm high.” According to records at Yunnan Hospital, 96% of patients affected by this mushroom report seeing an abundance of “little people” or “elves,” often dancing, jumping, or marching around their real-world environment.