<quote>
Have you ever heard of the Joined Together States? Or bosom peril? Kidney disappointment? Fake neural organizations? Lactose bigotry? These nonsensical, and sometimes amusing, word sequences are among thousands of “tortured phrases” that sleuths have found littered throughout reputable scientific journals.
They typically result from using paraphrasing tools to evade plagiarism-detection software when stealing someone else’s text. The phrases above are real examples of bungled synonyms for the United States, breast cancer, kidney failure, artificial neural networks, and lactose intolerance, respectively.
</quote>
During the whole BLM thing a few years ago my emplyer wanted "to do something" I guess so they added one of those "inclusive language" SaaS buttons.
This seems more plausible to me. "Bosom peril" is not in any way a synonym for cancer. But if you think 'cancer' as a word will trigger people then softening it with 'peril' makes sense.
It's not a woke sensitivity thing. They're using spinners, tools designed for spammers years ago (long predating LLMs) that just randomly substitute words from a thesaurus.
It works because so many academic journals are write-only media. The only things that actually "read" the papers are machines that check for plagiarism, compute citation counts and do other things used to decide on academic promotions. So it doesn't matter if the text is nonsensical.
And the tools work for spam, the original use case, because people who want to buy Viagra from spam mails don't care if they have to read weird English, they just want to find the web shops. Works great for bypassing keyword and Bayesian filters though.
Some papers show signs of being generated by context-free grammars or random phrase combination as well.
These papers are often being produced by "paper mills" that bulk generate fake studies and sell them to academics, who use them to get promoted without having to do any real research.
I wrote an article about the problem years ago if you want to learn more:
Sadly this substitution is rational since the Trump NSF went around cancelling grants based on CTRL+F “intolerance” and other “woke words” that also had technical meanings.
This example seems to precede the woke purge and perhaps is to do with citation laundering and evading plagiarism detectors.
This doesn‘t sound like AI. LLMs do not use dictionary definitions for translating a paper this way. It does look like some translation engine like Google Translate or worse, maybe with some text obfuscator trying to avoid plagiarism detection.
Anthropic’s watermark is just using a known collection of random seeds during output token generation. That’s it. The models always relied on some randomness, here the randomness is fixed and remembered for querying later.
I had sent this an hour ago as a quote with a few others (lactose bigotry, of course the kidney disappointment) to someone via iMessage. The Apple Intelligence summary was unfortunately not screenshotted by them, but I am told it was a doozy...
Most of these are authored by what appear to be non-native English speakers. So, the most likely explanation is a translation issue.
In engineering literature from Russia from the 1960s, one sometimes finds references to a ‘water goat’ in papers that are otherwise about heavy machinery. It turns out this is a twice-translated rendition of ‘hydraulic ram’.
> engineering literature from Russia from the 1960s, one sometimes finds references to a ‘water goat’
Oh, that's where it comes from!
My dad told me something close when I was a kid, but according to him it was the early EU doing dictionary translation between English and French and getting "water sheep". I never could find it.
Through an unrelated anecdote, in my head this is now bound to the music of "smoke on the water", with a sheep impression for the guitar (bass?) — "Sheep on the water, ba baa baaah, ba ba ba baaah"
This is a really confusing one, since neither the "AI generated paper" nor "translation issue" explanations hold.
I found [1], created all the way back in 2021, which seems to first introduce this "kidney disappointment" term, and I would have to assume that perhaps the authors preferred to translate it from their native language.
LLMs did not exist in their current form in 2021, so couldn't have used it to write down a coherent thought, let alone a paper, at the time.
At the same time, I do live in the same country as the authors, and while we don't speak the best English, being a student at a college without some familiarity of the English language to come up with the term "kidney disappointment" is, let's just say, hard to believe for me.
Edit: This other comment[2] made me realize what it might have been: plagiarism avoidance.
Unfortunately, many colleges here do not have the best reputation and often run like degree mills, and the aforementioned comment made me remember a conversation with a friend, who mentioned writing a review paper.
Again, LLMs were not really a thing at the time, so they mentioned how they had to run their paper through a plagiarism detection tool, and then they substituted words with their synonyms. I guess someone really did search for the word "failure" and found "disappointment" as an alternative.
LLMs were not available in 2021, but "Google translate" is much older.
Such translations were not unusual.
Around 2017 to 2019 I used a lot "Google translate", because it saved time despite its flaws, but there never was even a single sentence that I could let exactly as generated by "Google translate".
In each sentence I had to change at least a word or two, and many were hilarious. This was for translations between English and German. I assume that for other more distant languages the translations must have been much worse.
LLMs were absolutely available in 2021; the architecture is from 2017 and stuff like GPT-2 and BERT had been around for years. Conversational assistants, ie. ChatGPT, started in 2022.
It fits the "naive deplagiarization" explanation perfectly, however. Many of them are pre-LLM, when people had to plagiarize things the good old-fashioned way, and subsequently hide that fact via synonym replacement.
Before LLMs, SEO content spammers used something called an "article spinner" which could rewrite lots of versions of the same article (which may or may not be someone else's article) by substituting synonyms, producing just enough variation to not get flagged as duplicate content by search engines. Some of the substitutions produced truly bizarre writing, but those pages were written for Googlebot to count the back links, not for human readers. This looks like what you'd get if you turned random substitutions down to a minimum to avoid plagiarism detection...
Google Translate has existed since way back then. I remember seeing hilarious posts about bizarre translations from Chinese and other languages in the 2010s.
"Kidney transplantation is a surgery to eliminate a sound, working kidney from a living or cerebrum dead giver and embed it into a patient with non-working kidneys.
Kidney transplantation is performed on patients with persistent kidney disappointment, or end-stage renal illness (ESRD). ESRD happens when a sickness or confusion harms the kidneys with the goal that they are presently not able to do satisfactorily eliminating liquids and squanders from the body or of keeping up with the legitimate level of specific kidney-directed synthetics in the circulatory system. Without long haul dialysis or a kidney relocate, ESRD is deadly."
In the spring semester this year my teacher friends said they were getting a lot of student work that was unusually well written compared to the student’s other work, but also contain unnatural typos throughout.
The students were using ChatGPT and thought they would trick the teacher by editing some typos in before submission.
The cheating market is moving fast. There are now tools you can use to rephrase LLM generated output until the Pangram score drops low enough.
Versions of this cat and mouse game have been going on for a long time. Before ChatGPT was a household name there was a market for simple paraphrasing tools that spammers would use to send variations of email text to try to delay pattern matching spam filters.
Cheating detection technology will be a cat and mouse game, but a good teacher who knows his students will spot cheating almost immediately (proving it might be more difficult).
The only solution is to evaluate students in ways that they can't cheat. Hand-written exams in class, oral exams, etc. but that is quite time-consuming and most teachers at least in K-12 are already overloaded.
It seems like, particularly in scholarly works, we should cite AI contributions the same way we cite a third party, with similar consequences around plagiarism. If you’re copying AI whole hog, not quoting and citing it, and not catching basic failures. Is it really “your” work?
I find this kind of free-phasing completely ridiculous. Just ask anyone that's had a kidney transfer that did not work out. I'm pretty sure they considered a failure, not just a disappointment.
It's very clear what's happening: they're passing it through a writing tool that has an "Auto-improve" feature the users click. This would do the naive, pre-LLM, trick of taking "repetitive" vocabulary and varying it using synonyms.
I assume it is due to machine translation? I do not see mentions of "failure" at all, which would be expected to exist if it was to increase variety.
I think (heard) some used to translate an article back and forth a few times to get a "new" article, but maybe it could have been some kind of "paraphrasing" software like you describe (but for avoiding plagiarism detection rather than repetitions).
Huh.. Ok I'll play along. Could this be a translation issue? I confess the first thing that leaped to mind was.. could it be AI written? But I didn't want to leap to that but.. it's .. such a strange word that is not expected in .. I guess formal medical training.
Yeah some kind of translation issue. The papers seem to know what they're talking about, and I don't entirely get the AI gut feeling, only on the key words.
Good point, but I see lots of articles are using this "kidney disappointment" so maybe but.. holy toledo look at Impacts of Renal Transplantation and its Mental (2023) by Renan Kamath : “organ disillusionment,” “kidney relocate dismissal,” and “excusal of the moved organ.”
i kid you not, yesterday i was wondering what's the english word for bathroom scale, so i googled "pèse personne in english", and got answered "person weighs"
We were staying at a nice ocean-side resort in Dana Point, California. My wife, our daughter, and I, were chilling by one of the pools.
Suddenly security was announcing "There's been a sanitary inconvenience, everyone must leave the pool!!! We apologize, but there's been a sanitary inconvenience!".
A moment later we saw them scooping up some floating poop that had escape some baby's swim diaper.
In some languages disappointment does mean malfunction or problem. Like disappointment of stomach. Maybe they are using English words in papers instead of English language? Common for people whose English is not their first language.
I accidentally did this myself not long ago. Like everyone else I'm tired of cookie banners so I had ChatGPT make a law against it, it used "act" in its version, I search-and-replaced it with "Law" before getting Claude to ratify it. Claude pointed out the search-and-replace replacement, oops. The conversation links are in my comment here: https://news.ycombinator.com/item?id=48585778 and you can see Claude found these mistakes:
"
At some point an earlier find-and-replace swapped "act" for "Law" and garbled three real words. As written, the text says:
Section 1: "cookie-banner Lawions" — should be actions
Section 3(2)(b): "next ordinary feature update after enLawment" — should be enactment
Section 5(5): "a site-specific Law that the signal cannot satisfy" — should be act (or action)
These aren't stylistic quibbles; "Lawions" landing on the desks of four legal departments would cause real confusion about intent. The fix is a single reverse find-and-replace. With that correction, I'm comfortable signing.
"
<quote> Have you ever heard of the Joined Together States? Or bosom peril? Kidney disappointment? Fake neural organizations? Lactose bigotry? These nonsensical, and sometimes amusing, word sequences are among thousands of “tortured phrases” that sleuths have found littered throughout reputable scientific journals.
They typically result from using paraphrasing tools to evade plagiarism-detection software when stealing someone else’s text. The phrases above are real examples of bungled synonyms for the United States, breast cancer, kidney failure, artificial neural networks, and lactose intolerance, respectively. </quote>
This seems more plausible to me. "Bosom peril" is not in any way a synonym for cancer. But if you think 'cancer' as a word will trigger people then softening it with 'peril' makes sense.
It works because so many academic journals are write-only media. The only things that actually "read" the papers are machines that check for plagiarism, compute citation counts and do other things used to decide on academic promotions. So it doesn't matter if the text is nonsensical.
And the tools work for spam, the original use case, because people who want to buy Viagra from spam mails don't care if they have to read weird English, they just want to find the web shops. Works great for bypassing keyword and Bayesian filters though.
Some papers show signs of being generated by context-free grammars or random phrase combination as well.
These papers are often being produced by "paper mills" that bulk generate fake studies and sell them to academics, who use them to get promoted without having to do any real research.
I wrote an article about the problem years ago if you want to learn more:
https://dailysceptic.org/2021/10/03/436-randomly-generated-p...
“Subsequently, 1 mL of the mass killing of an ethnic group was opposed to 20 mL of the skin sample and unprotected to light for 7 min.”
From: https://bsky.app/profile/forbetterscience.bsky.social/post/3...
This example seems to precede the woke purge and perhaps is to do with citation laundering and evading plagiarism detectors.
In engineering literature from Russia from the 1960s, one sometimes finds references to a ‘water goat’ in papers that are otherwise about heavy machinery. It turns out this is a twice-translated rendition of ‘hydraulic ram’.
Oh, that's where it comes from!
My dad told me something close when I was a kid, but according to him it was the early EU doing dictionary translation between English and French and getting "water sheep". I never could find it.
Through an unrelated anecdote, in my head this is now bound to the music of "smoke on the water", with a sheep impression for the guitar (bass?) — "Sheep on the water, ba baa baaah, ba ba ba baaah"
My favourite example of troubled blind translation is still "naked conductor runs under the carriage".
I found [1], created all the way back in 2021, which seems to first introduce this "kidney disappointment" term, and I would have to assume that perhaps the authors preferred to translate it from their native language.
LLMs did not exist in their current form in 2021, so couldn't have used it to write down a coherent thought, let alone a paper, at the time.
At the same time, I do live in the same country as the authors, and while we don't speak the best English, being a student at a college without some familiarity of the English language to come up with the term "kidney disappointment" is, let's just say, hard to believe for me.
Edit: This other comment[2] made me realize what it might have been: plagiarism avoidance.
Unfortunately, many colleges here do not have the best reputation and often run like degree mills, and the aforementioned comment made me remember a conversation with a friend, who mentioned writing a review paper.
Again, LLMs were not really a thing at the time, so they mentioned how they had to run their paper through a plagiarism detection tool, and then they substituted words with their synonyms. I guess someone really did search for the word "failure" and found "disappointment" as an alternative.
[1] https://www.researchgate.net/profile/Suprodip-Mandal/publica...
[2] https://news.ycombinator.com/item?id=49319764
Such translations were not unusual.
Around 2017 to 2019 I used a lot "Google translate", because it saved time despite its flaws, but there never was even a single sentence that I could let exactly as generated by "Google translate".
In each sentence I had to change at least a word or two, and many were hilarious. This was for translations between English and German. I assume that for other more distant languages the translations must have been much worse.
https://blackwells.co.uk/bookshop/product/Handbook-of-Kidney...
https://books.google.com/ngrams/graph?content=kidney+disappo...
Wolters Kluwer seems to be a real multinational company?
Conterfeit Consciousness is gonna be the name of my new post-liquid analog modular synth IDM project.
I feel like these text spinners could actually be an untapped gold mine for creatively naming things.
And SCIgen was generating bullshit papers that actually managed to be published in 2005: https://en.wikipedia.org/wiki/SCIgen
The students were using ChatGPT and thought they would trick the teacher by editing some typos in before submission.
The cheating market is moving fast. There are now tools you can use to rephrase LLM generated output until the Pangram score drops low enough.
Versions of this cat and mouse game have been going on for a long time. Before ChatGPT was a household name there was a market for simple paraphrasing tools that spammers would use to send variations of email text to try to delay pattern matching spam filters.
The only solution is to evaluate students in ways that they can't cheat. Hand-written exams in class, oral exams, etc. but that is quite time-consuming and most teachers at least in K-12 are already overloaded.
In statistics: "tendency score matching" https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=%22t...
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C38&q=%22...
I think (heard) some used to translate an article back and forth a few times to get a "new" article, but maybe it could have been some kind of "paraphrasing" software like you describe (but for avoiding plagiarism detection rather than repetitions).
Update: looking at some of these, I think it is probably just poor machine translation.
"Regularly, it's more irritating than it is not kidding..."
(Pal and Gautam 2018)
Changing words for watermarking .. I understand why it may do that .. but .. what an unfortunate choice of where to do it.
Suddenly security was announcing "There's been a sanitary inconvenience, everyone must leave the pool!!! We apologize, but there's been a sanitary inconvenience!".
A moment later we saw them scooping up some floating poop that had escape some baby's swim diaper.
https://scholar.google.com/scholar?&q=%22renal+disappointmen...
Guess why the acronym doesn't match. It's called chronic kidney disease (CKD). Yeah the acronym is wrong too.