I don’t know if you heard about the scandal that is shaking the literary world here in Québec and France. Writer Thélyson Orélien published the novel « C’était ça ou mourir » in France in 2026. It is a best-seller, by French standards. Soon it was nominated for prestigious literary awards, until someone scanned it with the AI checker Pangram. It would seem that the book was written with the help of AI. However, some people are wondering how reliable is Pangram. Journalists tried out Pangram and seem to think that its highly reliable, with few false positives. Anyone here has any experience with it?
I sent a newsletter to my readers a few weeks ago that I wrote with help from AI. It took me about 90 minutes all up to write it. A lot of tooing and froing, a lot of rewriting. Just before I hit send, I noticed that substack offered to read it and tell me how much AI it contained. It used Pangram. I pushed the button with trepidation …
And it told me the newsletter was “Fully Human Written”.
Which is nonsense - it was written with AI by a human who cared about what he wrote, and - most importantly - the people who read it.
2 Likes
It seems to me the issue is that the author accepts responsibility for the content of what is written. Why does it matter what tool the authors used?
If we get to the point where a robot can overhaul the engine in your car, would it matter to you whether your local repair shop used such a robot? Why is that any different than what method they ued to caibrate the torque on the bolts or what size screwdriver they used?
The end result is what matters. Why does it matter what tool someone used?
1 Like
Because the author is claiming to have written it? 
I should add that I have no problem with someone using AI as an Editor-in-Chief. I use AI as my editor and it is invaluable, but that is a far cry from using AI extensively for the writing and then signing your name to it.
5 Likes
I’m irredeemably old school. If I read literature, I want to connect to the human mind that created it. I don’t want to connect to the robot.
Katie
12 Likes
He says it was primarily written 2017-2019 which would exclude AI other than for editing.
I don’t believe there is any AI checker which can reliably detect I use except the very recently introduced Anthropic fingerprint - but even there it can be hard to draw a line betwen AI auhorship vs reasonable editing.
Maybe this is like claims of “homemade cooking” in a restaurant or exotic cars which are “hand-made.” At some point people will use tools - that’s part of the craft. Judge the final product.
1 Like
Do you also use AI to help you think before you write, Barret?
I do that a lot, but then (if I produce something I want to share) I ask it to make a draft, which I can work with.
I extensively edit the draft so that it’s something I like. That editing is mostly by hand, but I also use AI as an editor, and advisor.
Good writing - for me - has always been a hard-work cycle of editing & rewriting.
But the hardest part is creating a rough draft to edit. That’s donkey work. I get the machine to do much of that donkey work.
No offense to donkeys.
“Write a 3-page essay on the causes of the American Revolution” with copy/paste is claiming AI as your own.
But what if you ask AI “Tell me 20 things about the American Revolution that are myths- with source links”
Then you go and read those 20 articles
Then you tell AI - “Summarize the details of citations 3,4,6, and 12”
Then you tell it “Add an introduction explaining that changes in the English language have lead over time to misunderstanding of primary sources about the American Revolustion”
Then you tell it “Research the history of the ENglish language and how it evoved since 1700.” Then you tell it to rephrase several things
Then you ask it ot add a conclusion
Then you ask it to edit the result to flow evenly
Then you raad the final result and make some edits in grammar or structure
When you are done… Did you “write” that or did AI write it? Would you be OK putting your name to it?
3 Likes
Pangram on its own isn’t reliable, in my opinion, but it’s enough of a signal to look further. My understanding is that Pangram was followed by multiple, closer analyses, and the author’s claims about his history of work and pre-LLMs artifacts were investigated.
I see no problem reserving certain contests for humans who do their own thinking and writing! We might start seeing it broadcast so there’s no question what happened, like track and field in the automobile era. 
That sounds simlar to arguments in suport of lie detectors.
Or lumbar discography or DTI Brain imaging (for those who have a medical background)
Or earprints for criminal identification
Or hypnosis to refresh eyewitness testimony
Or bullet-lead analysis in criminal cases
EIther it’s valid or it is not valid. Otherwise it is a slippery slope.
1 Like
None of the tools I’ve tried have struck me as reliable. Including Pangram. That goes for pre-LLM plagiarism checkers too.
It matters because, given the ways commercial LLMs are trained, they plagiarize. They use sentences, often consecutive sentences, even entire paragraphs, lifted from other texts. They will lift a sentence and make a minor change, adding a “not,” or changing the punctuation. They are elaborate pattern-matching engines. The words are just equivalencies and weighted syntax structures. They do not have meaning to the software.
At the same time, I have seen legitimate writers accused of LLM-assisted writing and/or plaigirism when their work predates the LLM and was used for training.
Even if the ethical aspects (plagiarism) don’t bother you (they bother me, a lot) the potential legal consequences of copyright violation, particularly in digital, should be a concern.
1 Like
That does bother me - but I think the liability should fall on the LLM creator.
Pangram is said to be reliable when it reports that AI was responsible. (False positives exceedingly rare).
False negative are more common - Pangram was “fooled” by an AI.
2 Likes
I did a test of AI detectors about a year ago. I took something I wrote and ran it through the detectors. It tagged ~50% of it as AI. It was not; it was entirely my own. I could only conclude two things: 1) the detectors could not be trusted as the primary means of holding students accountable for their own work, or 2) I write like AI. 
1 Like
I adopt a different approach. For substantial work, I review my gathered research articles on a topic. After reading them, I may use AI to draft a summary of the collection, which I then review and revise. I build an outline of the key ideas in OmniOutliner and then write or dictate a first draft. Using AI as an editor, I fix typos, grammar, logic, and flow. I also rely on AI to propose stronger section headings and similar adjustments. Finally, I use AI to ensure footnotes and references are accurate and consistent (I hate dealing with managing citations ).
For example, this morning, I’m working on a chapter in my book. I have OmniOutliner and Ulysses opened as I work on the draft.
Perhaps I’m wrong regarding this, but I consider the first draft to be the most authentic, challenging, and intellectually stimulating part of the writing process.
From my perspective, if I did what you described, I would not consider that my work, it would be primarily the work of AI, I would not put my name on it. If I do as described within the brackets, I would be comfortable putting my name on it because it reflects my substantive intellectual work.
The process you described makes AI the author and you the editor.
I spoke to someone in Paris yesterday who read the book out of curiosity. They said it is repetitive and horribly written, clearly not a human.
There is a spectrum/continuum, people will draw the line in different places. If you’re open about where the line is I can respect that even if I wouldn’t make the same choices.
Example, I’m writing an article about how most business strategies are just empty pieces of paper. Then I want to show how an Agile approach could make it better.
I’ve read over a dozen bad strategies from Canadian institutions. I was struggling to organize my thinking, so i got Claude to write a draft. Not one word of the draft will be used. However as I started to yell at the page, my own version became more clear.
My line (without judgement on others): All of the words in my writing must come from me. Even the shape of an article must be mine.
2 Likes
That is an interesting assessment.
I can see your rationale. On the other hand the viewpoints and key intellectual content in my example would be uniquely that of the human.
Let me ask the question in a different way. Suppose you take my example and I am a professor having a discussion not with AI but with a graduate student - otherwise the exact same scenario. Would it be acceptable for the professor to be the primary author on the paper?
Then how did it win awards?