Interesting responses [WaybackSave/Archive] Tom Sydney Kerckhove on X: “I haven’t found any programming tasks that an LLM could do even barely correctly. What kind of code are you all writing?!” and later

They all come down to

• excuses for using LLM without any substantial result (most of the results come down to one having to become the tester and fixer of the generated code without newer generated code being improved: the opposite of coaching an apprentice)


• become better at prompting (which is basically regarding the prompt as a new programming language: been there, done that)

[WaybackSave/Archive] One of the “become better at prompting” replies referred to a blog post disguising prompting as writing lots of unit tests: [Wayback/Archive] The Cline AI Assistant is Mesmerizing · mtlynch.io

I tried out the Cline AI assistant yesterday, and then I went into a trance for five hours where I couldn’t do anything but stare transfixed at Cline fixing bugs for me.

…

I should be able to just keep showing the AI assistant test cases with the behavior I want in, and it should be able to just keep editing the code until the test passes.

…

And that’s when I was hooked. I was so amazed that I could develop software this way. I just told the tool what I wanted, and it kept doing exactly what I asked.

Well, it is relatively easy to make unit tests pass no matter the programming environment without solving the actual problem. Examples for one are at [Wayback/Archive] java – How to make an unit test always pass? – Stack Overflow.

I responded to one of the longer reactions:

Shorter: LLMs can’t understand.

That’s their fundamental problem: they are just a bunch of statistics trained on ever dwindling levels of input.

and later added

Larger models don’t solve a fundamental problem: they are based on statistics instead of reasoning.

More training data fails another fundamental problem: the data pool is already poisoned by LLM generated mediocre code.

The pool level already was of average programmers at best.

plus

Unless brain-inspired machine learning ¹ takes off, LLMs can improve little as they exhausted the data lake already ² and the internet is already full of data poisoning new models ³.

1/

¹ https://pmc.ncbi.nlm.nih.gov/articles/PMC10630093/

² https://petapixel.com/2025/01/09/have-ai-companies-run-out-of-training-data-elon-musk-ilya-sutskever-thinks-so/

³ https://www.nature.com/articles/s41586-024-07566-y

and

What could help to a limited extent is human curation of the data pool. That is prohibitively expensive.

2/2

The only use case for me using LLM is to get a rough idea of a new field of work, but usually using the better search engines get me there just as quickly.

This response was epic: [Wayback/Archive] Catalin on X: “@kerckhove_ts “write better prompts” army coming.”

[Wayback/Archive] Gi7fb9PWIAAwQod.jpg (496×264)

[Wayback/Archive] video.twimg.com/tweet_video/Gi7fb9PWIAAwQod.mp4

[Wayback/Archive] Tweet JSON

--jeroen

My tweets:

1. [WaybackSave/Archive] Jeroen Wiert Pluimers @wiert@mastodon.social on X: “@DanielHoffmann_ @cssslinger @kerckhove_ts Shorter: LLMs can’t understand. That’s their fundamental problem: they are just a bunch of statistics trained on ever dwindling levels of input.”


2. [WaybackSave/Archive] Jeroen Wiert Pluimers @wiert@mastodon.social on X: “@DanielHoffmann_ @cssslinger @kerckhove_ts Larger models don’t solve a fundamental problem: they are based on statistics instead of reasoning. More training data fails another fundamental problem: the data pool is already poisoned by LLM generated mediocre code. The pool level already was of average programmers at best.”


3. [WaybackSave/Archive] Jeroen Wiert Pluimers @wiert@mastodon.social on X: “@ten_crowns @DanielHoffmann_ @cssslinger @kerckhove_ts Unless brain-inspired machine learning ¹ takes off, LLMs can improve little as they exhausted the data lake already ² and the internet is already full of data poisoning new models ³. 1/ ¹ … ² … ³ …”


4. [WaybackSave/Archive] Jeroen Wiert Pluimers @wiert@mastodon.social on X: “@ten_crowns @DanielHoffmann_ @cssslinger @kerckhove_ts What could help to a limited extent is human curation of the data pool. That is prohibitively expensive. 2/2”