The Slopping Forecast (LLM/AI Discussion)

[Jubal]

Since we keep coming back to this topic, I thought I'd at least have a thread to keep interesting links or updates in.


Another recent post in the "AI killed my ability to program well and now I'm stopping" genre:
https://blog.bustikiller.com/2026/09/25/one-month-without-ai.html

A thing I do worry about is whether these are actually relatively isolated cases in an industry where very very few people are going to buck the trend. But these sorts of things are definitely persuading me that coding with LLMs just isn't something I want to do at all even if it does become the sole long-term norm for how people write computer code.

A thing I find odd, conversely, is how little any software I use has increased its utility to me in years. We now apparently have faster coding than ever, but I don't think my own use of any software has improved since 2010 except that I can now play some games that hadn't been written yet then. I guess there must be a lot of software engineering that's about building weird proprietary business systems, but I've talked to very few (not zero, but few) people who've mentioned their workplace computer systems really getting noticeably better in recent years either. My workplace got a whole new academic reporting system and it's maybe one step per entry quicker than the old one but most of the effort seems to have gone into looks rather than functional improvement honestly. I guess the point here being: does coding faster actually solve anyone's problems/is that the actual bottleneck?

[Son of the King]

Another recent post in the "AI killed my ability to program well and now I'm stopping" genre:
https://blog.bustikiller.com/2026/09/25/one-month-without-ai.html

This was a good read. A lot of it resonated with me, although my personal use of LLMs has not been very extensive for various reasons.

A thing I do worry about is whether these are actually relatively isolated cases in an industry where very very few people are going to buck the trend. But these sorts of things are definitely persuading me that coding with LLMs just isn't something I want to do at all even if it does become the sole long-term norm for how people write computer code.

Lots of people in the industry are enthusiastically aboard the LLM train at this point. The viewpoint in this article isn't exactly uncommon, but in the last year or so it at least feels like its become more of a minority position.

My personal feeling has always been that LLMs will not fully replace writing code by hand, largely for the reasons outlined in this article. I've never been impressed by the quality of the output when it comes to code generation, although having said that (and I believe this is related to your second point) there is a large amount of similarly low quality code in production software already. LLM code often looks correct but is anecdotally much harder to review and catch bugs in, and often it contains particularly subtle bugs.

There is some utility in LLMs to assist in programming in my experience, none of which are actually using it to generate large amounts of code or "one shot" complex projects. It can be handy to review human-written code, and I think the fancy autocomplete mentioned in the article is also somewhat useful in terms of providing an actual speed improvement rather than just an apparent (but illusory) one. However, this level of usefulness doesn't justify the ethical minefield that LLMs are.

A thing I find odd, conversely, is how little any software I use has increased its utility to me in years. We now apparently have faster coding than ever, but I don't think my own use of any software has improved since 2010 except that I can now play some games that hadn't been written yet then. I guess there must be a lot of software engineering that's about building weird proprietary business systems, but I've talked to very few (not zero, but few) people who've mentioned their workplace computer systems really getting noticeably better in recent years either. My workplace got a whole new academic reporting system and it's maybe one step per entry quicker than the old one but most of the effort seems to have gone into looks rather than functional improvement honestly. I guess the point here being: does coding faster actually solve anyone's problems/is that the actual bottleneck?

Coding speed has never been the bottleneck in software development in my experience, which is why the fixation on "LLMs will make you code so quickly" has never made much sense to me.

I think some of the stagnation of software (which matches my experience on the whole, though some specific things I use have improved a lot in the last 15 years or so) is due to the change in where the money comes from. There's not much value in increasing utility for users, when the users don't pay anything to use your software. Advertising technology has advanced hugely in this same timeframe.

[Othko97]

In my experience observing LLM users (I've never touched one myself, save reviews foisted upon me), all "productivity gains" they get from velocity are lost into doing the low-importance tasks which LLMs help most with.  I think it's the same kind of pseudo-productivity that previously caused people to spend more time writing emails back and forth than doing the tasks that really "count."  I don't think that the actual throughput has really increased since these tools were adopted.  This is probably also related to writing code not being the bottleneck anyhow.

Advertising technology has advanced hugely in this same timeframe.

Has it?  Perhaps from 2010-2015, but I don't think there's been anything huge since then even there, although admittedly this is not something I know much about.  As far as I know, micro-targeting and recommender systems are still the basis, and this technology was around back then.  I'm also somewhat cynical of the claims that modern advertising is like mind control anyhow, it feels like criti-hype to me.

I agree that modern software isn't made for users though, but it's not for advertisers either.  Modern software is made to attract venture capitalists, eventually get acquired by a hyperscaler, and to get the founders enough money to go on a quest for immortality and/or Mars.

[psyanojim]

I've been thinking about this topic since the last pub too, especially the number of claims revolving around the idea that AI is somehow reaching the limits of its capabilities.

I don't want to dive into the rabbithole of the psychology of this (motivated reasoning, coping etc), and yes there is much hype, but it is also extremely dangerous to believe things are true simply because one wants them to be true. For example if ones livelihood depends on them being true.

In the last couple of weeks, there have been some quite astonishing announcements in the areas of mathematics and software (especially reverse engineering/clean room implementations).

Instead of writing an essay, I'll just link 3 YouTube videos from the last few days.

These are YouTubers I have been following for years, all very capable individuals, definitely not easily swayed by hype, and all in the category of what I would call 'realistic skeptics' who have been surprised by AI capabilities in recent videos.

Video 1: 'Has AI solved reverse engineering?' YouTuber 'LowLevel' uses AI to crack a complex firmware/encryption binary in 15 minutes, which he estimates would have taken him days/weeks without.

https://www.youtube.com/watch?v=wIe3eDfGKUo

Video 2: 'Can AI Make a Game Engine?' YouTuber 'The Cherno', who has been writing his own game engine called 'Hazel' for years, asks Claude and Astra to write game engines. You can almost see the existential crisis unfold in slow motion.

https://www.youtube.com/watch?v=R_uf5OfMGio

Video 3: 'The Mathematics are Mad' YouTuber 'The PrimeTime' looks at the dump of hundreds of AI-generated math proofs that are causing outrage and horror in the mathematical community.

https://www.youtube.com/watch?v=WXh4LF3zJ1Q

-----

The mathematics issue is of particular interest to me, since it starts to ask fundamental questions about the nature of knowledge such as 'intuition' vs 'brute force' in cognition.

Some have reacted with horror/depression/nihilism etc. Social media is suddenly awash with depressed mathematicians, but others have seen new possibilities.

For example from Video 3, a problem space which was seen as 'solved', that AI brute-forced a crude and tiny improvement to. Then once mathematicians realised an improvement was even possible, this reinvigorated interest which has already led to humans making further improvements on the AIs work.

What a fascinating world we inhabit.

[Jubal]

it starts to ask fundamental questions about the nature of knowledge such as 'intuition' vs 'brute force' in cognition.

Yeah, one of the things I've seen commented elsewhere is that a lot of these solutions are in a sense too ugly for humans to ever come up with them, the LLM just doesn't get bored trying possibilities.

I think my biggest point from last pub was that the "it's dangerous to believe what you want to be true" cuts both ways here, exacerbated by the lack of good external benchmarking. We have a tool that can do a big range of things pretty flexibly, but what it can do, what it can't do, and exactly how well it can do the things it can do are all being obfuscated from every possible direction. So we have some people who still want to believe that it can't do big maths breakthroughs and these results are all just built by "scooping" current mathematicians and will run out pretty shortly, but conversely you have people going "well if it can do all of this maths it'll be able to design us a room-temperature superconductor and replace the whole movie industry". And I don't think either of those positions are very well evidenced.

I agree that modern software isn't made for users though, but it's not for advertisers either.  Modern software is made to attract venture capitalists, eventually get acquired by a hyperscaler, and to get the founders enough money to go on a quest for immortality and/or Mars.

I am not enough of an economist but it feels like investment decisions have become increasingly divorced from most people' lives as systems are built for marketing to investors more than utility for users, and the users actually don't get a feedback mechanism on this because the purchasing decisions are made by someone in another department who doesn't know what their job is and has very little incentive for them to do it better.

I'd also say that advertising technology doesn't feel any better than a decade ago to me, but then the only social media I use regularly that tries to advertise much to me is Facebook and I guess it does get the ad targeting right sometimes? But it also e.g. spends a lot of time advertising the same businesses I'd have used/wouldn't use regardless for day to day stuff, or things where it's got part of it right with a fatal flaw (I am interested in politics but do not want far-right political ads, and I do have a hobbit aesthetic but have decidedly the wrong body shape for cottagecore dress shopping.)

[psyanojim]

"it's dangerous to believe what you want to be true" cuts both ways here

It really reminds me of the Dot-Com Boom/Bubble.

Some hype merchants thought that adding '.com' to the end of a company name would change everything. Pets .com!

There was another, often forgotten cohort who believed that the internet was a passing fad that would soon go away.

Both attitudes were dubious at the time and hilarious in hindsight.

-----

but what it can do, what it can't do, and exactly how well it can do the things it can do are all being obfuscated from every possible direction

This is one reason why I question whether I'm on the wrong forum sometimes. Tom Wolfes analogy of the 'Ready, Fire, Aim' mentality vs the 'Ready, Aim, Aim, Aim, Aim...' mentality.

I'm happy to analyze a problem, but looking for rigorous, peer-reviewed, perfect accuracy in a fast moving noisy environment like this simply means that the problem and any opportunities will be history before the analysis is complete.

Since I'm looking to make predictions and ultimately get ahead of the curve and adapt, this necessarily involves making decisions based on imperfect information. Mistakes will be made, corrections will be required. Nature of the beast.

Neither attitude being superior or inferior, but very different priorities.

-----

So a real anecdote - I received an email last week that a company I use is no longer serving the UK. I have been connecting to this company live via an API for the last decade or so, and I've now got 1 month notice to find a replacement.

So its a technical/business problem with a hard deadline and hence time pressure with severe consequences for failure to meet the deadline.

Evaluating and rewriting to connect to a bunch of competitor APIs, some REST, some in c#/java/python, is incredibly tedious work.

Not difficult. Just brain-dead, boilerplate and time-consuming.

So I pasted in some snippets of the existing API calls I was using to ChatGPT, and said - write me an adapter to these competitors APIs.

It probably would have taken me DAYS without AI, trawling through incomplete documentation and idiosyncratic code samples in multiple languages from multiple companies. ChatGPT did it in MINUTES.

So now, rather than rushing and panicking, I can evaluate multiple competitors in parallel at my leisure.

Again, just an anecdote, not a peer-reviewed data set, but it's my most successful use of AI so far, with a crystal clear cost/time benefit, and definitely has given me food for thought.

[Jubal]

I'm happy to analyze a problem, but looking for rigorous, peer-reviewed, perfect accuracy in a fast moving noisy environment like this simply means that the problem and any opportunities will be history before the analysis is complete.

Since I'm looking to make predictions and ultimately get ahead of the curve and adapt, this necessarily involves making decisions based on imperfect information. Mistakes will be made, corrections will be required. Nature of the beast.

Neither attitude being superior or inferior, but very different priorities.

Yeah, obviously here I'm someone whose background is in academia: understanding things, thinking about them rigorously without trading that off against speed, and informing people about them is a core part of why it makes sense for society to have jobs like mine. And that also then feeds into what sort of solutions-thinking each of us is doing: this is not for the most part a problem I need to solve personally, it's a problem I need more to think about systemically. I'm also in a field where "getting ahead" on any specific problem is barely a concern, because in this funding environment there's nobody left to get ahead of: I work pretty fast for a humanities academic in part because I'm in a non-teaching job, but I'm only in a race with myself and how much I can get done in one human lifetime. If I don't publish on something the effect is not that someone beats me to the punch, it's that the thing doesn't get studied in that particular way because there isn't anyone else. All of which is a long way to say that I agree - different priorities and different underlying logics.

That said, I think there's also a difference between imperfect information (which I think is what we have for questions like "how well can this thing really do maths") and near-zero information (which I think is closer to the case for questions like "how well can that ability generalise to fields where frontier labs haven't shown any major breakthroughs yet"). And on the macro-scale, I fear we're gambling much of the global economy's investment budget on the latter questions, which is a bit disconcerting. I totally agree with you about information speed at the macro level: I'm cautious because I'm lacking good information, but I shouldn't be lacking good information. One of my feelings on the whole thing is that academic policy has failed the moment, because governments or intergovernmental organisations should have set up and been hiring for independent benchmarking labs with decent levels of funding at least a year ago by this point to actually provide the sorts of cross-checks that would help businesses and science academia adapt better to the moment.

I think it's good that we have these discussions in this space, re the "being on the wrong forum" comment - for all of us with our different angles, being in a space where everyone has the exact same incentives to think the same way as you about a phenomenon is a good way to leave yourself without independent checks on your thinking, and this is one of those areas where, as we've discussed, checks to avoid oneself herding in any given direction are probably particularly important! And we usually manage to have these conversations without concluding that everyone else in the discussion is an idiot, that we're one week away from discovering immortality or that the world is about to end, which seem to be very common outcomes elsewhere on the internet. So I'm glad about that.