This is a must-read series of articles, and I think Kyle is very much correct.
The comparison to the adoption of automobiles is apt, and something I've thought about before as well. Just because a technology can be useful doesn't mean it will have positive effects on society.
That said, I'm more open to using LLMs in constrained scenarios, in cases where they're an appropriate tool for the job and the downsides can be reasonably mitigated. The equivalent position in 1920 would not be telling individuals "don't ever drive a car," but rather extrapolating critically about the negative social and environmental effects (many of which were predictable) and preventing the worst outcomes via policy.
But this requires understanding the actual limits and possibilities of the technology. In my opinion, it's important for technologists who actually see the downsides to stay aware and involved, and even be experts and leaders in the field. I want to be in a position to say "no" to the worst excesses of AI, from a position of credible authority.
> Just because a technology can be useful doesn't mean it will have positive effects on society.
You say it in a way that it sounds like automobiles don't have a positive effect. I don't agree - they have some negative effects but overall they have a vast net positive effect for everyone.
Their negative effects are much more vast, subtle, and cultural. You could say many of the broad and widespread mental issues we have in the US is the result of automobiles leading to suburbanization and thus isolation of people. It has created an expensive barrier of entry for existing in society and added a ton of friction to doing anything and everything, especially with people. That's not even getting into the climate effects.
The upsides of automobiles generally all exist outside of the 'personal automobile', i.e. logistics. These upsides and downsides don't need to coexist. We could reap the benefits without needing to suffer for it, but here we are.
> You could say many of the broad and widespread mental issues we have in the US is the result of automobiles leading to suburbanization and thus isolation of people.
Yes, you could say that, though I'm not sure who would actually say that seriously.
This is a willfully ignorant and wildly incorrect take. Your isolation argument completely neglects socialization with family and friends that is supported via automotive mobility. Do you also somehow have the impression that automobiles somehow forced suburbanization? I think not- you don't want others to have the freedom to choose anything other than some industrialized urban existence. The effects of the automobile are vast, subtle, and cultural- and overwhelmingly positive
The upsides of automobiles, or personal mobility in general, are enormous. I can go wherever I want, whenever I want along with other people and cargo. I don't have to wait for a schedule set by someone else, or worry about union strikes. I love my cars!
this is so funny, even when trying to talk positively about cars you cant help but throw in a 'fuck you, I got mine'. Unions are cool, and good for workers. Enjoy your weekend! Thank a union.
Those are all enormous benefits to you and you alone. The greatest thing about cars are the things they do for you.
In order for someone else to have those benefits, they also need a car.
If as a society, if we could feel the same way about public transit, bike lanes, sidewalks, that you do about your own personal vehicle - we'd be better off.
I'm hardly alone, there are millions and millions of us. But the HN bubble skews toward affluent childless male urbanites, so discussions here tend to be weirdly disconnected from the real world that regular middle-class Americans experience.
They have a net positive effect for every owner, except that they seem to facilitate and encourage ways of living that require automobile ownership as a condition of adulthood in most places. So I'm not entirely sure they're a vast net positive in every value system. In yours, yes, but not in mine.
There is very little connection between ownership and who does the driving. I still want to own my own cars even if a computer does most of the driving. That way it's always available, and more importantly I can keep my own stuff in it.
This is such a fascist take- "they seem to facilitate and encourage ways of living that require automobile ownership as a condition of adulthood" i.e. I don't agree with that way of living so I wish others didn't have the freedom to choose it
It's not fine if that choice denies other people the choice not to.
And there seems to be a lot of the latter.
For example, when shopping facilities or hospitals are built so as to be, de-facto, only accessible by automobile, that locks people out of the choice to say no thanks.
Automobiles are one of a key pillar of logistics. Getting things (food, medicine, construction materials, etc. etc.) to and from backbones like rail, harbors, airports etc. So even for those who don't own a vehicle or even want to own a vehicle, automobiles are still a vast net positive.
I'm not sure what the alternative would be. Maybe everyone lives in giant 10 million+ population cities that are all connected to each other by rail (and rail connects all airports, harbors, etc.) and then you have to show up at rail station to get your groceries or whatever else?
Personal cars are not the same as using them for logistics.
Yes cars/trucks/busses are still useful overall and are an incredible last-mile solution for freight.
But on a personal level, it means we all must live far apart and maintain our own individual vehicles, along with the average total costs of $11,500/year PER CAR. [0]
I’m not saying they should’ve even been banned for personal use - owning a car and living in a rural suburb should still be an option, but it is very expensive to choose that lifestyle.
However the auto companies on the early to mid 1900s had heavy influence on policy, even buying and shutting down their public transit competitors, converting cities into “car cities”. This is where it drove into “negatives outweigh the positives” territory. Everything before that was more positive, but this was a massive negative on society and continues to handicap cities today, making them expensive and even just dangerous to walk around (due to high speed roads and limited sidewalks)
You don't get highways and the interstate system if vehicles are not for personal use. And if you don't get those, you don't get the modern logistics system.
I guess what I don't understand is, given the current state, 1) what do you want? 2) how much will it cost? (and how will we pay for it?) and 3) what are the tradeoffs?
On a related note, it seems like a lot of the anti-car/urban planning wonks have a belief that everyone really wants to walk, ride bikes, or take mass transit everywhere, and I think they're wrong. Most people want to drive personal vehicles.
Maybe if we lived in a world where mass transit had very strictly enforced behavioral norms, more would consider it. But even then, I still think most people prefer the many conveniences afforded by personal vehicles.
> Personal cars are not the same as using them for logistics.
Yes, they are in fact, the same. You cannot introduce such massively useful technology into the world and then say that it would be used only for logistics and not for personal transportation. Short of a worldwide totalitarian government, such arbitrary restriction would be completely unenforceable.
It is possible to shape things with regulation, but only to some degree. With any great technology, you have to take the good with the bad. And the good outweights the bad in any historical technology. AI will be no exception.
Exactly. These arguments are all buttressed by the "if everyone would just..." argument [1]. In fact, everyone will not just. And so if you want to build your Utopia, it will have to be compelled by force.
I've always lived in walkable cities. I don't own a car and with pollution, congestion, accident risk, pavement obstruction, etc. other people's cars unequivocally make my life worse.
We can argue about whether this is a good trade off, but the claim that cars make everyone's life better is straightforwardly false.
Troll post? No, they do not "unequivocally" make your life worse. "Other people's cars" facilitate thousands of aspects of modern living and society that you apparently take for granted. You can choose to ONLY look at the negative impacts, but the comment as stated is ridiculous.
The only way you receive food (except from your backyard inner-city garden?) is through people DRIVING. The way you receive packages is by DRIVING. They city infrastructure you enjoy is maintained through skilled laborers and tradespeople DRIVING.
Troll post? You state that "other people's cars" facilitate thousands of aspects of modern living, then go on to talk about things that trucks do, not personal vehicles
I don't think it's possible to clearly separate personal vehicles from commercial ones. The technology is the same. Any regulation that tries to ban the one while allowing the other would be a huuuge clusterfuck.
> "Other people's cars" facilitate thousands of aspects of modern living and society that you apparently take for granted. You can choose to ONLY look at the negative impacts, but the comment as stated is ridiculous.
THIS! I am shocked that some people don't realize that modern civilization and our modern quality of life depends on cars to a huge degree, even for people don't personally drive. Such a lack of knowledge about modern industry and logistics..
In aggregate, benefits of cars outweight the cons for 99% of people. Perhaps if you live right next to a busy highway, you might the the exception..
There's a difference between personal vehicles and special purpose vehicles like ambulances and delivery trucks. I don't think anyone in this thread is saying all automobiles are bad, but car-centric development is definitely bad. You don't have to theorize from first principles about this. There are many places around the world that aren't as locked into the personal car as the US is, and they are still functioning societies where you can receive food, packages, medicine, workers maintain infrastructure, etc.
I think it's most obvious in hindsight, probably it was a long time (some decades) before cars were understood to have much of a negative effect at all. Nobody* thought much about air pollution (even adding lead to the gasoline) or climate effects, or what would happen when cities were built enough that they were then dependent on cars, or what happens when gas or cars gets expensive.
All they saw was that trips taking a day could now be done in an hour and produced no manure, and that meant suddenly you could reasonably go to many more places. What's not to like? A model T was cheap, and you didn't even need to worry about insurance or having a driver's license. Surely nobody would drive so carelessly as to crash.
*well, not technically nobody, but nobody important.
If you read the period news, pretty much everything except lead poisoning and climate change was well known by the 1920s. Rich people wanted cars but a ton of places had resistance from everyone else to what they correctly recognized as removing the public spaces they used and shifting externalities to, for example, the people being hit by cars.
What’s really interesting is that you can find newspaper columns in the 1920s recognizing what we now call induced demand as even by then it was clear that adding road capacity simply inspired more people to drive.
The problem is we are numb to it. 40,000+ people are killed in car accidents every year in just the USA. Wars are started over oil and accepted by the people so they can keep paying less at the pump. Microplastics entering the environment each day along with particulate from brakes, and exhaust. Speaking of exhaust: global warming. Even going electric just shifts the problems as we need to dig up lithium, the new oil. We still have to drill for oil for plastics and metal refining, recycling and fabrication.
Cars I'd argue are a net negative for everyone. In the article it goes over this pretty well.
The automobile was a revolutionary tool, but I think it has been overprescribed as a solution for the problem of transportation.
The grips of capitalism and consumerism have allowed for automobiles to become a requirement for living nearly everywhere in America except for the densest of areas.
I love cars, I enjoy working on them, driving them, the way they look, the way they sound and feel. They do offer a freedom that is unparalleled, and offer many benefits to those who truly need those guarantees.
Ultimately, to me they are a symbol of toxic individualism. I would be happy if we could move on from them as a society.
This reminds me a bit of the ending of In the Beginning Was the Command Line:
> The people who brought us this operating system would have to provide templates and wizards, giving us a few default lives that we could use as starting places for designing our own. Chances are that these default lives would actually look pretty damn good to most people, good enough, anyway, that they'd be reluctant to tear them open and mess around with them for fear of making them worse. So after a few releases the software would begin to look even simpler: you would boot it up and it would present you with a dialog box with a single large button in the middle labeled: LIVE. Once you had clicked that button, your life would begin. If anything got out of whack, or failed to meet your expectations, you could complain about it to Microsoft's Customer Support Department. If you got a flack on the line, he or she would tell you that your life was actually fine, that there was not a thing wrong with it, and in any event it would be a lot better after the next upgrade was rolled out. But if you persisted, and identified yourself as Advanced, you might get through to an actual engineer.
> What would the engineer say, after you had explained your problem, and enumerated all of the dissatisfactions in your life? He would probably tell you that life is a very hard and complicated thing; that no interface can change that; that anyone who believes otherwise is a sucker; and that if you don't like having choices made for you, you should start making your own.
I fear that outside of cataclysmic global warfare or some sort of butlerian jihad (which amounts to the same) this genie is not going back into the bottle.
This tech is 100% aligned with the goals of the 0.001% that own and control it, and almost all of the negatives cited by Kyle and likeminded (such as myself) are in fact positives for them in context of massive population reduction to eliminate "useless eaters" and technological societal control over the "NPCs" of the world that remain since they will likely be programmed by their peered AI that will do the thinking for them.
So what to do entirely depends on whether you feel we are responsible to the future generations or not. If the answer is no, then what to do is scoped to the personal concerns. If yes, we need a revolution and it needs to be global.
It can't. It can't even deal with emails without randomly deleting your email folder [1]. Saying that it can make decisions and replace humans is akin of saying that random number generator can make decisions and can replace people.
It's just an automation tool, and just like all automation tools before it it will create more jobs than destroy. All the CEOs' talks about labor replacement are a fuss, a pile of lies to justify layoffs and worsening financial situation.
Yes, but that isn’t the question as long as those wealthy people control most of the system: companies aren’t going to lose executives, they’ll shed the jobs which they don’t respect. Someone wealthy does not need to accept a bad deal to avoid sleeping on thr street. It’s everyone who isn’t insulated who has to actually compete for work.
I'm tempted to (bitterly) point out that feeling responsible for future generations was already off the table decades ago when we decided to ignore our ecological footprints.
Not to collapse the economic system, but to present a credible threat of collapsing the economic system which AI development, as these elite and their platforms know it, relies on. When they're freaking out, we call for negotiations.
This only works if people with "secure" livelihoods not just participate, but drive the effort. Getting paid six figures or more in a layoff-proof position? Cool, you need to be the first person walking out the door on May 1st (or whenever this happens), and the first person at the bank counter requesting your max withdrawal.
You're free to take a vacation or quit working if you want to. Go ahead.
As for bank runs, no one cares. The big banks no longer need retail customer deposits as a source of capital for fractional reserve lending. Modern bank funding mechanisms are more sophisticated than that.
Geopolitical realities and considerations require that the effort is synchronized and global. Assume great power X's society revolts and decides to reign in the financial and technological barons and lords, and do away with such things. Meanwhile, great powers Y, Z etc. are not doing this and one day people in X will wake up to AI drone swarms of these powers taking them over and they're back to square 1 and now not even a great power.
Collective humanity needs to think this matter through and take global action. This is the only way I fear, short of natural calamities (act of God) that unplugs humanity from advanced tech for a few generations again.
To take the car analogy: it matters how we use the car.
The car in itself can be used to save time and energy that would otherwise be used to walk to places. That extra time and energy can be used well, or poorly.
- It can be squandered by having a longer commute that defeats the point
- Alternatively, it can be wasted by sitting on a couch consuming Netflix or TikTok
- Alternatively, it can be used productively, by playing team sports with friends, or chasing your kids through the park, or building a chicken coop in your back yard
It’s all about wise usage. Yes it can be used as a way to destroy your own body and waste your time and attention, but also it can be used as a tool to deploy your resources better, for example in physical activities that are fun and social rather than required drudgery.
I think it’s the same for LLMs. Managers and executives have always delegated the engineering work, and even researching and writing reports. It matters whether we find places to continue to challenge and deploy our cognition, or completely settle back, delegate everything to the LLM and scroll TikTok while it works.
I agree with you that the majority of people will use it to feed their attention and energy to the attention economy.
But the majority have always chosen the path of least resistance. This is not new! Socrates’ famous exhortation is “the unexamined life is not worth living”.
I think if you want to give a call to action, as this piece does, the right call to action is “think carefully about how you can make a good use of your time and energy, now that the default path has changed.” I know it’s not as simple or emotionally powerful as “go down kicking and screaming, stick it to the man”, but as a rule of thumb, the less fiercely emotional path is usually the right one.
Two years ago, I was enjoying a drink with my wite, her friend, a very senior female VC partner, and another friend.
Somehow we talked AI in some depth, and the VC at one point said (about AI): “I don’t know what are kids are going to do for work. I don’t know what jobs there will be to do.”
That same VC invests in AI companies and by what I heard about her, has done phenomenally well.
I think about that exchange all the time. Worried about your own kids but acting against their interests. It unsettled me, and Kyle’s excellent articles brought that back to a boiling point in my mind.
I agree with the general sentiment that the structure of society is going to change, but I don't know what the satisfying solution is. It's hard to imagine not participating will work, or even be financially viable for me, for long.
I agree. I'm the AI luddite on my team of red team security engineers, but I'm still using it in very limited use cases. As much as I disagree with how the guardrails around AI are being handled, I still need to use it to stay relevant in my field and not get canned.
I'm using claude but then refuse to do much cleaning up of what it spews. Im leaving that for the PR reviewers who love AI and going through slop. If they want slop, I'll give them the slop they want.
Not advocating that people should follow this but:
As someone that loves cleaning up code, I'm actually asking the vibe coders in the team (designer, PM and SEO guy) to just give me small PRs and then I clean up instead of reviewing. I know they will just put the text back in code anyway, so it's less work for me to refactor it.
With a caveat: if they give me >1000 lines or too many features in the same PR, I ask them to reduce the scope, sometimes to start from scratch.
And I also started doing this with another engineer: no review cycle, we just clean up each other's code and merge.
I'm honestly surprised at how much I prefer this to the traditional structure of code reviews.
Additionally, I don't have to follow Jira tickets with lengthy SEO specs or "please change this according to Figma". They just the changes themselves and we go on with our lives.
Just started work on a project. Greenfield and "AI accelerated". PRs diffs are in the range of 10s of thousands of lines. In the PR, it is suggested to not actually read all the code as it would take too long.
If you push a change, or you approve, you're responsible for the change and its effects later. Regardless of size. If change is too big, tell your teammates its too big to review and to refactor to bite-size with their great coding agents. Use AI models also for review of large changes, consider a checklist . Setup CI and integration tests (also can be AI assisted)
based. our CEO has made it clear that we're expected to use LLMs to shit out as many features as we can as quickly as we can, so that's exactly what I'm doing. Can't wait to watch leadership flail around in a year or two when the long term consequences start to become apparent
That's exactly it. This person does not understand the coercive competition of the market. If you don't use new tech, you are going to be undercut by people who do. And every HR dept is going to expect to to have experience with AI even if the department that’s hiring doesn't really use it. If the author's supposed solution to the problem has negative personal consequences, why would you do it? To be nice?
Because I don't like the feeling my conscience gives me by doing something I think is evil and bad. Some people have moral lines that they won't cross when finding jobs.
If my competitors are filling their flour with sawdust, guess I got to just do the same?
No. I'm doing it because I care more whether I can live with myself than whether I impress people with the name of who I work for. Hence my recent comment history, for example. I don't want any of these people getting the idea they should want me to work with them, either. I do want my name on every industry blacklist I can possibly get it on. Those will eventually be revealed - remember Franklin's dictum, fellas! That shit always comes out in the end - and I look forward to that day with pleased and eager anticipation.
At the moment I'm more looking at menial work for one of the local universities. Money is money, and my needs are small; the work is honest, I still should have a decade or so of physical labor left in me, and it carries the perk of free tuition for the degree I never had time for. I would have the time and energy to write, perhaps, even! And, however badly the people in charge are running things lately, the world will always need someone good at cleaning a toilet.
I've been thinking about this a lot recently, and I don't know if it is possible to stop. I've been thinking the most impactful thing would be to create open-source tools to make it easier to build agents on top of open-source models. We have a few open-source models now, maybe not as good as Gemini, but if the agent were sufficiently good, could that compensate?
I think that would democratize some of the power. Then again, I haven't been super impressed with humanity lately and wonder if that sort of democratization of power would actually be a good thing. Over the last few years, I've come to realize that a lot of people want to watch the world burn, way more than I had imagined. It is much easier to destroy than to build. If we make it easier for people to build agents, is that a net positive overall?
> If we make it easier for people to build agents, is that a net positive overall?
If we make it easier for people to drive and have cars, isn't that a net positive? If we make it easier for X, isn't that better? No, not necessarily, that's the entire point of this series of essays. Friction is good in some cases! You can't learn without friction. You can't have sex without friction.
> ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand: the cultivation of what James C. Scott would call
Imagine being starting university now... I can't imagine to have learned what I did at engineering school if it wasn't for all the time lost on projects, on errors. And I can't really think that I would have had the mental strength required to not use LLMs on course projects (or side projects) when I had deadlines, exams coming, yet also want to be with friends and enjoy those years of your life.
The epilogue looked weak to me. The previous sections explored why it was essentially wrong to use current LLM technology, the answers can be wrong, or not even wrong, and why it has to be that way. The epilogue focus more in (our) obsolescence in a paradigm shift towards widespread LLM use scenario and not in them doing their work right or wrong.
And that should be the core. There is a new, emergent technology, should we throw everything away and embrace it or there are structural reasons on why is something to be taken with big warning labels? Avoiding them because they do their work too well may be a global system approach, but decision makers optimize locally, their own budget/productivity/profit. But if they are perceived risks, because they are not perfect, that is another thing.
The reasons laid out in this article are why it's so important to share how we are using AI and what we are getting in return. I've been trying to contribute towards a positive outcome for AI by tracking how well the big AI companies are doing at being used to solve humanitarian problems. I can't really do most of the suggestions the article, they seem like a way to slow progress. I don't want to slow AI progress, I want the technology we already have to be deployed for useful and helpful things.
The comparison to automobiles changing streets is thrown around a lot. But I feel AI is fundamentally different. It is not a technological change like the internet which brought us huge amounts of opportunities in so many different directions. AI’s goal is to automate (in other words, replace) us.
As a consequentialist who shares the author's concerns, I feel fine (ethically) using AI without advancing it. Foregoing opportunities meaningful to yourself for deontological reasons when it won't have any impact on society is pointless.
the epilogue is what speaks to me most. all of the work I've done with llms takes that same kind of approach. I never link them to a git repo and I only ever ask them to make specific, well-formatted changes so that I can pick up where they left off. my general feelings are that LLMs make the bullshit I hate doing a lot easier - project setup, integrate themeing, prepare/package resources for installability/portability, basic dependency preparation (vite for js/ts, ui libs for c#, stuff like that), ui layout scaffolding (main panel, menu panel, theme variables), auto-update fetch and execute loops, etc...
and while I know they can do the nitty gritty ui work fine, I feel like I can work just as fast, or faster, on UI without them than I can with them. with them it's a lot of "no, not that, you changed too much/too little/the wrong thing", but without them I just execute because it's a domain I'm familiar with.
So my general idea of them is that they are "90% machines". Great at doing all of the "heavy lifting" bullshit of initial setup or large structural refactoring (that doesn't actually change functionality, just prepares for it) that I never want to do anyway, but not necessary and often unhelpful for filling in that last 10% of the project just the way I want it.
of course, since any good PM knows that 90% of the code written only means 50% of the project finished (at best), it still feels like a hollow win. So I often consider the situation in the same way as that last paragraph. Am I letting the ease of the initial setup degrade my ability to setup projects without these tools? does it matter, since project setup and refactoring are one-and-done, project-specific, configuration-specific quagmires where the less thought about fiddly perfect text-matching, the better? can I use these things and still be able to use them well (direct them on architechture/structure) if I keep using them and lose grounded concepts of what the underlying work is? good questions, as far as I'm concerned.
Rudolph built his engine, Henry built his car, Popular Mechanics published it. 2000 biofueling stations across the nation. All made illegal by special interests months before the article was published. Information didn't move fast enough to let the editors know that innovation was illegal.
From the article: "I’ve thought about this a lot over the last few years, and I think the best response is to stop. ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand: the cultivation of what James C. Scott would call metis."
"What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune
> ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand
On one hand I intuitively think this is correct, on the other hand these very concerns about technology have been around since the invention of... writing.
Here is an excerpt of Socrates speaking on the written word, as recorded in Plato's dialogue Phaedrus - "For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom"
And you know, Socrates was right. We did lose our memory with writing! How many phone numbers do you remember now that you have a phonebook in your phone? Humans will lose skills due to LLMs. That's just obvious on its face by the fact that if you don't do a skill regularly, you will lose it (or lose to do it as well as you once had).
There's a distinction to be made between "worse off" and "worse". Socrates was arguing that writing-users would be worse as people, not that they would experience lives they didn't like as much.
> "What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune
I always preferred this take:
“Civilization advances by extending the number of important operations which we can perform without thinking of them.”
― Alfred North Whitehead
It's both opposite and complementary to your Frank Herbert quote.
I think it's important that we recognize and understand how those operations are being done, and ignorance of the complexity of all the parts of our lives leads to the death of expertise. People who would learn a lot just from reading the course description of a 100 level class in a field are assuming their lack of knowledge means there's no complexity there.
> “There is a cult of ignorance in the United States, and there has always been. The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.” ― Isaac Asimov
The easier society makes it to be unaware of the complexity of everything around us, the easier it becomes to assume everything is actually as simple as their surface-level understanding.
Also Frank Herbert: "Once, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."
I mean, people are talking about the butlerian jihad without any sense of irony or subtext. Dune is literally a feudal hellscape that takes place in the wake of that event. It didn’t make things better. Lmao
The idea that Claude might be able to help you change the color of your led lighting as a legitimate counter to things like a less usable world wide web, worse government services, the loss of human ability, etc. is excellent parody.
I read that as an example of how we're seduced into using things - we start small because surely this one small thing won't hurt. And then it becomes one more thing. And one more. It'll start with him using it to change the color of his lights and 5 years from now AI will be embedded in his life.
completely fair, and I agree. but let's talk 6 months/a year down the line - when a local LLM will be able to offer what claude code does only slower and a smaller context window. then do you whip out the local llm to handle the project, or is it still objectionable?
The front page is currently home to the announcement of Qwen 3.6 35B, which has comparable performance to the flagship coding models of a few months ago, and can be run at home by those with a gaming computer or MBP from the last five years. It is happening, but there will always be some lag.
Yes, but every time the capabilities, security, accuracy, or any other quality of LLMs is challenged, the default answer is that we'll essentially have AGI in a quarter or two. It's very tiring to try to argue with people about current quality, when the argument is always to wait and/or pay for a super expensive model.
That's not what the grandparent poster was saying, but sure. They have been steadily improving across those metrics, as Opus 4.6 / 4.7 / Mythos demonstrate. They're certainly not perfect, and I understand your fatigue (it is certainly fatiguing to follow, even if interested!), but each new release pushes it that bit further, and the improvements percolate downwards to the cheaper models.
The Industrial Revolution - the greatest thing ever to happen - required the British govt to deploy more troops against Luddites than they had fighting Napoleon at the same time.
Damaging machinery was made a capital offense and they had dozens of executions, hundreds of deportations.
At every stage, the steady progress of civilization is fragile and in danger of being suffocated. Its opponents cloak themselves in moral righteousness, call themselves luddites, the green party, or AI safety rationalists. Its all the same corrosive thing underneath.
This kind of black and white moral thinking is corrosive to one's intelligence. You're allowed to talk about who benefits from massive society change and who suffers. You are allowed to talk about the ways that technology is implemented and how that leads to pros and cons. An attitude of "if we ever stop moving forward and think then the evil bad people win" is deeply anti-intellectual.
I read couple of articles in the series and I still couldn't get what was the point author is trying to make. Reads like, "let me give you 100 arguments why I think this is bad".
Do LLMs lie? Of course not, they are just programs. Do the make mistakes or get the facts wrong? Of course they do, not more often then a human does. So what is the point of that article? Why my future is particularly bad now because of LLMs?
The argument isn't that LLMs are bad because they can hallucinate. Author (clearly) argues that LLM use has negative cognitive effects on their users and on society as a whole. Plus, the technology would wipe out a large, large number of jobs.
How can you argue they don't lie, as if they have any idea of correct vs wrong? There is no brain there. When statistics overwhelmingly say "yes" is the correct answer to something, it will say "yes" -- completely independent of whether that's the correct answer.
Complaining about AI slop is starting to become its own kind of slop. There isn't anything novel in this little essay. It might as well have been written by AI because I've seen this type of dude complain about this exact type of thing countless times at this point, and none of them have a solution other than empty moralizing or call your representative or whatever. None of that’s going to work. Fortune, Gizmodo, The Verge,Ars Technica, etc. all circulate the same negative headlines and none of them have a solution, and their writers are probably going to be totally replaced by AI so what difference does it make? They're just capitalizing on the negative sentiment and they have no intention to come up with a solution. At that point it's just complaining and I'm sick of it.
Agreed, and I think if you asked most people in the developed world, they'd say the invention of automobiles has been a net positive (to say the least) despite all the very real negatives. Stopped reading the article after that. It seems like the people expressing these sentiments are a loud minority, and I know from having spent way too much time online that if LLMs didn't exist in their current form, they'd be angry about something else. Then again, Maybe I'm just out of touch. It's a distinct possibility.
Frankly I think it’s kind of childish to just put up a massive Uk wide block on your website. “Call your representatives”, ok dude, can I give you a list of things I want to change about your country’s policies?
A statement that can be reversed onto the speaker without effort is meaningless. It has no content. It just means, "I am rational and you are not." Ok, then.
The comparison to the adoption of automobiles is apt, and something I've thought about before as well. Just because a technology can be useful doesn't mean it will have positive effects on society.
That said, I'm more open to using LLMs in constrained scenarios, in cases where they're an appropriate tool for the job and the downsides can be reasonably mitigated. The equivalent position in 1920 would not be telling individuals "don't ever drive a car," but rather extrapolating critically about the negative social and environmental effects (many of which were predictable) and preventing the worst outcomes via policy.
But this requires understanding the actual limits and possibilities of the technology. In my opinion, it's important for technologists who actually see the downsides to stay aware and involved, and even be experts and leaders in the field. I want to be in a position to say "no" to the worst excesses of AI, from a position of credible authority.
You say it in a way that it sounds like automobiles don't have a positive effect. I don't agree - they have some negative effects but overall they have a vast net positive effect for everyone.
The upsides of automobiles generally all exist outside of the 'personal automobile', i.e. logistics. These upsides and downsides don't need to coexist. We could reap the benefits without needing to suffer for it, but here we are.
Yes, you could say that, though I'm not sure who would actually say that seriously.
In order for someone else to have those benefits, they also need a car.
If as a society, if we could feel the same way about public transit, bike lanes, sidewalks, that you do about your own personal vehicle - we'd be better off.
It's not fine if that choice denies other people the choice not to.
And there seems to be a lot of the latter.
For example, when shopping facilities or hospitals are built so as to be, de-facto, only accessible by automobile, that locks people out of the choice to say no thanks.
I'm not sure what the alternative would be. Maybe everyone lives in giant 10 million+ population cities that are all connected to each other by rail (and rail connects all airports, harbors, etc.) and then you have to show up at rail station to get your groceries or whatever else?
Yes cars/trucks/busses are still useful overall and are an incredible last-mile solution for freight.
But on a personal level, it means we all must live far apart and maintain our own individual vehicles, along with the average total costs of $11,500/year PER CAR. [0]
I’m not saying they should’ve even been banned for personal use - owning a car and living in a rural suburb should still be an option, but it is very expensive to choose that lifestyle.
However the auto companies on the early to mid 1900s had heavy influence on policy, even buying and shutting down their public transit competitors, converting cities into “car cities”. This is where it drove into “negatives outweigh the positives” territory. Everything before that was more positive, but this was a massive negative on society and continues to handicap cities today, making them expensive and even just dangerous to walk around (due to high speed roads and limited sidewalks)
[0] https://www.nerdwallet.com/auto-loans/learn/total-cost-ownin...
I guess what I don't understand is, given the current state, 1) what do you want? 2) how much will it cost? (and how will we pay for it?) and 3) what are the tradeoffs?
On a related note, it seems like a lot of the anti-car/urban planning wonks have a belief that everyone really wants to walk, ride bikes, or take mass transit everywhere, and I think they're wrong. Most people want to drive personal vehicles.
Maybe if we lived in a world where mass transit had very strictly enforced behavioral norms, more would consider it. But even then, I still think most people prefer the many conveniences afforded by personal vehicles.
Yes, they are in fact, the same. You cannot introduce such massively useful technology into the world and then say that it would be used only for logistics and not for personal transportation. Short of a worldwide totalitarian government, such arbitrary restriction would be completely unenforceable.
It is possible to shape things with regulation, but only to some degree. With any great technology, you have to take the good with the bad. And the good outweights the bad in any historical technology. AI will be no exception.
[1] https://x.com/eperea/status/1803815983154434435
We can argue about whether this is a good trade off, but the claim that cars make everyone's life better is straightforwardly false.
The only way you receive food (except from your backyard inner-city garden?) is through people DRIVING. The way you receive packages is by DRIVING. They city infrastructure you enjoy is maintained through skilled laborers and tradespeople DRIVING.
THIS! I am shocked that some people don't realize that modern civilization and our modern quality of life depends on cars to a huge degree, even for people don't personally drive. Such a lack of knowledge about modern industry and logistics..
In aggregate, benefits of cars outweight the cons for 99% of people. Perhaps if you live right next to a busy highway, you might the the exception..
All they saw was that trips taking a day could now be done in an hour and produced no manure, and that meant suddenly you could reasonably go to many more places. What's not to like? A model T was cheap, and you didn't even need to worry about insurance or having a driver's license. Surely nobody would drive so carelessly as to crash.
*well, not technically nobody, but nobody important.
What’s really interesting is that you can find newspaper columns in the 1920s recognizing what we now call induced demand as even by then it was clear that adding road capacity simply inspired more people to drive.
The problem is we are numb to it. 40,000+ people are killed in car accidents every year in just the USA. Wars are started over oil and accepted by the people so they can keep paying less at the pump. Microplastics entering the environment each day along with particulate from brakes, and exhaust. Speaking of exhaust: global warming. Even going electric just shifts the problems as we need to dig up lithium, the new oil. We still have to drill for oil for plastics and metal refining, recycling and fabrication.
The automobile was a revolutionary tool, but I think it has been overprescribed as a solution for the problem of transportation.
The grips of capitalism and consumerism have allowed for automobiles to become a requirement for living nearly everywhere in America except for the densest of areas.
I love cars, I enjoy working on them, driving them, the way they look, the way they sound and feel. They do offer a freedom that is unparalleled, and offer many benefits to those who truly need those guarantees.
Ultimately, to me they are a symbol of toxic individualism. I would be happy if we could move on from them as a society.
> The people who brought us this operating system would have to provide templates and wizards, giving us a few default lives that we could use as starting places for designing our own. Chances are that these default lives would actually look pretty damn good to most people, good enough, anyway, that they'd be reluctant to tear them open and mess around with them for fear of making them worse. So after a few releases the software would begin to look even simpler: you would boot it up and it would present you with a dialog box with a single large button in the middle labeled: LIVE. Once you had clicked that button, your life would begin. If anything got out of whack, or failed to meet your expectations, you could complain about it to Microsoft's Customer Support Department. If you got a flack on the line, he or she would tell you that your life was actually fine, that there was not a thing wrong with it, and in any event it would be a lot better after the next upgrade was rolled out. But if you persisted, and identified yourself as Advanced, you might get through to an actual engineer.
> What would the engineer say, after you had explained your problem, and enumerated all of the dissatisfactions in your life? He would probably tell you that life is a very hard and complicated thing; that no interface can change that; that anyone who believes otherwise is a sucker; and that if you don't like having choices made for you, you should start making your own.
This tech is 100% aligned with the goals of the 0.001% that own and control it, and almost all of the negatives cited by Kyle and likeminded (such as myself) are in fact positives for them in context of massive population reduction to eliminate "useless eaters" and technological societal control over the "NPCs" of the world that remain since they will likely be programmed by their peered AI that will do the thinking for them.
So what to do entirely depends on whether you feel we are responsible to the future generations or not. If the answer is no, then what to do is scoped to the personal concerns. If yes, we need a revolution and it needs to be global.
It can't. It can't even deal with emails without randomly deleting your email folder [1]. Saying that it can make decisions and replace humans is akin of saying that random number generator can make decisions and can replace people.
It's just an automation tool, and just like all automation tools before it it will create more jobs than destroy. All the CEOs' talks about labor replacement are a fuss, a pile of lies to justify layoffs and worsening financial situation.
[1] https://www.pcmag.com/news/meta-security-researchers-opencla...
If AI is smart enough to replace the 99.999% it's also smart enough to replace the 0.001%.
Energy. The key is controlling their access to energy.
This only works if people with "secure" livelihoods not just participate, but drive the effort. Getting paid six figures or more in a layoff-proof position? Cool, you need to be the first person walking out the door on May 1st (or whenever this happens), and the first person at the bank counter requesting your max withdrawal.
As for bank runs, no one cares. The big banks no longer need retail customer deposits as a source of capital for fractional reserve lending. Modern bank funding mechanisms are more sophisticated than that.
Collective humanity needs to think this matter through and take global action. This is the only way I fear, short of natural calamities (act of God) that unplugs humanity from advanced tech for a few generations again.
To take the car analogy: it matters how we use the car.
The car in itself can be used to save time and energy that would otherwise be used to walk to places. That extra time and energy can be used well, or poorly.
- It can be squandered by having a longer commute that defeats the point
- Alternatively, it can be wasted by sitting on a couch consuming Netflix or TikTok
- Alternatively, it can be used productively, by playing team sports with friends, or chasing your kids through the park, or building a chicken coop in your back yard
It’s all about wise usage. Yes it can be used as a way to destroy your own body and waste your time and attention, but also it can be used as a tool to deploy your resources better, for example in physical activities that are fun and social rather than required drudgery.
I think it’s the same for LLMs. Managers and executives have always delegated the engineering work, and even researching and writing reports. It matters whether we find places to continue to challenge and deploy our cognition, or completely settle back, delegate everything to the LLM and scroll TikTok while it works.
But the majority have always chosen the path of least resistance. This is not new! Socrates’ famous exhortation is “the unexamined life is not worth living”.
I think if you want to give a call to action, as this piece does, the right call to action is “think carefully about how you can make a good use of your time and energy, now that the default path has changed.” I know it’s not as simple or emotionally powerful as “go down kicking and screaming, stick it to the man”, but as a rule of thumb, the less fiercely emotional path is usually the right one.
Somehow we talked AI in some depth, and the VC at one point said (about AI): “I don’t know what are kids are going to do for work. I don’t know what jobs there will be to do.”
That same VC invests in AI companies and by what I heard about her, has done phenomenally well.
I think about that exchange all the time. Worried about your own kids but acting against their interests. It unsettled me, and Kyle’s excellent articles brought that back to a boiling point in my mind.
As someone that loves cleaning up code, I'm actually asking the vibe coders in the team (designer, PM and SEO guy) to just give me small PRs and then I clean up instead of reviewing. I know they will just put the text back in code anyway, so it's less work for me to refactor it.
With a caveat: if they give me >1000 lines or too many features in the same PR, I ask them to reduce the scope, sometimes to start from scratch.
And I also started doing this with another engineer: no review cycle, we just clean up each other's code and merge.
I'm honestly surprised at how much I prefer this to the traditional structure of code reviews.
Additionally, I don't have to follow Jira tickets with lengthy SEO specs or "please change this according to Figma". They just the changes themselves and we go on with our lives.
If my competitors are filling their flour with sawdust, guess I got to just do the same?
At the moment I'm more looking at menial work for one of the local universities. Money is money, and my needs are small; the work is honest, I still should have a decade or so of physical labor left in me, and it carries the perk of free tuition for the degree I never had time for. I would have the time and energy to write, perhaps, even! And, however badly the people in charge are running things lately, the world will always need someone good at cleaning a toilet.
I think that would democratize some of the power. Then again, I haven't been super impressed with humanity lately and wonder if that sort of democratization of power would actually be a good thing. Over the last few years, I've come to realize that a lot of people want to watch the world burn, way more than I had imagined. It is much easier to destroy than to build. If we make it easier for people to build agents, is that a net positive overall?
If we make it easier for people to drive and have cars, isn't that a net positive? If we make it easier for X, isn't that better? No, not necessarily, that's the entire point of this series of essays. Friction is good in some cases! You can't learn without friction. You can't have sex without friction.
Imagine being starting university now... I can't imagine to have learned what I did at engineering school if it wasn't for all the time lost on projects, on errors. And I can't really think that I would have had the mental strength required to not use LLMs on course projects (or side projects) when I had deadlines, exams coming, yet also want to be with friends and enjoy those years of your life.
And that should be the core. There is a new, emergent technology, should we throw everything away and embrace it or there are structural reasons on why is something to be taken with big warning labels? Avoiding them because they do their work too well may be a global system approach, but decision makers optimize locally, their own budget/productivity/profit. But if they are perceived risks, because they are not perfect, that is another thing.
Having the "call your representatives" link be to your website as well isn't particularly helpful... I already can't get to it
and while I know they can do the nitty gritty ui work fine, I feel like I can work just as fast, or faster, on UI without them than I can with them. with them it's a lot of "no, not that, you changed too much/too little/the wrong thing", but without them I just execute because it's a domain I'm familiar with.
So my general idea of them is that they are "90% machines". Great at doing all of the "heavy lifting" bullshit of initial setup or large structural refactoring (that doesn't actually change functionality, just prepares for it) that I never want to do anyway, but not necessary and often unhelpful for filling in that last 10% of the project just the way I want it.
of course, since any good PM knows that 90% of the code written only means 50% of the project finished (at best), it still feels like a hollow win. So I often consider the situation in the same way as that last paragraph. Am I letting the ease of the initial setup degrade my ability to setup projects without these tools? does it matter, since project setup and refactoring are one-and-done, project-specific, configuration-specific quagmires where the less thought about fiddly perfect text-matching, the better? can I use these things and still be able to use them well (direct them on architechture/structure) if I keep using them and lose grounded concepts of what the underlying work is? good questions, as far as I'm concerned.
"What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune
On one hand I intuitively think this is correct, on the other hand these very concerns about technology have been around since the invention of... writing.
Here is an excerpt of Socrates speaking on the written word, as recorded in Plato's dialogue Phaedrus - "For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom"
I always preferred this take:
“Civilization advances by extending the number of important operations which we can perform without thinking of them.” ― Alfred North Whitehead
It's both opposite and complementary to your Frank Herbert quote.
> “There is a cult of ignorance in the United States, and there has always been. The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.” ― Isaac Asimov
The easier society makes it to be unaware of the complexity of everything around us, the easier it becomes to assume everything is actually as simple as their surface-level understanding.
It's the first step on the road to hell.
Damaging machinery was made a capital offense and they had dozens of executions, hundreds of deportations.
At every stage, the steady progress of civilization is fragile and in danger of being suffocated. Its opponents cloak themselves in moral righteousness, call themselves luddites, the green party, or AI safety rationalists. Its all the same corrosive thing underneath.
Do LLMs lie? Of course not, they are just programs. Do the make mistakes or get the facts wrong? Of course they do, not more often then a human does. So what is the point of that article? Why my future is particularly bad now because of LLMs?
The solution is obviously some form of socialism but a lot of tech people are blinkered libertarians who refuse to put two and two together.