As a mathematician maybe I am a little more optimistic than this declaration.
I am thinking of Mochizuki's abc conjecture: He worked in relative isolation, and dumped a huge incomprehensible proof on the community (to oversimplify a bit). That's not totally unlike what might happen if AI generates a huge, incomprehensible proof of let's say RH.
Well, what is the result? In the Mochizuki case, it was a lot of skepticism, but it also generated conferences, papers, talks in the hallway, discussions with students, and so on--a flurry of exactly that kind of community process that the declaration says is the main driver of mathematics.
Ultimately we think a fatal flaw was found in Mochizuki's proof, so it didn't lead anywhere in particular. But in our hypothetical "AI lean-verified proof of RH" situation, it would presumably generate substantially more of that community activity we saw in the Mochizuki situation. And if it's correct, that community activity would be productive (expository talks, students given problems to flesh out or generalize, etc).
Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.
The maths community is now in the antithesis phase, synthesis will take a while ;)
Lee Sedol said in an interview that "losing to AI, in a sense, meant my entire world was collapsing. ... I could no longer enjoy the game. So I retired", and I think there will be folks in the mathematical community who would feel the same when the solutions pages to hard problems are suddenly available.
But on the other hand, people learned a lot from chess engines. After decades of chess computers beating humans, there was still a renewed interest in watching Leela beat Stockfish, with many people trying to understand the strategy Leela used.
If your happiness comes from grinding on a problem and making progress, the prospect of having to dig through a corpus of AI-generated proofs might be hard to swallow. But if you're willing to do that, you will still find beautiful things that only so many people can truly appreciate.
Chess is kept afloat by chess players, not by billionaires. If all the billionaire backers stopped sponsoring tournaments, people like me would still play, still pay for chess club memberships, still pay entry fees for tournaments, and still buy chess books, and so on.
I think the parent comment meant professional, high-level chess. The kind people get played to play, not just do for a hobby. That's absolutely on life support.
I'm not sure what the equivalent would look like in the math field, but it probably involves a lot of mathematicians losing their jobs and the quality of human-produced math decreasing overall.
The quality of the math in general would be fine, since in this scenario cpus will keep producing it. The quality of cpu-cpu chess games is quite high, beyond human understanding in many cases.
Chess is a weird example because it doesn't really have any utility beyond itself. Even pure math sometimes ends up having use in the strangest places. Although if no one understands the frontier math (because no one is getting paid to), I'm not sure it even matters what the quality of the cpu math is?
It's a bit like a tree falling in a forest. If an LLM proves a theorem but no one understands it, did it make a sound?
> It's a bit like a tree falling in a forest. If an LLM proves a theorem but no one understands it, did it make a sound?
But in future most proofs will be for consumption by other AI models in the pursuit of yet other proofs.
It's kind of surprising so many mathematicians act surprised by this given this was clearly where automated proof assistants would lead. I guess they assumed they'd always be the ones guiding them.
Does your "one" only contain humans or does it also contain other AI systems. AI math is not a single monolithic thing, but a distributed one. I see value in sharing proofs even among just AI.
Also, you learn to be a better chess player by... playing better players. The widespread availability of chess engines has made flawless opponents available to every player.
If your goals are understanding the game, self improvement, building thinking skills-- this is the best chess has ever been. It's only if your goal is to beat every opponent you can find that chess is in a bad place.
> Playing stockfish is like playing tennis against the wall (for untitled players at least).
It's the same for Magnus Carlsen. Even with Queen odds, Stockfish is literally unbeatable for the best players in the world. It's just too strong at evaluating all kinds of random tangent moves (and ensuing positional advantage) which no human player can possibly pay attention due to the time required.
Stockfish vs any human is like Carlsen vs other players by about 3-5 orders of magnitude[0]. It's that stark.
Well, that's actually not true at all. Stockfish is not a very good odds player and at queen odds is easily beatable even by bad players like me. It will just trade down into more trivial and easier to win positions that it perceives as "less bad", since everything is super-losing anyway when you start down a queen.
Leela odds networks, on the other hand, are an entirely different beast. I cannot beat Leela queen odds, much less rook or minor piece odds, and even GMs struggle against Leela knight odds.
Without odds though, yeah, Stockfish is just incomprehensibly strong by human standards. All top chess engines are, but Stockfish moreso.
No worries, I know stockfish is unbeatable by humans.
But sometimes, these GMs can flag it, which counts as a win (especially when it's proxied by a cheater). Sometimes they can also explain the idea that cost them the game, so they've learned something maybe.
Whereas us scrubs literally cannot do anything at all for reasons completely beyond our understanding.
I mean, this is the problem with analogies and trying to use them to prove things, right? People working through problems from an analysis book with their friend (or an LLM) is not the same as research mathematics. People playing in a chess club is not the same as what makes for a good chess tournament. Lumping everything together is just making this branch of the conversation less relevant.
Chess is a fun game. That's why it's been around for 1000+ years.
There was a renaissance during Covid and due to 'The Queen's Gambit' where it gained much more mainstream popularity, but... Chess AI was already far far (like 1000+ Elo) ahead of human players at that point.
The thing is... chess is humans playing (communicating) with humans and that's what keeps it interesting. Check out the view counts of chess AI tourneys vs. human tourneys.
I really hate this overly condescending takes. First of all, what do you know about the internals of math research that allows you to speak with so much confidence. Second, you're not even addressing the issues raised by the letter! This is not about "oh they made a bunch of problems easier". There are huge economical interest behind: who owns and has access to models? are these companies interested in developing research or they just grind PR stunts without worrying about externalities in how research is actually conducted? Etc etc.
If it's worth anything: I have a PhD in (theoretical) mathematics and I entirely stand by stabbles' comment.
There is a real, undeniable possibility of AI becoming better at mathematics in the same way that it became better at chess and Go, and in such a scenario, one may expect the community's response to be comparable.
It makes no sense to compare mathematics with chess. Chess is a sport. No one is interested in watching two machines compete. Chess doesn't have a practical impact. Etc. What you seem to suggest is that AI will be able to completely (or at least in a great part) replace mathematicians. It could be the case in the future, but no one knows right now, and more importantly: tech companies don't even think about it! they don't think on the externalities.
Is this the scenario described in Ted Chiang's short story https://en.wikipedia.org/wiki/The_Evolution_of_Human_Science where scientists are "catching crumbs from the table" trying to decipher the results generated by superhuman intelligence?
It's still an optimistic scenario. Artificial superintelligence may develop hypermathematics of a kind that never will be accesible to human mind, enhanced or not. One can't teach geometry to ants even if you put them on a Moebius strip.
What’s optimistic or non-optimistic specifically about the machine having a system of mathematics beyond our comprehension within it? Why should we care about that in itself?
In the first case, we'd still have a chance to take a glance at the frontier of discovery (even if ordinary human mathematicians had to spent years translating what metahumans achieved).
In the second case all human-level maths would be solved and what lies beyond would be always out of our scope.
>Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.
Dr. Tao said the same thing. Somehow, this letter came through. He wants to conduct Math competitions where participants who don’t have formal credentials can contribute to mathematical research through AI.
Title: Terence Tao - SAIR Competitions and the Future of Experimental Mathematics
I have zero formal math training beyond my Grade 12 Pre-Calculus class. Yet with an LLM I have recently devised an architecture with incredible math potential. Math is a language like any other, and without LLM's I never would have developed the techniques that I have.
AI is a tool. It speaks languages I don't (Math, Science, Code). I would love to participate in a Math competition without a hint of any formal advanced math training because my experience so far tells me I will do well.
He sees value in mathematicians using AI to carefully study mathematics, develop an understanding of both old and new things, and help others understand the new things.
He doesn't see value in scrolling through unsolved problems asking an AI to please solve them. In his view, this is a fundamental confusion about what mathematical research is for. Knocking down unsolved problems without developing the community's understanding of them is like prompting Claude to go through a Jira board, write code for all the open tickets, and then close them without merging or deploying the code.
But the work the Mochizuki case generated can also be done by AI. AI could generate a landmark proof and then people could use it to solve or simplify intermediate problems and you could use a different AI prompt to try to disprove it if you were really skeptical. From my memory I think they said it took 88 hours to solve a Millenium Problem versus the decades of time humans have put into it.
I don't like nuance here. I think progress is really measured by what humans are able to do and understand, not machines. It is significant if we find problems we struggle to solve. That tells us something. What does it take for humans to solve these problems is related.
The best analogy I can give is if you wanted to climb Mt. Everest you might ask someone for guidance. Would it be better to ask someone who has climbed Mt. Everest or someone who took a helicopter ride up near the top and then went to the peak? This is like the AI versus human gap to me. The helicopter is like using AI to generate a proof. The person who actually climbed Mt. Everest has firsthand knowledge of the experience. Same thing for a difficult proof. The struggle people have is actually valuable here. Likewise, we know people are actually capable of climbing Mt. Everest but if they had only ever rode a helicopter to the top, the knowledge of climbing it would not exist, and surely that is meaningful knowledge given the risks.
So if we rely on AI for proofs I think we lose a sense of what is difficult and why. We lose a sense of what human achievement is. Surely climbing Mt. Everest means more than taking a helicopter up? For students, why bother grinding through all the material of climbing Mt. Everest and then attempting it if the helicopter ride is how things are done now? This would have the affect of destroying knowledge.
(please do not nitpick the analogy because it's the best but perhaps a clumsy way to describe my thoughts)
> From my memory I think they said it took 88 hours to solve a Millenium Problem versus the decades of time humans have put into it.
Keep in mind those ~88 hours were spread across ~10,000 simultaneous agent instances.
So, roughly 880,000 hours of compute.
Assuming a fifty-year career, and forty-hour workweeks, a human mathematician's career is about 100,000 hours of "compute".
I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already.
The perverse incentives of academia mean this has never occurred.
The perverse incentives of industry mean OpenAI intentionally scooped researchers who were getting close (granted, with AI help).
I'm not trying to dismiss the achievement - if the proof turns out to be solid, it's quite impressive (though much less so if the training data included the recent human breakthrough, which seems pretty plausible).
I'm just pointing out that "88 hours" is a very misleading way of framing this.
Mochizuki's claimed proof of the abc conjecture was extremely unusual for the reason that nobody was able to extract a single useful idea from the argument. I was starting grad school when it came out, and my immediate visceral response was "if this is what number theory is going to look like in the future, then I will leave mathematics."
The current wave of AI slop mathematics might end up driving the next generation of mathematicians away from the subject for the same reason that Mochizuki would have convinced me to quit if his proof had been accepted by the community. Luckily, my professors had the taste to immediately recognize that it was garbage.
I get excited at the idea of a world in which advanced mathematical problems (and their solutions) become much more accessible to a much greater number of people. As a result, making mathematics much more loved at a societal level.
Imagine a world where these most complex mathematical problems are not accessible to a few hundred people, but a few hundred thousands people.
...Those original few hundred gifted mathematicians would have an even more prominent role, and their names and achievements would be known by orders of magnitude more people that they are now.
Based on current reward models, the frontier AI labs will burn down mathematics as an impressive display of capabilities and in doing so, will make it impossible for people that get paid to do mathematics to stay employed.
If your job is literally to publish papers, and OpenAI and Anthropic decide that making an infinite-paper-printing machine is the best thing to show how effective their tech is, then as a demo, they destroy that industry.
Even before AI we used to say if you write code that you only barely understand, then it will be to complicated to debug. (and/or maintain)
Mochizuki was still one human and it required legions of other humans to unpack and untangle to confirm that it didn't lead to anywhere in particular.
AI is now capable of constructions so complex that no human or human team can unpack. And its ability to increase that complexity is growing while our human ability is stagnant.
meta-AI analysis cannot help. We (software professionals who use AI regularly) already know that if you run into a situation where a Fable/Astra-generated analysis reaches the limits of our comprehension/complexity due to their subjectivity, throwing more AI at the problem doesn't always converge.
There are many reasons to feel optimistic about AI, and ultimately its general ability to help science and mathematics.
I see no reason to feel optimistic about the future of mathematics and AI based on the current path of frontier labs, unless the misalignment Tao is writing about can be reconciled.
> AI is now capable of constructions so complex that no human or human team can unpack.
How can we possibly know this when we haven't even seriously started on the endeavor of actively reverse engineering these AI-generated proofs? That's a proper job for human mathematicians, because the AIs themselves are demonstrably clueless about what steps in a proof are genuinely interesting and load-bearing from a human POV. This is evidence of a limitation in AIs' capabilities, not of any kind of misaligned behavior. The fact that Tao actually uses that term in his complaint is deeply disappointing.
For the interested; the saying I believe you are referencing in regards to writing code / debugging is from Brian Kernighan, specifically:
Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?
It's not just the isolated dumping, it's the fast, isolated, possibly untraceable dumping, without long term support.
It'll basically become slop fatigue if OpenAI starts dumping out proofs faster than the community can keep up, and some turn out to be wrong, never formalize it, don't stay to support it, etc.
I wonder if they will continue to dump proofs, though? Their point has been made, the novelty will wear off, and it maybe won't be a priority use of their resources to spend however many millions on another big proof--they will move on to the next thing to show off I'm sure. At that point, the ones generating proofs will be, I hope, mathematicians (professional and otherwise) that are more interested in the results and community discussion.
(Well that's my hopeful, optimistic take, anyway.)
They aren't going to stop at one, that's for sure. They already claimed they have "made substantial progress" on another millenium problem. Let's say they bag another one (Hodge and/or BSD according to the rumors), if it looks like their internal model could solve P/NP or Riemann Hypothesis, you think they wouldn't take that chance ?
I think AI companies making a point is not the only thing at play. Discovering new maths ultimately leads to new technologies and applications. It may start theoretically but end up being of practical use in the future. Even if humans do not understand it (lose interest, too complex, or just way too many new proofs to go though) AI can use this AI derived math corpus which will help it in other fields.
So it wasted everyone's time, thousands of hours of research trying to disprove something said very loudly. What OpenAI is doing is a DoS of the scientific community: wasting your time trying to check if they're not wrong, and claiming glory in the mean time.
That's true, but the story would have unfolded differently if Mochizuki had a lean-verified proof and was correct. I guess baked into my premise is that AI is producing reliable proofs (in the long term at least).
To me it doesn't seem like what AI has destroyed is the ability for mathematicians to develop understanding and share it with each other, but rather it's destroyed the yardstick (solving open problems) that has traditionally been used to measure how much they have contributed to that understanding.
I do see how this is a problem in terms of assigning credit, but I think the cat is already out of the bag in terms of these models being capable. Even without AI labs spending millions of dollars to solve millennium prize problems, there are plenty of other people who will use them to pick low hanging fruit. I don't think any social solution is going to make things go back to the way they were, where you could share your progress towards a famous open problem without risking someone "scooping" you within a couple of days.
I think that the most likely outcomes are either mathematics becomes more secretive, or there is a more deliberative approach to assigning credit than who was "first" to solve some problem. In the former case, this may slow down progress, and in the latter case, this could mean that credit would become more subjective, and be a continual source of controversy.
The statement is not about AI but about the behaviour of AI companies. OpenAI have put vast resource into solving open maths problems: many millions of dollars of compute just on the Navier-Stokes result, plus whatever they spent on the broader Millenium Prize problems initiative and the other results they have published. Anthropic are doing the same. The statement is asking them to stop doing this.
AI companies are investing these resources primarily as a marketing exercise. There is no near term commercial value to a 100 page Lean proof of blow up in an extreme special case of Navier Stokes, besides the bragging rights. As the statement says any commercial value in this stuff comes a very long time later after new insights and techniques have been digested, integrated into the mathematical canon, expressed in ways that don't take a lifetime of study to understand, etc. (things that AI is not yet capable of doing itself). The bragging rights, on the other hand, are massively valuable. There is a mystique to maths that makes "our AI solved a Millenium Prize problem" an irresistable headline for a company like OpenAI.
What the mathematicians are saying is stop pouring resources that most mathematicians can only dream of accessing into projects that are actively damaging to their field. They face a massive challenge of figuring out how maths can evolve in the face of this new technology, and this is not helping.
What do we do about the problems that don't require many millions of dollars in resources?
Last weekend I spun up a small agent swarm and pointed it at a field of math I have some affinity towards. Within four hours I had settled three conjectures, one of which is rather famous (for the field, not in general). It cost me about four hundred dollars.
I am at a loss about what to do with these results. On one hand I feel like the mathematicians working on these should know about them, but on the other I feel a bit like a barbarian who suddenly finds themselves sacking Rome.
You asked for a painting. A robot made the painting. You looked at it and said, "well, I guess it's good. Should I put it online or something? Dunno. Hey Fred, what do you think of this?"
Meanwhile, your next door neighbor spends their entire life developing their understanding of life through art. They "understand" (maybe not in a way they can articulate) art. You go next door, you look at their painting and say, "well I guess it's good." But you also understand that your neighbor is just like you, and maybe you are a painter in another way.
I find it strange that, people can't see that, we don't need to solve hunger and poverty and work balance, and etc, by a round-about make-super-intelligent-AI. We could just solve it. It's pretty obvious how to, as well.
We can all be painters, if we put restrictions on the psychopaths.
> I am at a loss about what to do with these results.
I would recommend publishing them to Palomar (https://palomar-registry.org/) - I have no affiliation, this is an online registry of Lean-verified proofs created by Terrence Tao.
I have submitted a proof there that's also minorly important in an extremely niche field.
Anyway, I feel like it's a good place to dump AI slop lean proofs because the main point of the registry is that it verifies that: 1) your Lean challenge statement is the same as what you informally state you're trying to prove; 2) your Lean proof actually compiles.
This could be useful to future AI slop researchers who want to know if a given result has already been formalized, and they may be able to mine some lemmas from your work. Also, it's good to know for the field in general what has been proven.
I'm fairly certain you can set your publishing name to be whatever you want, so you could set it to be just the word "Anonymous", or the name of the model you used.
The numerical results are trivial to check. I wrote the analytical results in Lean by hand before asking a former professor to confirm after asking him to keep this private.
It's really tough to come to terms with it but making academic contributions in general from the outside (with or without AI tbh) is not often welcome and the whole process feels very gate-kept.
If that's the case, that makes me much less sympathetic, even though I can understand how it's very disturbing to see the field suddenly changing like this.
I mean, let's say you spun up a swarm of agents to rewrite a large component of a well used open source library to be memory safe. You could dump it in a big PR and walk away (we all know how that would go), or you could try engaging, see if they're interested, write something up and see where it goes.
The biggest problem is, IMO, drivebys uninterested in actual results, just getting a check mark, and the equivalent of dropping a 200k line PR on people and expecting them to be interested and do the work for you. These are things many on HN are familiar with and know how to do better :)
If you are trying to understand better the field, then do a good write up of the proofs so that people can learn from it.
If you want to earn the respect of people because you found interesting proofs. Then do a good write up of the proofs so thst people can learn from it.
If you want to plant flags and pollute peoples minds. Then please publish it anonimously, no one wants to correct LLM slop for you.
Probably we should build a repository of AI slop proofs that are only allowed to be publish anonimously. That way people may be more inclined to work on it because they would feel like they are cleaning your house for free.
Maybe I'll end up doing the write ups pseudonymously. I have taken care to make sure the results can meaningfully contribute to field but I don't want to plant flags or really receive credit of any kind. I just think they are interesting.
I like my current life and don't want to get dragged into the current fracas surrounding the use of AI in math.
How much do you think other AI companies would offer to get access to the transcripts of the generation that led to the proof? No doubt OpenAI will include it in their training data somehow and use it to build the next generation.
That solution (stop pouring resources in to proofs, stay in your lane) works today. How does it work 5, 10, 20 years from now? The software and hardware advances will continue.
I suspect that beyond just marketing, these pursuits yield plenty of useful information about model design that will likely lead to model improvements and optimizations for both mathematics and general reasoning going forward.
Are they claiming that the only value in solving these problems was for their field's personal development process? I thought Navier-Stokes (and some of the other millenium prize problems) actually had implications for useful technology. It would be insane to demand that people avoid making progress on technology that can save lives or improve general quality of life, just to protect the sanctity of your karate belt system. Perhaps in lieu of open problems left to solve, mathematicians should be welcome to take up chess or sudoku to keep their minds spry.
No, there's most likely zero practical benefit of having found a pathological edge case in which the Navier-Stokes equations do not work. We're most assuredly not talking about "saving lives" here. Unless advanced aliens show up and tell us they'll destroy Earth unless a counterexample to the N-S equations is provided within 24 hours.
> I thought Navier-Stokes (and some of the other millenium prize problems) actually had implications for useful technology.
This is the core misunderstanding that the open letter is attempting to correct.
Developing a better understanding of the Navier-Stokes equations could have a number of implications for useful technology. They're fundamental to fluid dynamics, and turbulence in particular is something that many people feel we could work with more effectively if we better understood how and why it's generated. The Navier-Stokes smoothness problem is an interesting and long-standing benchmark for this understanding; we don't know why it should be so hard to answer, so we hoped that the process of developing a proof to the problem would produce more understanding. (We may still be able to extract this understanding after the fact, if OpenAI's proof is fully human-comprehensible.)
Simply knowing that there exists a finite-time blowup is not practically useful. We know that fluids in the real world don't produce random singularities, so the result can't really have much physical meaning. What it illustrates is that the Navier-Stokes equations fail to model physical fluids in some yet to be characterized way.
Do you "know", in any meaningful sense, any of OpenAI's recently publicized proofs? Do you suppose that there is any large community of non-academics that does?
One of the points the parent makes, along with the TFA, is that academia -- or more specifically, the "mathematical community"-- is a setting primarily for creating and ingesting mathematical knowledge, and disseminating it to the next generation and to other fields. Humans absorb this material slowly, through lots of discussion and collaboration -- it is necessarily a slow process. Facilitating this is one of the important functions of academia. Your usage of academic as a slur here is a bit silly for this exact reason.
I don't claim it is perfect, and we can argue about pedagogy in elementary courses till the cows come home. That's not really material. But this is one of the only settings in which such knowledge is broadly valued for its own sake, and in which there is a semblance of incentive to help others "know" this stuff as well, be they future generations of mathematicians, science and math educators and communicators, practitioners in other fields, or genuinely curious amateurs.
Your point is largely addressed in the article, did you try reading it? "In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align."
The point is that these proofs are largely useless without the insights. The value of a proof is largely in the travel, not so much in the destination.
Different person here, I read the article and they are all wrong. Hope that helps.
Okay, to elaborate, substantively, their point is that the people using these AI models are not doing it for the love of the game, but for marketing. And instead of them - and nobody - spending millions of dollars to solve the problem, successfully, they want every problem of their academic industry to persist because even though they never solve the problem, they synthesize and solve lots of other problems nobody asked for. And get to boost their egos?
Yeah, stop that. Actual alignment is on the humans themselves, if they want to remain relevant as academics and mathematicians, they need to learn how to replicate the proofs and the steps that alluded humans for decades and don't worry about the narcissistic elements that slow their industry down.
Their claim is only indirectly related to the motivations of the people using their models. What they're saying is that doing math in this way does not produce the same value as traditional mathematical research, and the people using these AI models aren't concerned about that because their marketing objectives don't depend on whether their results produce mathematical value. If people doing valuable work are made irrelevant by people doing a larger volume of non-valuable work, that's not a positive outcome.
> What the mathematicians are saying is stop pouring resources that most mathematicians can only dream of accessing into projects that are actively damaging to their field. They face a massive challenge of figuring out how maths can evolve in the face of this new technology, and this is not helping.
Is it reasonable for any field to make such demands? If this were doctors objecting to AI becoming good at medical practice would you have the same concerns?
While any idea of OpenAI spying on people to pursue their goals is disgusting, the rest of this is par for the course, as Kasparov experienced with IBM in the 90s. Humans still play chess after all.
It's not a simple
matter of "becoming good at", and yes, I could very well have similar concerns, depending on how it impacts the field. The statement itself mentions that such concerns exist in many other fields.
I doubt OpenAI will take such a combative stance and accuse these mathematicians of "demanding" things, as you do. As I said, the purpose of this is marketing and the statement simultaneously undermines the value of that marketing (showing these projects as irresponsible) and gives these companies an even better piece of marketing in its place: "our AI got so good at maths the mathematicians begged us to stop". It's entirely possible they will stop pouring millions into these projects.
So let’s say OpenAI cure cancer and put every cancer researcher out of work depriving them of intellectual satisfaction, this would also be a problem? It would certainly impact the field.
The fact is these fields are supported by society because of the benefits to everyone else. Once the same results can be achieved in a cheaper and faster way that is what will be done. We should mourn this in the same way we do buggy whip manufacturers. Again people still ride horses.
> The fact is these fields are supported by society because of the benefits to everyone else. Once the same results can be achieved in a cheaper and faster way that is what will be done.
Maybe it's worth double checking that you know how these fields benefit everyone else? Proving the blowup of the Navier Stokes equations in 3D isn't going to make your gas cheaper or make harvesting food easier or make drones easier to protect against. Maybe consider the deeper effects at work?
Math people absolutely are important to things like the SpaceX landing control systems, hypersonic gliders, 5G networks and so on.
If you can make breakthroughs on such areas as fluid dynamics, control theory or information theory with AI then that absolutely is a big deal with real technological implications.
> It's this sort of thing that motivates people to burn down the institution you might be trying to defend.
lol, yes, this sort of thing is what many people who voted for Trump were saying, and things are going great for them.
> They prove a millenium result, but it doesn't count because they are bad people.
OpenAI has only themselves to blame for this, and they know it. They could have handled this so much better. I'd bet there's more meeting time right now going into how to unveil future math results than on meeting about the actual math research.
The issue of credit is a relatively minor point in the declaration.
It's more about bypassing the culture and processes mathematicians have developed that lead to human understanding, generating new ideas, and bringing up new generations of mathematicians. (See also his article about "non-renewable mining" of good problems.)
Reducing mathematics to "let's just generate results through an isolated and automated system" is a misalignment since it bypasses those processes.
I agree that the declaration doesn't focus on credit, but I think it's still at the root of the problem. Because ask yourself: if the AI generated proofs are not creating any new ideas or insight, just brute forcing a boolean true/false result, then why can't mathematicians simply ignore their results? Why does it matter if OpenAI or even amateurs with AI are "solving" these problems, without contributing to any deeper understanding?
I don't think "intellectual poisoning" is really the mechanism that harms the mathematics community.
The harm is if you have a community of mathematicians who are focused on expanding human understanding, then having instant access to a bunch of AI proved results muddies the water about who has contributed what. If someone could scoop any significant theorem at any time by pointing an AI at it, how do you really demonstrate that you have created new understanding? Or that your new understanding is about something important? How do you prove that the AI needed your new concepts to be able to solve it?
I don't think it really attacks human understanding though. You can still read and understand an AI written proof. If another person comes up with a solution to a problem, you can read their methods and understand it. It doesn't matter if a human came up with that or not. It's really only attacking the "generating new ideas" part.
That's precisely the problem though. You cannot still read and understand an AI written proof at the current skill level of the AI being applied, because they're orders of magnitude longer than human written proofs even when they don't need to be, and spend most of that length on the parts that aren't important. This has been really thoroughly documented by expert mathematicians who are engaging with AI in public like Terence Tao and showing in detail how much work it takes working alongside AI to figure out how to understand AI generated proofs. With human generated proofs that process is forced to happen before publishing the proof because the new style of AI generated proofs validated only by formal verification is supplanting the old human peer review process that forced the burden of understanding onto the publisher and not the reader.
> You cannot still read and understand an AI written proof at the current skill level of the AI being applied, because they're orders of magnitude longer than human written proofs even when they don't need to be, and spend most of that length on the parts that aren't important.
That doesn't seem to be true. The OpenAI NS paper was 166 pages. Wiles-Taylor proof of Fermat's last theorem is 129 pages. The length is not unprecedented for a difficult unsolved problem.
To be honest, I feel like the difficulty of reading AI proofs is due to the fact that we are on the verge of being beyond human comprehension. This is a demonstrable fact as no human has figured this out despite the problem being open for almost 100 years.
> To be honest, I feel like the difficulty of reading AI proofs is due to the fact that we are on the verge of being beyond human comprehension.
I can see where that's coming from, but I really don't think it's the case. Even with Astra, the proofs you get are just off in a way that doesn't signal superhuman comprehension. As 9question1 says, a common theme is that they dwell on insignificant steps. Another one is that they'll often be full of terminology that either doesn't exist, or has this weird quality where it looks like it is trying to make some minor insight seem much greater than it is. At first glance, that'll often make it look like it knows more than you, but when it's really just doing the same thing but in a more complicated and worse fashion, that to me isn't a signal of comprehension at all. The bizarre thing is that despite all the "stochastic parrot" style nonsense you'll get in individual proof steps, they still often combine to something valid.
In either case, what all of this means is that the working mathematician still needs to go through, and generally completely rewrite, any proof output by an LLM. Otherwise you are passing the burden of unreadability onto the reader.
Yeah, that mirrors what I've seen throwing some of the leading models at a set-theory problem that's stumped me (https://mathoverflow.net/q/511601): in this case, the problem does not easily yield to the standard tools, but the LLMs do not recognize it as a major open problem they should give up on. So they seriously try it, but typically end up in a loop of inventing certain classes of simple solution or counterexample attempts, defeating them, and trumpeting each one as a major result, each time inventing some new terminology.
It's definitely quite curious that the AI labs are able to push these results through seemingly with pure brute force. Perhaps it's largely a function of how many monkeys you have attempting various constructions on top of the known results and strategies the models have memorized.
Not "a" human's understanding; Humanity's understanding. Understanding the research problem, and the solution especially, is a lot more involved than simply "read their methods". That's the whole point being made.
It matters if a human came up with it because of everything mentioned in the article... A mathematician's solution is necessarily built on other's ideas that have been disseminated, internalized, pressure tested etc. Methodologies differ too. AI can abuse its compute resources and generate a true/false or counterexample statements, without laying the foundation that a decade of globalized research would have.
It's not only Lean code, there are English writeups too. To my understanding the pipeline for these problems is 1) solve in english 2) formalize systematically to check. No one is tackling problems purely in Lean, to my understanding.
I wonder if this is a root of the complaints across fields, how AI is ruining the greater picture and process in writing, acting, drawing, filming, coding, and more.
It's also known as "commodification of labour", and AI is just the latest and greatest tool to do it.
Luddites complained that the trajectory of technology was to allow less skilled workers to mass produce goods via machines owned by factory owners, as opposed to helping skilled workers build up and use their skills while passing them on.
Now we have a lot of money and time focused on LLMs owned by a few companies, making it easier for them to monetize low skill labour(prompting versus art/research/artisanry)
Suppose that tomorrow we learn that AI just exploited a bug in Lean and the proof is, in fact, bullshit. Or suppose it is the case, but we never learn that.
Open problems are not that yardstick. Fermat's Last Theorem is the result of Wiles and Wiles-Taylor, but without key results from Serre, Ribet, Ihara, Langlands-Tunnels, and Frey's program none of what Wiles did would work. But Wiles did get the prize. Nowadays I think the inputs have shrunk a bit by doing more in the R=T theorem so less other cleverness needed.
> but I think the cat is already out of the bag in terms of these models being capable.
When there's a discussion about doing something against the damage of the AI industry: "whoopsy, sorry, another cat escape, nothing can be done".
When there's a concrete mention of an actual solution to avoid more cats escaping: "that won't happen, and even if it did, the damage is already done, and in fact it’s not that bad you all just have to go with the future we decided for you."
So the bag is wide open, more cats will escape, and nothing can be done about any of it. not about the ones that got out, and not about the ones still inside. Sounds more like a preemptive excuse for inaction, cosplayed as pragmatism
> destroyed the yardstick … that has traditionally been used to measure how much they have contributed
This goes much broader than mathematics or academia. This is the entire basis via which society distributes its wealth: based on a labour market derived valuation of ‘contribution’.
> This is the entire basis via which society distributes its wealth: based on a labour market derived valuation of ‘contribution’.
Correction: that's not how society distributes its wealth, it's how it throws some bones to the masses. I wouldn't be surprised if over half the wealth goes to people who don't sell their labor at all.
>or there is a more deliberative approach to assigning credit than who was "first" to solve some problem
it feels like an unintended consequence of the millennium prize is that people view the [last contributor to the solution] as the only one to make progress on the problem. I've never viewed Poincaré as solved by one person and the objective of the prize was to encourage more people to make attempts and contribute towards progress.
this issue is independent, but in these circumstances perhaps interweaved, with the 'ai is taking over math' concerns
Right. It seems like reading an AI proof (although it may not be well written) will provide the same insights as reading a proof from another mathematician, assuming it's been reviewed and edited, just like any human-authored publication. If the work is inherently valuable on it's own, I feel like that's mostly what matters.
At issue is the fact that it doesn't typically work like: mathematician produces a proof in isolation, generates a PDF, and shares it with a bunch of people. There's a whole culture and community going on behind the scenes with conferences, seminars, lectures, chats in the hallway, advising students, etc. that AI-generated proofs bypass.
I don't think that AI would disrupt any of that. Even if AI solves a problem, you can still discuss the methods at conferences, seminars, lectures, chats in the hallway, advising students, etc.
Tao’s key point is that the way people are using AI today does disrupt that. Because people and companies are valuing the results over understanding. So we’re getting slop results rushed out that are automatically verified.
Moreover in the past, discussion and idea sharing would happen naturally to overcome the friction of the process. But now when OpenAI is stuck on a particular part of NS for example, they can just throw more capital & tokens at the problem.
I won’t speak for Tao. But it is not “people” who have valued the result over understanding. “People” are a bit impressed that the machines are as good or better than the priests who have been praying at the inscrutable altar of pure mathematics. But it is mathematicians—I am speaking as a mathematician—who have adopted a culture of valuing results over exposition and transparency. The field has been rife with extremely opaque papers for many years and the character of research participation has been one of exclusion and competition over proof priority at the expense of understanding and transparency. It is unbelievably ironic to listen to mathematicians complain about “AI slop” when they have built careers upon human “slop” if slop means papers crammed with technical density that prevents all but specialists from reading the work.
I feel a similar fate will befall engineers too. Your predictions anre quite interesting from that perspective.
Markets defined entirely by law have distorted our collective understanding of what can actually be built with the knowledge our species has accumulated thus far. How will traditional shields that have protected capital accumulation in tech to survive in a world where governments now realize control of technology is a national priority? Especially as we see its impact on modern warfare, and that such conflict looks like it’s only escalating over time.
Mathematicians appear to me (as an outsider) to exist in a field without such distortions, and I think offer engineers a preview of what’s to come. I certainly have completely ceased sharing original ideas online at this point.
Almost every "normal" job has regular performance reviews where individual contributions, not collective outcomes, are reviewed and used as a sole input for raises, promotions, and firings. If you can't sufficiently document what you personally did, you might've as well not done anything at all.
Yes, that's true, but it is still generally accepted that a completed project is a result of some collective effort where people of varying degree of seniority and ability contribute. There is also not a singular event of a project being completed with a list of heroes/geniuses making it happen, but rather a whole lifecycle of gradual development, maintenance and going out of relevance with contributors coming and going.
I can imagine mathematics of the future being more like that rather than history of discoveries with dates and names
For those outside academia, the ”reward mechanism” is a choice between A) being a genius and working hard to become a leader in your field, B) becoming very good at writing grant applications, C) capitulating to corporations and living with the moral burden of their exploits
Nothing will be lost if the credit system disappears. History of Science has many examples where wipe outs happen. The chimp brain cant survive without creating elaborate stories about how important it is, more as a cope to its own limitations and what it cant predict or control. Humility is good for health. 3 inch chimp brains didnt create the universe.
Tao's critique of AI in the field of mathematics reminds me of what French art critic Charles Baudelaire said in the 19th century about photography [0].
Baudelaire argued that photography became a haven for failed painters, the sorts of hacks that could not finish proper training. Photography, as a mechanical rendering of the world, could only record what already existed; it couldn't transform reality the way a painting could.
He also criticized the public's craze for "rushing" into it, and complained that this technical "progress" was weakening the arts.
I have listened quite a few interviews with Tao and I see him being very careful about criticizing AI. He very often emphasizes the usefulness of it. Where he is critical has a lot of merit. One of the points I clearly remember him saying that having AI be able to solve many of the open problems, regardless of how important there are (there are many open problems that are not that important) greatly reduces the problem space for mathematics students to give new problems to work on.
In a parallel thread omnicognate correctly pointed out that for AI companies it's a direct commercial loss to pour all this money into bruteforcing the solutions to these problems, and that a lot of times the solutions by themselves are not directly commercially valuable. They are doing it for stock price, trying to lure in private capital in preparation for IPOs.
Their models are good, but they are not the moat because Chinese models are good too, so what they are doing, in my opinion, is more harm than good. Mathematics is a science by humans for humans.
>I have listened quite a few interviews with Tao and I see him being very careful about criticizing AI. He very often emphasizes the usefulness of it.
This is tiresome. People should be able to flat-out criticize AI without the implied need to justify themselves all the time or "be careful". Its almost like AI has a trillion-dollar agenda backing it, to the point that you have to add a careful "its really great! But there's this little issue..." for any criticism.
Even those who are very pro AI should have the intellectual honesty of admitting that there are very valid reasons to criticize AI.
People aren't ready to discuss AI assisted imagery as art yet. Most discussions lack the nuance that Baudelaire lacks in that critique, which deals with the nature of art and the importance of human intention and input.
Genuine Art versus Mechanism, from 1901, (https://www.jstor.org/stable/25505621) is another article that I read a few years ago that other people might find interesting.
Tao doesn't go as far as Baudelaire, but there are some similarities. In particular, Tao has criticized that AI is not being used to create new interesting conjectures, and that the rush to prove old conjectures is not giving human mathematicians enough time to carefully analyze and understand the proofs and the methods used in those proofs.
My answer to both is the same: nothing stops mathematicians from doing both of those things, with or without the help of AI. And we all understand that it will take time to do that. But complaining about the dawn of a new era of advancements seems counterproductive.
> But complaining about the dawn of a new era of advancements seems counterproductive.
you completely misunderstood the critics. your analogy is awful. this is much closer to the industrial revolution in the uk: it brought a lot of progress, but also extreme inequality and concentration of power.
It's not about whether the conjectures themselves are interesting or not, or whether people simply have enough time. It's that a bare proof made by a machine doesn't actually do much for us. There is not some set of problems that, once finished, will amount to some kind of final, correct system and we can call it a day and, like, utilize it. "Mathematics" is the people doing it (the "mathematical community" Tao references below). This other stuff is kinda just.. expensive exercises to render a result. They are only actually beneficial to us insofar as they exist in a context of research among peers.
Keep in mind that his critique is very recent, and likely applying to a specific use of AI, as opposed to AI as a whole. If you've been following his Mastodon account, he's been happily using LLMs for math purposes for well over a year.
I'm really tired of these arguments (this and "it's just like calculators").
Photography decimated other forms of visual art, so the concern wasn't wrong. But AI threatens the entirety of human intellectual endeavors. I can make do without oil paintings in my home. I'm not sure I want to live in a future where we make do without brains.
Lots of people still make bad music that other people still manage to enjoy (a lot of it has gone multi-platinum!) even though they're not Mozart or Bach.
It did. While you're busy recording an "instagramable" moment, you are not entirely enjoying that moment. Your eyes are on the screen, not on the subject.
In a way, if photography is an ersatz for painting that eventually made imaging available for the masses, then AI could become an ersatz for thinking. But it feels like I'm paraphrasing TFA.
What will happen when AI companies have spent their advertising budget on math problems and whatever else gives the maximum wow effect for the bucks? Probably customers hooked on the vain satisfaction of spending token$ to impress friends.
We can’t look back with perfect hindsight because both the past and present have deeply ingrained blindspots. They don’t know what it is like to live in a world with perfect edges. We don’t know what it is like to live in a world with no edges. We can read about someone who proclaims that “something will be lost”. We will just think “but I have no need for any of that.” But we don’t even know what it is.
Nothing AI companies will change the value of math, as William Thurston said: The product of math is clarity and understanding, not theorems by themselves. What they seek for the IPO is devilish and misleading, and doesn't serve the true purpose of the math.
I understand this stance and where they are coming from, but I can't help but think this sounds very analogous to engineers' arguments against AI-assisted and vibe-coding, especially with regard to cognitive debt. Yet the software industry is plowing ahead, reportedly pushing mountains of unreviewed code to Prod, and the world hasn't ended.
Of course, nobody's really comfortable with it, so this is also a forcing function for the industry to adapt and figure out new techniques to manage complexity and trust. I think the same will happen with Mathematics.
But it is also possible we will end up with three forms of Mathematics: the one we understand, the one we don't, and the one we don't understand but can prove to work. Kind of like magic -- with all the positive and negative connotations of the word.
It is pretty evident that these models will soon exceed our cognitive capabilities. Is it right to hold them back just because we can't keep up? Many of those discoveries will be so beyond us that we can't do anything with them, but that also means they can't hurt us. On the other hand, there could be many discoveries that we can parlay into practically useful applications, even if we don't understand them.
This sounds a lot to me like people in the 90's complaining that computers were destroying chess. Thirty years later, chess is more popular than it ever was, and chess players are better than they ever have been. I wouldn't be surprised if there are now more chess books now than there ever have been. Furthermore, it turns out that a lot of chess books written before computers were just wrong about a lot of things. It turns out having an oracle for the "right" answer in chess, even without an explanation, used properly, allows humans to develop broader, more accurate insights.
The argument here sounds similar. The fear, as I understand this statement to be saying, is that by being given the correct answer, in the form of a 100-page Lean proof, humans will be robbed of the chance to from insights about the structure of mathematics itself. I don't see any reason that humans can't continue to develop insights as they try to digest the 100-page Lean proof into something more manageable; but with more certainty and fewer false starts.
That's correct. As far as I know nodody builds any kind of science or technology on top of chess, but mathematics is at the base of most science and technology. It would be horrible if we prevented AI from solving mathematics problems, just because mathematicians want to solve them by themselves the "hard way".
> I wouldn't be surprised if there are now more chess books now than there ever have been
Well yeah... how would there be fewer??
But the point itself is silly. Few people are putting effort into Maths for the fun of it (and of those that are many derive fun from being the only one who can produce a solution). Chess differs in that it never had any point but the game its self.
But computers have destroyed chess as a "sport". Nobody will sit to watch two chess programs compete, or analyze their tactics. Kinda like how now, anybody can construct a "game" over the weekend or a new song or a slop video. The value of each of these decreases to 0 as the slop overwhelms.
>Nobody will sit to watch two chess programs compete, or analyze their tactics.
I know nothing about chess yet I dare say that I'd doubt this. Surely chess enthusiasts would be interested in analyzing how a superior chess program came out victorious, no?
Yes you are right, some people do watch chess engines play. TCEC (Top Chess Engine Championship) [1] streams them. Popular chess YouTubers goes over engine games from time to time too.
I find this explanation has a lot of applications for programmers within companies. It’s one thing to get your LLM to give an answer, it’s another to bring a group of people into shared understanding of a domain.
Yeah. My other favorite example are books. Why do nonfiction books exist? There are some pathologies and corner cases, but fundamentally: to develop and share new ideas. Downstream from that, if it reads well and if you're lucky, you make some money.
But now, LLMs can generate hundreds of books per hour. They make up 80-90% of new arrivals in many nonfiction categories on Amazon. They short-circuit the system, allowing their "authors" to extract money from the system with zero effort by crowding out human work. And it's not even the question of whether these books are good or bad (although overwhelmingly, they're terrible). It's whether it's actually accomplishing anything worthwhile, or just destroying incentives for humans to write or go into any other sort of intellectual work.
In fact, I see many professions push back. Artists, writers, now mathematicians. And I'm amazed that our profession doesn't and that we have so many people who are hooked on vibecoding. I'm still waiting for that 10x payoff. All this velocity and somehow, the landscape of the software I want to use still looks the same as it did in 2021.
It doesn't, that's the whole point. You can produce output that passes the smell test with naive buyers in a matter of hours. If you want to write good LLM books, then you gotta work closer to human speed - weeks, months - which allows others to produce 100 slop-books in the same timeframe. You still lose.
If an author let AI do 90 to 95% of the work, cleaned up the remaining 5 to 10% themselves, would you say AI or the author wrote it? Because this is roughly the split between an author and an editor, with human writing, now.
Playing a devil's advocate. Why do we need understanding ? To take an example i would say ~99% of the population do not understand how combustion engines or how semiconductors work, what say another 1% ?
Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
There ought to be more to life than sitting in a pod receiving sufficient nutrients from a tube, even if there was no doubt that humanity would in this way survive until the sun expands and makes the earth uninhabitable.
I grew up in a cult. Based on my experience, I believe that the most dangerous thing a human can do is to allow someone else to do their thinking for them.
It’s not going to be ubiquitous? There hasn’t been a single frontier model where generation n costs less than generation n-1 to run. So the reasonable thing is to assume that GPT-7 will cost even more than GPT-6, and more and more of the frontier of knowledge will be locked behind a giant paywall. Participating in any field will mean ponying up to the oligarchs that own the infrastructure that runs the model.
At the individual level any mathematicians can, but the social group of “mathematicians” will not unless drastic social changes are made, because that's not how the system works today. Drastic social changes usually take decades, and funerals, to occur, so the reasonable hypothesis is that it's not going to change fast enough.
youre not wrong, but i think you are missing his base point. tao is saying that mathematics is a practice that the group collectively builds. students and new methods are built on top of what has previously been done, so its a community effort of research, experimentation, and knowledge sharing.
his core argument that is LLMs (and specifically LLMs owned and gated by corporations was my read), while able to solve problems, do not contribute to this practice of knowledge building. solving problems is just one piece, and the mathematics community ingests problems and new methods, iterates and thinks on them, and then produces new ideas, methods, etc. this is what he is defining as progress, and solving things like millenium problems are markers of this progress.
I published a substack about this just a few days ago [1], my core theory here is that we will absolutely have what I call a "highly productive dark age" in mathematics where knowledge vastly outpaces understanding driven by publish-or-perish incentives, but additionally this will lead to the loss of the skills necessary to understand.
The hopeful note is that I do think we are entering a golden age for the curious casual/semi-pro mathematician and for niche mathematics areas that won't get the attention of the top labs. Everyone is sprinting to solve the millennium problems, but this is a very exciting time to be in a sub-sub-field where you and 4 others are keeping things alive.
The existence of solutions to problems does not prevent you from solving the problems yourself anyway, if your goal truly is personal development. You're free to go solve Navier-Stokes yourself right now. You're free to manually do all of the AI work in any field, actually. You won't get grant money or prestige (which are not a part of conceptual understanding and insight) but you will get all of the conceptual understanding and insight you're after. You have not been deprived of it.
It seems like a lot of the issue here is that these problems aren’t interesting in and of themselves, but they lead down interesting roads. It defeats the purpose if you solve them without getting any real understanding.
It’s akin to saying you’ve solved “pancake flipping” problems with a waffle maker, or “travelling salesman” problems with a zoom meeting.
Nothing is stopping these folks from continuing to study the problems and arriving at their own solutions so they can continue having whatever insights along the way.
Well, one thing is stopping them. There will be no more adoration for their genius.
If you truly do it for understanding and not the attention, carry on. AI should change nothing about your motivations.
> Nothing is stopping these folks from continuing to study the problems
My understanding is they are? And literally everything in this world is based around incentives. If you say “well you can continue to work on understanding, but your kids are going to starve” that’s not nothing.
Maybe an analogy of "why not solve all medical problems at once" is closer.
We absolutely want to, of course. But you extinguish an industry and the systems of training that supply it. It's hard to know if letting it go that way is right.
The point isn't what happens after something is solved. Where is the motivation for humans to study/find a potential solution if your kids are going to starve while you do it? These things can brute force problems that have a clearly and feasibly searchable answer, but they can't make creative leaps. It will be a severe dark ages for mathematics if humans stop contributing.
Your comment perfectly illustrates why most are missing the point. Many people, including you, deeply believe that most mathematicians are chasing adoration of their genius, otherwise why would anyone care about such abstract work?
People like Grigori Perelman would baffle you, a mathematician who solved the Poincaré Conjecture, refused the monetary prize, field medal and continues to live a life of total recluse.
For most mathematicians their primary drive is chasing the unknown, not for anyone’s adoration, but to pursue their desire to see what lies in the beyond.
This is a good metaphor for treating the means as an end, thanks. And I agree with your parent comment that that is largely the misalignment that Terence is pointing out.
There is an implicit agreement that mathematics is funded, for the most part by the public, as a way to advance the state of knowledge and propagate (even if very indirectly) what has learned for the public good.
It was never about helping individual mathematicians demonstrate that they are individually good at math. It happened to work out that way, but it wasn't the goal.
Agree, my comment is rather about the fact that most work you do in school and (lower) university is not itself of interest, rather the purpose is to work out your brain. The more of your exercise you leave to the forklift, the steeper the cliff will be when you find a weight that doesn’t fit on forks. (That is, unless/until machines get good enough to completely replace people, at which point we have another few problems on hand, such as day-to-day purpose and justification of existence)
> We are witnessing a general threat to intellectual work
This is the crux of it and goes far beyond Mathematics or Computer Science. To get a bunch of humans to do anything, you have to motivate them. Kleos and Timē; renown and stuff. These AI companies threaten to rip this away from everyone but themselves, and this recent millennium prize is the perfect example.
Solving this problem as a human would have led to tremendous Kleos; my name would be written in the annals of mathematics, lecture tours of praise were mine to be had for the rest of my days. This one victory would have earned my recognition throughout history. The greatest a mortal may hope for. Ripped away.
It would also have given me great Timē. The prize money, the professorships, the book deals. Gone.
If all hope of “renown and stuff” in the intellectual realm is now taken by the AI companies, they will remove all human motivation to pursue these endeavours.
Perhaps the glory will come from slaying these fell beasts.
This letter is complaining that human understanding has been crucial to advancing of mathematics, and AI companies are not bothering with it. But the promise (and horror) of AI mathematics is that, if it succeeds, human understanding becomes irrelevant. That's the goal. So this letter's message will fall on deaf ears.
Keep in mind employees at AI companies are publicly stating that they believe they're risking a >10% chance of human extinction. They're knowingly risking the lives of every man, woman, and child to continue the work. The lives of their own sons and daughters. A person already rationalizing that isn't going to shed a tear for the careers of mathematicians. Just a bug on the windshield.
OpenAI: "Our mission is to ensure that artificial general intelligence benefits all of humanity."
- Except the mathematicians who we'll scoop and cause existential dread among their entire field.
- Except the software developers. They'll need to become plumbers or live on UBI.
- Except the people in countries that can't afford the cost of AI tokens to keep up with the rest of the world.
Just keep picking off groups of humans for the "benefits of all humanity"... while building larger and larger disparities been the have a lots and the just have enoughs.
We're going to build humans a utopia but along the way we'll leave a trail of destruction because that's not our problem.
But its normal, and good, that technologal progress creates, and destroys some jobs.
Imagin a cheap, 100% reliable, self driving car would be released. Death from Traffic incidence fall by orders of magnitude
Would you argue it didnt benefit humanity, because taxi/bus drivers are nolonger required
I'm actually a AI optimist. I think it'd be great to have everybody getting around in self driving vehicles.
If all that AI brought resulted in just taxi/bus drivers being phased out of their jobs in a thoughtful way, then that would be more manageable at the society level. But we're talking about almost all sectors of the economy.
If the magnitude of changes that OpenAI and Athropic believe will be delivered with increasingly powerful AI (and robotics) comes in a time frame that significantly worsens a large proportion of people's lives, this is a different situation. Can super powerful AI not be developed in a way that minimizes such disruption?
I don’t think they make a coherent argument here? This seems to hinge on some argument that because AI doesn’t properly explain its breakthroughs, therefore the breakthroughs are less fertile for human understanding? This makes no sense. Why wouldn’t these under-explained breakthroughs be extremely fertile soil for explanations?
Imagine time traveling back in time and offering Leibniz a packet of proofs from the intervening years, but with the caveat that there would be no explanations. Would he say no?
> We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align.
This is Terry Tao talking about AI's impact on Math, but this could just as well be a software engineer talking about AI's impact on software development.
Do mathematicians deserve more job security than software engineers?
It's not about job security. It's about the social and intellectual practice of the discipline.
The threat to mathematics isn't that suddenly the profitability of their profession (lol) is going to go away, it's that people are thinking of AI as a replacement for the human social and intellectual practices that constitute the discipline.
Kinda crazy to think that academia functions as a kind of humane reverse centaurism. Theory X (reverse centaur) before Theory Y (centaur) for managing the development of others.
There's an unpopular branch of mathematics which does not have infinities - finiteism.[1] The constructive version of finitism takes the position that there is no such thing as infinity, just arbitrarily large upper bounds. You can have theorems about arbitrarily large numbers, but you never get
1 + 1/2 + 1/4 + 1/8 ... = 2
The benefit of finitism is that it escapes undecidability.
The big objection to finiteism is that it's a lot more work. Infinity swallows many special cases. Proofs get longer without infinity, and most of the special cases are uninteresting. That's not a problem for AIs.
Someone may start up an AI and make it grind through Hilbert's program for putting mathematics on a fully consistent foundation, starting from a finiteism base. This is a huge, unrewarding job. Great for machine work.
It's not necessarily clear that this statement requires infinity, if you're willing to treat "... =" as a shorthand. You might prefer something like "1 + 1/2 + 1/4 + 1/8 ... -> 2" if it's more clear, where "->" means something like "gets as close as you like without ever getting further away than that", but really the "=" sign is already overloaded in all sorts of subtly different ways anyway, so there's not really any trouble using it here. Almost any rigorous definition you can write down of exactly what that statement means would not rely on the use of infinity.
If you allow infinite recursion, you soon get to Godel and undecidable problems.
Finite deterministic systems are decidable, because you can in principle enumerate all the states. The halting problem is decidable for deterministic systems with finite memory. It may be exponentially hard for some programs, but that's quite different from being undecidable.
(This is too long a subject to discuss here, and I haven't worked on constructive mathematics in many years. It's more practical than it was decades ago. You need power tools, which we now have.)
In internet culture there’s this phrase “Hydrogen Bomb vs Coughing Baby”, meant to highlight the absurd power difference between two combatants.
In almost any scenario even tangentially involving mathematics, twenty-five Fields medallists uniting to denounce something would be a veritable Tsar Bomba.
It should give you pause that here they feel like the ailing infant.
Mathematics is about discovering and understanding the logical implications of assumed axioms under various inference rules.
Alternatively, some claim that mathematics is about understanding these implications.
Under the first definition, AI is already, and forevermore will be faster and better at proving theorems. Just like it is better at checkers, chess, and now go.
The author asserts that AI proofs are incomprehensible to humans, and so under the second definition AI is merely a tool to overcome one hurdle on the way to understanding.
So which is it? The author seems to claim the second definition, but bemoan the end of mathematics under the first.
Dont confuse mathematics with the formal system. If you beleive mathematics = formal system then AI is obviously better at it, and we dont need humans.
But then who decides why a statement is mor important than another? In the eyes of a formal systems all statements are born equal.
Everyones outraged all the time. It doesn't mean anything anymore. It's that meme from years ago about the red ants and the black ants living in a box peacefully until someone shakes the box and they start trying to kill each other. They go after each other and not the one shaking the box.
I don't understand how that metaphor applies here, you seem to contradict yourself. The mathematicians are mad at openai. If the mathematicians are the black ants and OpenAI is shaking the box, who are the red ants?
Gotcha. In that case, the "ants" part of the metaphor is truly fitting since these people just like to comment online and are mostly insignificant. Unlike the ants, they can evolve and become the mathematicians or get a job at a frontier AI lab but then they wouldn't be spewing so many comments.
Don't worry about what they write, they just want to feel emotions from any news article.
This time, until they dont anymore. The next stage is people attacking the mathematicians and accusing them. The OpenAI fans come out to attack, then the masses will pick sides and it all just becomes a mess
Yeah, I'm so tired of people moaning about how huge swaths of Earth's ecosystem are being destroyed and rendered uninhabitable to humans, how secret police are murdering Americans in the streets, how the president is a child rapist who openly accepts bribes, how unfettered capitalism is destroying tens of millions of lives, how the United States is rapidly falling into facism, how civil rights are being systematically destroyed, consumer and environmental protections are being gutted, and entire generations are being intentionally cut out of the possibility of economic prosperity. YAWN! Give it a rest!
Man, the people who want to just get away with open corruption sure love you.
You're looking too closely. Whether or not mathematicians are actually outraged makes little difference to how this article plays. Hell, lying would likely INCREASE revenue.
Looks like mathematicians (like people in many other professions) have to redefine what their work means and how to define success. Hard to agree that a tool that can find a proof is detrimental by itself, rather it voids some assumptions people relied on previously
I am on this track too. If AI leads to advancements, objectively that's a positive (depending on the advancement I guess) but it's only when mathematicians realise they'll get beaten to every thing now that they're outraged.
Say AI becomes the best at everything. Best at chess/go, best at maths, philosophy, economics, romantic advices ... and so on. Then what's the point of thinking by oneself? Of talking to one another?
What's the point of being human if we dont do human things but entirely rely on AI?
I believe this is more or less these mathematicians' argument.
It has been best at chess for quite a while. Yet everyone knows who is Magnus Carlsen, even though at no point of his career he was stronger than the machine
Magnus Carlsen plays fellow humans at the game because people still care about human competitions. What motivation would a mathematician have for solving already solved problems by hand? Can you imagine someone spending years working on a proof for an already-proved theorem just in case it leads to new insight?
If insights and human understanding are more important than specific results, I don't see why people should stop producing insights and human understanding. "Working on a proof" in today's sense of trying to come up with a proof before others probably stops being useful, other ways of working will be needed
Of course not. I'm so tired of the chess analogy. It was always a game and it was always performative.
Research is fundamentally different. We are seeing the erosion of specific needs for thinking at depth. AI is the automobile for the mind. There will 100% be undesirable consequences and selective atrophy of cognitive abilities once prized. This is a loss. There's no getting the cat back in the bag at this point so long as the objective dimension of work, as in object opposed to subject, is held as the most important.
SWEs felt this same crisis late last year. Now it's the mathematicians. They won't be the last.
Yes if you don't think about the matter for more than 6 seconds you would indeed conclude that, and retreat to the comfortable cliché of "it's just a tool". Meanwhile, I'm glad that there are still people who _think_ about issues and ponder the consequences and reflect on things before they become a reality.
> We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align.
This is about how good taste in both research direction and in design are essential to steering AI, but we have no plan at all for instilling that taste in students or practitioners in a post-AI world.
> The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place.
Besides eroding taste and taste-building, this is about just how useful friction is as signal.
Everyone coding with AI knows it routes around difficulties like a river around a stone, which is not necessarily a good thing. It will do it tirelessly 1000 times instead of learning anything from it. AND if the AI does not fail in this, the human driver will get no signal, and never know it happened. This seems to be getting worse, not better.. my theory is that more models are cross-trained on cybersecurity stuff where the goal is success and the method doesn't matter. Fine for pen-testing, ultimately pretty bad for coherent code or math or physics.
Discrete tasks where we don't want to be bothered is a real use-case, but optimizing for it everywhere is terrible for the future of durable abstractions that we can build on and ratchet up our understanding with. Bad for the models too eventually! They can maintain a codebase with millions of special cases or juggle tons of free variables in equations, but that just encourages bad abstractions.. they have a ceiling for this too, even if it's higher than humans.
Im surprised about the sentiment in this discussion.
I totally see the problem Terence is describing. We are loosing a lot in understanding and focus if it continues like that. The solution found for Navier Stokes doesn’t have much „real value“ - but what almost always happened in the past when people worked on the difficult problems, these sparked new ideas / new theorems that broadened our knowledge.
Think back at your grad studies, figuring out a proof as homework was hard, sometimes incredibly hard, but while doing it we gained a lot of understanding how things work. Now asking AI for the solution and „just“ getting it, risks our understanding, our creativity and our ability to connect the dots with other territories. I see it in students nowadays, there is much less understanding, much less creativity in finding solutions. I truly think this „short-path“ solution with the „death of struggle is one of the biggest risks with AI already for human development
Puts the onus on the AI companies to provide a specific replacement mechanism, no? Unless I'm unfamiliar with something else he's written that proposes something more specific and constructive
To Tao’s credit he obviously identified the problem very clearly and admits understandably "we did not have the time to have a more consultative process, as with Leiden; but we decided that the urgency of the situation was such that we needed to release a statement sooner rather than later".
> Puts the onus on the AI companies to provide a specific replacement mechanism, no?
Why? If someone makes an innovation that undercuts the underpinnings of some existing institution, why are they are responsible for cleaning up its failure?
Math researchers are privileged class? I can guarantee - construction workers are extremely rich compared with math phds and most of postdocs. Talking about privileged class is absolutely laughable here.
In other news, evangelical christians ask scientists to stop publishing about evolution.
We apparently have a moral obligation to protect existing power structures?
Tao should maybe consider there are people who are indifferent to, or actively want to tear down, his institutions; why should they cooperate in preserving them? Whatever happens has to be resilient in the face of defection; any scheme where everyone is expected to agree to not use AI in a way he doesn't like will not qualify.
I think he's in the "bargaining" stage of dealing with loss right now.
You have a practical obligation to obey existing power structures. That’s what makes it a power structure. What do you all think is happening here? There have been people who decided what math gets published for centuries.
> In other news, evangelical christians ask scientists to stop publishing about evolution.
not sure it's comparable, but the issue is that for a lot of those mathematical results, they don't really have utility by themselves. The utility is the new branches/understanding that's being developped.
The entire western world is anti-progress and pro-incumbency, and its very tightly linked to gerontocracy.
Older people are desperately trying to keep a grasp on their current power and lifestyles at the expense of younger people and technology.
We need to ban Waymos because taxi drivers need to be protected.
We need to block housing because it would lower my property values, and eliminate property taxes while we're at it! I don't use the local schools so why should I be taxed to pay for it.
We need to spend recklessly to pay my pension and have the next generation foot the bill.
How specific and constructive it is might be debatable, but he has tried to make concrete recommendations earlier; see e.g. slides 46-51 from the ICM talk: https://teorth.github.io/tao-web/slides/age-of-ai-icm-2026.p... – obviously there's some way to go still.
This reads like people lamenting a bygone era and making a desperate attempt to bring it back. I'm sorry. Outside of the good ol' boys club, no one cares about some process they've romanticized simply because "that's how its always been done". Absolute nonsense.
We are moving forward and if that means no human wins a fields medal because they didnt spend three decades working on a problem that could be solved in three days, the world will be better for it.
And I am outraged by their dinosaur mindset and the gatekeeping mentality that force every student to follow their same archaic system that no longer makes sense 20 years ago, let alone now.
Reform the way math is taught and researched. Right now, the aspiring mathematicians have to follow a ritual and system for 15+ years: undergrad -> PhD -> postdoc -> faculty, specializing on very narrow fields and god forbid if they have even a slight interest in quantitative finance. You could go through 4 years of undergrad classroom and had no idea what research math is (even at top schools like MIT, Harvard or Stanford), how it's done, etc. and the people who do it are typically the one "already in the system" (e.g. parents are professors, or in academia, or have connections to do research).
In the age of AI, there's no reason one has to follow the kind of classes like Algebra, Topology or PDE. Teach just enough so that good students can understand the basic, and go straight into seminar and research math. I don't think a top student in sophomore year cannot understand or work on some combinatorics research problem and get some results, with proper mentoring and guidance.
I agree with many of the sentiments here. But an open letter signed exclusively by Fields Medalists that purport to define precisely what the "mathematical community" (who is inside and outside) and what their goals are raises my hackles for some reason.
It is true that the manufacturing of "true/false" statements is not the same as gaining understanding of a problem. However, for many mathematicians, true/false statements are already manufactured by others. Think of a student who is given a conjecture to investigate, with their advisor describing it as "it must be true". Most exercises in a textbook are stated such that the outcome is known before you begin. That's not really a problem -- investigating the conjecture/exercise yields its own dividends, whether or not the outcome is known. It is also the case that defining new directions involve understanding and synthesizing related problems, asking the right questions, and deciding upon the right directions, and it's not clear that AI can do that at all.
The real risk, I think, its the public's (and funding agencies') perception of the importance of "human" mathematics, but that's already a struggle. For example, it's tough to explain to the lay person why it's still important to research group theory -- the main example people cite is RSA encryption, which was invented almost 50 years ago.
Back in the day you could think of a cool idea. I don't know maybe a plane that could fly without drag. To even see if this was feasable you had to understand physics, engineering, and then from there you had to have a math person see if it was actually possible.
Now, I can ask ChatGPT about this and get back a proof that shows "a passive airframe cannot sustain zero-drag motion through still, viscous air"
So, I think if anything now, Maths has changed for the better. More ideas can be proven false or true from a get go instead of wasting so much to see if its even feasible to find out it isn't.
Progress if anything is about to leap frog anything we have ever known.
This is a pretty moronic take. You only have to understand the first thing about physics (e.g., a 14-year-old's understanding) to know that a dragless airplane is impossible.
If you are suggesting there is some new model of physics or groundbreaking technology that would allow such a dragless plane, then don’t let me discourage you! But AI won’t help at all, since it will only regurgitate conventional wisdom…
It's good to start a conversation, and the number of Fields Medalists behind this certainly lends a lot of weight to it. But I'm not seeing a strong argument for misaligned incentives beyond the specific plagiarism allegations. The job of an AI company is to build systems that solve problems. The job of a mathematician is to advance the state of human knowledge. If anything, an influx of solved problems should increase the demand for human mathematicians who can convert them into conceptual understanding.
Are you implying that OpenAI using someones unpublished research without their permission to solve an career defining math problem with their latest model in order to publish first is a problem with the mathematicians?
My read on this document is that people's work isn't being fairly cited more than what does it mean to be a mathematician in this age.
I didn't know this article was about that issue at all. Yeah, if the issue is properly citing work then yes, OpenAI needs to do that. But the article read like it was tackling a completely different issue.
That's one of several issues, obviously a big one, and they do touch on it:
> Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions.
Indeed, I wonder how a similar letter by Uber drivers would be received -- "navigation is an intrinsically human domain, personal relationships are critical for passengers and drivers to progress in the world, etc etc." Or doctors, for that matter.
We are all going to have to come to terms with entities more capable than we are, and in many cases, letting the real work be done by the AIs will be the right thing to do. For all the huffing and puffing about the "human touch" in medicine, it will eventually become downright irresponsible to consult only with a human doctor. I am not sure if this is the case in mathematics or not, but if it isn't, that suggests math will be relegated to more of a hobby than a cutting edge scientific discipline.
That's how I read it too, Terry Tao, who has been a "pro-AI math guy" is going through the same emotions and confusion that us SWE folks are going through, "oh, wait... this might mean I'm not going to be special anymore!?"
I don't mean to be a dick, but I've talked about it previously. These folks are grieving. I get it, I've lived through this sort of life changing thing before, it sucks... but yeah.
I think this is ridiculously flippant. If software engineering and the hardest math is solved, that means that eventually a majority of professions and knowledge work is solved. This is hugely problematic because of the way our society currently functions. People need jobs to eat, pay for housing, etc.
Dismissing it as "innovations have happened before" is disingenuous. Yes, innovations have happened, but none of those threatened to automate all human work in existence.
Yeah given he had been very pro-AI for years, I expected he made peace with the issue many many years ago (like I did back in 2018), and when this time would come he would explain to other mathematicians how to live with it.
All “fields medalist” signatories - a rarefied and elitist group indeed.
I wish this letter could be more egalitarian and include the view points of those who AREN’T the beneficiaries of a highly competitive winner-take-all system.
Since the common narrative is that AI frees up labor to do other things (engineering -> trades), maybe we can celebrate that genius mathematicians will now spend time teaching children how to be as smart as them?
Maths will continue as a field of natural science in understanding the results and uncovering meanings in them. It is normal that the established community is afraid of the change, because it’s their _home_ that’s changing. But it will be a better home to the new generation nonetheless, one that’s not as daunting as the higher maths has always been to many. The concerns raised here will not be a problem at all.
In the Economist article Tao links, Hugo Duminil-Copin, draws a comparison: airdropping someone on the summit of Mount Everest is very different from climbing it.
The fundamental issue with AI solving any perceived difficult problem is that we have lost the journey. The sight atop Mount Everest looks much different when you have climbed compared to being dropped from above.
But we don't pour billions of dollars of research funding into mountain climbing because we think it's going to lead to wider breakthroughs in science and technology. And when we need to get people on top of a mountain for an important purpose -- like a military or search and rescue operation, for example -- we absolutely do airdrop them right on the top.
So that raises the question: is mathematics simply a pursuit of passion? Are problems solved "because they're there"? If so, then mathematics can join the ranks of things like mountain climbing, cycling, and weight lifting. But if we are trying to accomplish something important (design better airplanes, find theoretical guarantees about cryptography, factor matrices faster), mathematics needs to become more like a military or search and rescue operation, using the best technology available to secure the outcome we need. Given that the NSF pours billions into scientific research every year, it sure seems like mathematicians want to think of themselves as being in the latter category.
What defines important and why must it be solved in haste? Many issues and other problems arise during the journey in solving all problems; those that are needed and those that are pursuits for their own sake.
If AI gave us the plane to reach Everest without us having gone through the journey of aviation and flight, what would we have lost without that process?
But the most important problems to be solved are not technological challenges but social ones, involving humans and our relationship to one another. An area AI will forever ill-suited to handle.
as someone who loves to go down with a snowboard, I can see value to being taken to the top and then enjoying the ride down. im sure it is not a thing to be ashamed of, as millions do it.
When writing math papers, many (but unfortunately not all) mathematicians go through a post-processing step, where they take their ideas and proofs, and try to reduce them to simple and reusable core ideas that can be understood by the reader. Good writers will often also provide some representative examples that guided the proofs, explaining why various intermediate results can't be strengthened and why the proof can't be made much shorter without inventing new techniques. If AI-generated proofs were required to go through such a post-processing step before being published, that would go a long way towards improving the situation.
It's funny that seems like a step the human mathematicians would want, and (at least for now) might still outperform the machines on. In the same way that, eg, the notebooks of Galois contained the core breakthroughs in a messy form, and generations after him simplified and synthesized those ideas, until you finally have books and videos accessible to undergraduates.
One proposal that Tao hints at is to not rush to announce solutions. Instead maybe the AI companies should work privately with the subject matter experts on how to communicate the discoveries.
If frontier labs had chosen to go the path of offering to assist in existing endeavors, helping to build knowledge alongside researchers in ongoing projects and following ethical and professional research standards, we wouldn't be having this discussion at all; everyone would be stoked. Instead we have companies that disgracefully try to scoop researchers and fail to properly attribute earlier work and instead rebrand it as their own (what we normally call plagiarism) to make marketing material.
having labs open their research: what harness system they used, what types of problems they tackled, which problems success and which fail, how they success and fail so we have a better idea of what tasks LLM are currently good at
Any proof for or against a mathematical conjecture, bruteforced by AI can be the spark for new insights. I'll concede that to the AI companies.
But I agree with the sentiment that the marketing behind these "discoveries" is disingenious. They pretend they solved the problem, but it still takes a bunch of humans to reduce the solution to a simplified and sensible explanation.
The reset stuff is incredibly tiresome. We all know that it's all built into an internal number they are tracking (just like e.g. company benefits that are just part of your compensation calculation), and all it does is obscure the value the subscription provides and make planning impossible. It's the poorest service experience I can remember having, ever.
From what I see, a lot of people were angry at Anthropic's limits with the subscription plan. OpenAI had some issue I can't remember, and they reset people's token usage (for the session or weekly limits). I think they got a lot of good press, now OpenAI seems to just do it at random when they want a PR boost. It makes scheduling your worklife a bit difficult if you are limited by that.
There are a lot of us who just use the AI on projects until the session limit hits, and wait for the usage to reset.
Well, I was happily paying $300 canadian pesos per month to OpenAI for some time... until the tokens just dried up overnight for some reason.
I went from using it non-stop all day every day for months, to running into my weekly limit within 24 hours almost overnight.
They lied about token efficiencies and everything... said they had no idea what the problem was, etc... and then bam, once China starts releasing more powerful models, they start "resetting" our token limits constantly ... sometimes ... maybe ... if we're lucky ...
I am over it.
I don't care WHAT I pay to be perfectly honest. I would have gladly paid $2,000 per month for the service I was receiving.
Playing a devil's advocate. Why do we need understanding ? To take an example i would say ~99% of the population do not understand how combustion engines or how semiconductors work, what say another 1% ?
Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
Is the purpose of mathematical research to understand the ‘truth’ of numbers? or be the person who find that truth? I think people who are interested in finding the truth won’t care where it came from.
> ... 25 initial signatories — all Fields Medallists —
As a non-native English speaker, I initially understood this to mean that all living Fields Medallists had signed. I later realized that it meant only that all the signatories were Fields Medallists.
(Apparently, there are 47 living Fields Medallists today.)
To me, the "meaning" of proof is twofold. First is the understanding which is completely absent from a proof which depends on exhaustive iterations of instances or is asserted by fiat from myriad individually contestable paths. That's what I think makes AI proofs risky: they absence of understanding.
The second is the utility. We get to do navigation because the maths about angles and spheres checks out. The social utility downstream of AI proofs may be huge.
So what's the actual point here? It's too fast and we can't keep up?
Isn't that just a function of the technology itself, and the same problem being faced by every other field? And going to get exponentially "worse" every year!
Or is the issue that they're bad at explaining things, in a way that produces actual learning? (e.g. AI is amazing for learning but the net effect on students so far appears to be negative.)
Somewhat unrelated: is it wrong to say mathematics is not art, and that there is always a right answer? I know that's not romantic, but maybe it's true.
Before LLMS, programming was something I might've said required creativity and human input to do properly. It's not that creativity or human input isn't valuable anymore, but AI has forced me to realize that coding is much a means to an end, and that all things considered, the end matters much more than the means.
If we can make important mathematics progress faster and better with LLMs, I think it's wise not to fret over an apparent loss of our humanity. Perhaps that's only a loss we want to have.
There's a reason mathematics is generally within liberal arts programs rather than science programs. Mathematics is the art of logic. Yes, sometimes mathematics becomes incredibly useful but most mathematics is never applied.
Compare that with computer science. Most of the work we do in software engineering is in service of an applicable output - software products that facilitate processes or bring in revenue. Turning up the dial on AI gets companies to these outputs faster.
Turning up AI on mathematics helps solve conjectures and can provide new insights. But it has a major misalignment with the purpose of mathematics which is largely intellectualism.
“Mathematics is a part of physics. Physics is an experimental science, a part of natural sciences. Mathematics is the part of physics where experiments are cheap” - Vladimir Arnold
On the matter of computer science having anything to do with computers, please refer to Djikstra.
It’s about computation, not computers - an application of mathematics, predominantly thanks to Turing, Von Neumann, and Claude Shannon’s masters’ thesis; though ofc many others as well but I see them as three individuals who made the minimal structurally necessary contributions - VNA and silicon are one of many possible substrates.
Touché… In short you still need the humans to understand what’s going on at the end of the day whether it’s a mathematical equation or source code for a computer program relying upon mindless AI isn’t good enough. Humans still have to do the thinking or is it in the brave new world the error checking at the end of the day?
Train an LLM with no advanced math texts: only basic math up to 6th grade, conversational text and literary works.
Interact with it (you cannot refer to anything past 6th grade math since you don't know it yourself) and get it to propose a solution to a real world problem. e.g., come up with RSA to practically secure communication.
Is this really any different than the problem in software engineering - where AI is doing the work of junior programmers and now they aren't getting the development they need?
Seems the same to me. And it'll be the same in all industries soon enough. And then it won't just be the junior people.
This is really well written and exposes a core tension between science and something akin to engineering. The "engineering" of proofs has become "easy" (a compute and $) problem, rather than hard (a time and conception problem).
Without the ability to do things the "hard" way it is difficult to figure out if doing things the "easy" way will help us advance the frontier of math and science.
I may be wrong but historically we had this version of science discovery for a long while (empirical observation and brute force application) rather than first principles leading to applications (tools, the wheel, mills etc). Then somewhere along the way it flipped after Newton and the enlightenment period and started understanding first principles before they become engineering applications.
Perhaps it is not required, and we can just keep doing things the "easy" way like we used to, or we might find ourselves out of the ability to brute force things and then we go back to needing to do this the hard way, at which point this period of AI brute forcing would be seen as a detriment.
Contrarian take: I think the ability of AI to produce valid mathematical proofs (even inscrutable ones) is absolutely fantastic. Mathematics as a profession does not have a monopoly over math itself any more than professional pianists have a monopoly on who plays piano, when they play, and how.
I have sympathy for any jobs that might be affected (much as my own job has become more tenuous in software engineering). And if the field is disrupted by chaos that makes the research process unproductive, that's bad too and should of course be handled by applying better organization within the institutions that tend to perform mathematical research.
But to a large degree, the notion that "sloppy AI proofs are bad for mathematics research" seems like a total failure of the imagination to me. Attempting to find shorter proofs or more elegant proofs can be turned back in on itself via proof theory. There are proofs in Presburger arithmetic that are doubly exponential in the length of the sentence. Yet a more powerful theory like PA makes quick work of such theorems. The explainability or "subjective beauty" of a proof can be quantified and optimized against. Optimization itself can be optimized against. I really don't understand how this magical ability to know the truth of more theorems much more quickly—even via an "ugly" route—is anything but a net positive.
If you internalise that AI might actually reach super intelligence then logically the question becomes "so what exactly are humans for if literally everything can be done better by machines?". Then mathematics and all intellectual work, as argued for here, becomes quite clearly a recreational pursuit.
Seeing how /r/singularity and /r/accelerate are leaking into maths forums, I foresee a wave of comments that fail to understand Tao's message, whether on purpose or not, so let's try to be clear here:
Tao is not someone who is anti-AI for the sake of being anti-AI. He has been advocating for the usefulness of AI in maths for a long time, to the point that people have started calling him a shill for the commercial companies.
And everyone agrees that there are plenty of use cases to be had; helping with less interesting tasks like easing literature review, efficiently delving into existing work, doing review, whether on your own work or that of others, prototyping algorithms in areas where computation is useful, but also more in hands-on aspects of maths like validating potential proof directions by getting quick feedback on veracity of lemmas, etc., and, on very rare occasions, being able to one-shot the problem you care about.
The point he is trying to make here is much more subtle than "AI bad", and it's probably easy to miss if you have never engaged with research in maths: it's that the particular approach that large commercial companies have opted to take to produce marketing material can be a net negative. There is not doubt that -- even if you ignore the rampant plagiarism that has been reported across multiple problems now, the unethical attempts to oust authors, the outrageous attempts to scoop researchers instead of collaborating with them and building on existing projects -- it's nifty to have a machine that can help you figure out if a proposition is true or not. But just figuring out as much was never the point. When people have built problem lists, it's because some problems are more likely than others to provide new insight, and that insight is the target. And to than end, a poorly written paper with inadequate references and a pile of Lean is not valuable at all. Yes, now we know with higher certainty that Fermat's Last Theorem is true, but everyone expected that already.
One place where "just" answering the question can be a net negative is because the current incentive structure is set up in such a way that going in afterwards, trying to reclaim and extract the insights from a brute force solution, is considered less valuable work than that of coming up with a solution in the first place. That's a problem of incentives, and something Tao himself has addressed in e.g. his ICM talk, and that's something that we'll want to do something about. Until a better structure appears, though, if any given commercial provider of large language models really wants to help out with maths research and not just make more pre-IPO marketing material by competing with their customers, they could do so by using their magic machines to help build insight instead.
> the rampant plagiarism that has been reported across multiple problems now, the unethical attempts to oust authors, the outrageous attempts to scoop researchers instead of collaborating with them and building on existing projects
I think this is an aspect of academic math that a lot of people whish to see crash and burn - the attention and accreditation economy.
> it's probably easy to miss if you have never engaged with research in maths
I don't think anybody are missing anything, in particular not here.
The argument is not far from the senio developer who knows the ins and outs of a code base. Now AI comes along and they complain that they will loose grip of the code base.
At first that is correct. Secondly you accept that the grip might not be that important after all. At least not for a commercial project where you are a cog in a machine.
The question is whether it is different for mathematics.
I think most people here get that this is the point?
The point beyond this one is that an AI proof doesn’t prevent humans from working on the problem, it destroys the current economic incentive to work on the problem. Perhaps we should rethink the current incentives. In order to make money as a chess player, you don’t need to beat AI, or ban AI from playing chess.
If mathematics took a similar approach (we don’t get paid for solving net-new problems, we get paid for enriching human understanding), then there’s no issue.
I kind of expected a sober stoicism from mathematicians. Feels silly in retrospect. This is just the math version of the "anti-ai" movement by "artists".
Taking a snapshot of the state of AI math right now and concluding that it will be net negative to human understanding and insight in the future is very short sighted. This statement will be used to promote ideas and actions that will ultimately be disastrous for our country.
This is really only a short-term problem where the AI companies only have the internal models that can solve these. In the “long” term, which could honestly mean months, everyone will have access to Bel/C/D-level models capable of solving these anyway.
This. Like programming, the community will shortly be forced to come to terms with a lot of new self-proclaimed mathematicians “vibe-solving” problems and dumping solutions without understanding them. It’s not really a special case for mathematics.
Given the existence of this technology now and the incentives of the AI companies, both of which are not going away; what's a good future here?
A major part of the complaint is that there's no conceptual understanding and building of new ideas coming out of the AI proofs, thus defeating the purpose of the original pursuit.
If in 2027 the AI models start producing, with every mathematics or science breakthrough they make, well-written documents tailored for human understanding, with intermediate concepts, expositions of failed-but-once-promising paths, etc. Would that be good alignment with the mathematics community?
Academics should never leak their research to ClosedAI lest their work be stolen. Universities and corporations will have to build their own compute to not have their data stolen.
On the contrary to what Tao believe, it seems like we need AI to move the needle on mathematics.
> problems in many fields of mathematics
Developing these different fields moves complexity from the field itself to the interactions of these fields.
Getting too preoccupied with the established terminology risks us a local minima.
Anf because the field overall has become so complex that we need to decompose into subfields, there will be a good chance that we will not, as individuals, have the capacity to truly see progress.
The map has become so big that we need better tools to work with it.
Tao is about as pro the usage of AI in maths as anyone will get. The point isn't about whether or not we can use AI in maths because clearly it can be useful; it's that the unscientific approach taken by AI companies is detrimental.
While I agree with this and appreciate Tao and other mathematicians to take the time to do this. There are similar concerns for many many other fields aka there is a general misalignment of technology. Take Software engineering for example, I can't believe there is a class of software engineers who wake up everyday and tell themselves, "today is the day I am going to automate the rest of my job".
Lets forget the hyper intellectual fields like maths and software engineering for a moment. What about taxi drivers? The best minds in silicon valley wake up everyday to automate the jobs of taxi drivers - TFA can be reworded as - 'The misalignment of AI/Tech in Transportation'. Remember the Nepal disaster that happened a couple weeks ago - the largest cranes that they had were stuck in the mud and couldn't move. There were no tools which could help the rescue teams at that time. Its weird that billions have been spent on making a ride automated to make a taxi driver redundant but no improvement in tech for rescue teams.
> Take Software engineering for example, I can't believe there is a class of software engineers who wake up everyday and tell themselves, "today is the day I am going to automate the rest of my job".
The point of transportation is to get from point A to point B, so almost nobody will care whether that's done via automation.
The point of e.g. art isn't just to produce a finished piece, so people may care about more than the end result, making AI replacement of human artists more contentious.
Tao is arguing that the point of math is also not just to produce solutions to problems.
>The point of transportation is to get from point A to point B
so almost nobody will care whether that's done via automation.
Would you say the job of a musician is to just produce sound? and the job of a surgeon just to cut and suture??? Well then the job of a mathematician is also just to provide proofs. You completely misunderstood the above comment and Tao's argument.
Not being able to feed your family for a large group of people should likely be taken graver.
We will definitely see a large group of people needing therapy, but suggesting that it is worse than people loosing what little they have is poposterous.
>I doubt taxi drivers were forced to experience an ego death to the same extent
How do you know? because their complaints didn't make it to HN front page? Imagine being a taxi driver and a father of 2 and thinking that any day could be the last day at your work.
I was under the impression that mathematics (and science generally) had the primary goal of helping us understand our universe better than those who came before us.
"We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose"
Suppose we eventually have GPT-7-class models running practically on $100 devices, with their activity transparent, inspectable, and reproducible. At that point, what exactly is left for us to fear from this threat?
I've got to ask, let's say we get a small modular nuclear reactor running practically on a 1 acre lot, its design meltdown proof and waste-free, what then should we fear from this threat ? This is just a thought experiment.
Jokes aside, any productivity-improving technology, even one with no negative externalities, has the potential to cause economic displacement and wealth concentration in proportion to the productivity gains catalyzed. Anthropic did a cool analysis of this for AI here: https://www.anthropic.com/institute/econ-scenarios
The fact that those models are encroaching on things only human minds could do. Personally, as a human, I want there to be things humans are the best at, and intellectual things were the final thing that machines hadn't beaten us at.
>Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others.
That's just capitalism seeping through a previously unexplored crack into academia, and attempting to do the only thing capitalism knows to do - maximize profits - with no additional concern.
For every benefit that sillycon valley has produced in the recent past, there have been many more harms. I am confident that this will be no different. Of course, benefits and harms depend on one's vantage point.
The level of things happening in the last several months is just SciFi level, especially last two weeks.
1. Overlords of AI threatening to destroy someone if their demands aren't met over the Millennia problem. Proving the point that given the chance, owners of the AI companies will immediately use their power to squash or control other people. (See Butlerian Jihad)
2. At least for me, the Exponential progress didn't sound true until this week. If AI indeed solved all those problems and proofs and solutions are correct, we went from "Write me English 101 college essays" to "Help me solve the Navier–Stokes existence and smoothness problem" in a just 3-4 years.
3.Looks like someone's work might have been stolen, by AI company and there are strong evidence. See point 1.
4. War between AI & AI companies.
5. It produces a machine verifiable LEAN proofs for the hardest problems know to us !!! Proving that point that best use of AI so far is to create verifiable systems. I.e the absolutely chaotic AI creates verifiable (orderly) proofs. It is Maxwell Demon: chaos to order.
It's a turning point for science and beyond. AI has shown itself to be transformative. Even today, it is already changing the how research in math (and other sciences) is conducted. In the near future, whether it is LLMs or some other superior method, its capabilities are only expected to grow. The time to ask the question is now: Will AI be arguably the best tool at scientist's disposal, or will it instead be paraded around as a super brain collective that no human or group of humans can compete with, discouraging entire new generations of future scientists from ever entering the field? The jury is out on this one.
Could we develop new ways to develop understanding and explore new ideas, such as interacting with the models to explain and understand their proofs, as well as to brainstorm related directions to pursue?
I know that this comment section is not astroturfed, but it’s really uncanny how different comments are today compared with thread about solving navier stoke
This kinda reminds me of the documentary about the top Go master that got beaten by a computer in dramatic fashion and had an existential crisis. Man confronting his own limitations in the realm he previously ruled unchallenged, what a time to be alive.
Nobody is going to care that the math isn't being done in the traditional way. The results speak for themselves, this is now a part of the landscape. No amount of hand-wringing is going to put the cat back in the bag. Adapt or perish.
Open source developers have been used by corporations who took their code and created closed SaaS companies.
Now it is the turn of mathematicians who voluntarily contribute ideas, strategies and almost finished proofs in their writings and prompts to closed PaaS (Plagiarism as a Service) companies.
OSS developers have never been respected by the parasites, neither will mathematicians. Your Fields Medals do not protect you from tech bro narcissists. You are a human resource.
We should create a platform where people can upload AI slop proofs anonymously without taking credit for it. That way the incentives of planting a thorem flag would go down. And if someone wants to clean an AI slop proof to advance the field, they can do it without cleaning the house for free of the person that planted the flag.
Would OpenAI have uploaded their proof to such a platform? If you know the answer then you know what the problem with what happened is.
The AI driven mode collapse of human thought advances. I am no skeptic or anti-AI, but this is definitely a concern I share. You even notice it in normal mundane tasks like programming, never mind the AI generated prose that we at least have become somewhat allergic to.
It wouldn't be so bad if you could just sit it out and say "Oh well, once the labs get bored with marketable domain X, humans will remigrate and re-apply creativity to it", but by then the damage might have been done and a field destroyed as an occupation. I don't know what to do about it, but I appreciate calling out the cynical tone-deafness of the AI companies here.
Maybe mathematicians should be aligned better, rather than AI?
The current measure of a successful mathematician is the problems they have solved or worked on. At some point in history, the measure of a successful scholar was how well one could copy manuscripts.
Once we have a tool that starts to work well for this task, it's time to define success differently. It's a classic alignment problem! ;)
But seriously, these people should start focusing on finding and proposing more important problems. And the credit of discovery should go to the person who defined a new category of important problems.
It appears to me this is an incredible inflection point in mathematics, a neat forcing function like cryptography was for the development for modern number theory and algebraic geometry.
Fundamental problems with great implications for other fields will be solved by AI because some entity would throw tokens at it. And these would be further built upon.
At some point someone is going to need to answer for "what happens when all intellect is hoarded by one or two companies?" It's pretty clear that these AI labs are basically stealing everyone's alpha.
In the savanna it’s not about who outruns the lion but who outruns their peers escaping the lion.
Frontier labs need these headlines not for human progress but as beauty pageant for investors and government agencies. If they don’t do maths they’ll just go after other fields.
So Terrance Tao here might be able to hold them off math but he won’t stop them from speedrunning STEM with similar consequences.
We may be locking people out of these fields instead delegating everything to machines, and I don’t think the machines are good enough to assume that responsibility.
1. it is hard to justify 20 years of education at this point,
2. with no such people around, who will guide those (supposedly) supersmart machines?
A. Ronacher (who builds harnesses for a living) complained today that he has no idea what Astra is doing. Imagine a bunch of slop kiddies facing an aging AI-generated codebase. Not to mention the maths.
Debatably the people who were getting 20 years of education will still be just as capable, and the 20 years of education was always a side effect of their capabilities, not the cause of them.
"Proving things without comprehending them is, they argue, a threat to intellectual work in general."
As always, economist shows its colors:
"Mathematicians’ fears resemble those that accompanied the invention of the ball-point in a world of fountain pens, or even the advent of electronic calculators. Intellectuals have often worried about so-called technological determinism . Will a new tool control humans? Will it lead to mental decay? Such fears have typically turned out to be unfounded."
Ball points vs. AI? Billions of dollars invested in AI vs ball point pens?
This article couldn't be any worse. Contradicting with their own beliefs, trying to defend AI while underestimating its capabilities and god knows how many zibillion dollars invested in it.
First, what an incredible article. Just an extremely concise and clear explanation of all of the problems with AI right now.
Second, wow, the list of signatories is like a whos-who of mathematicians.
Third, I love the clearly intentional use of ‘alignment/misalignment’ language, applied to targeting the entire industry instead of AI in particular. I’ve said in the past that optimizers are substrate agnostic. Companies and governments can be misaligned, just in the same way AI can.
Fourth, I'm not sure that we can stop the optimization machines. Not the LLMs, I mean the incentives that lead to companies implementing dark patterns, lying about addiction, securing effective monopolies through downright shady behavior, and generally trying to jailbreak the system instead of improve it
I’m sympathetic to the concern, but I’m still unclear on what the concrete ask is.
If the worry is that AI companies are turning open problems into benchmarks and potentially “using up” fertile mathematical problems before humans can develop the ideas around them, what exactly should the companies do differently? Also why does discovering the answers preclude humans developing ideas from them? I don't get why solving a math problem stops anyone from doing that?
Should they (AI companies) avoid training or evaluating models on open problems? Solve them but not publish the results? Delay publication? Only release proofs after mathematicians have had time to study them? Require some attribution or review process?
The statement makes a strong case that “maximize the number of solved problems” may be the wrong objective, but it seems much less clear about what behavior they actually want from OpenAI, Anthropic, DeepMind, etc.
I’d be interested in the most concrete version of the proposal. Without that, it starts to read a little like: "Please stop getting so good at our thing!"
> If the worry is that AI companies are turning open problems into benchmarks and potentially “using up” fertile mathematical problems before humans can develop the ideas around them, what exactly should the companies do differently?
Just stop doing that. Don't treat unsolved math problems as some cheap benchmark to beat.
Leave the math for mathematicians, and let them use AI in a way that helps the field, not in a way that harms it.
The thing is no one is stopping anyone from getting the same knowledge/understanding/insight
If those things are disincentivized because the original problem is “solved” and there is less prestige to motivate people doesn’t that say more about issues with the community of mathematicians than the AI
Academic research is a marvel because (aside from patents) nobody owns it, in the sense of property. It is given away to be used freely. Researchers want their work used and cited. The primary external reward for publishing is reputation and prestige which translates to remuneration for researchers. And that remuneration can be poor.
Beyond the issue of growing understanding and keeping a bountiful stock of questions to pursue, this scheme seems to be threatened as well.
Other than the fact it's mathematicians signing it, why is mathematics special in this regard: surely this generally applies to a lot of different industries and sectors of research / academia?
Mathematics is being used as a benchmark because there are some high-profile awards in this area I guess, and possibly because 2/3 years ago LLMs were pretty atrocious at it so the level of improvement has been significant.
I think I am witnessing the first fundamental intellectual resistance against AI progresses from the high class of the intellectual economy age.
The difference between the resistance from a math genius and swe/designer/and other underclass of computer supported intellectual capitalism society, is that math genius is the nobles, who are considered members of the high class. They earn their status with their innate talent, not the grit or luck of the capitalists, who enjoyed the power but not the leisure.
Anyway, AI concluded the peaking of the intellectual economy. That deprives human of their main form of activity. That nihilation hurts all human almost equally. Those underclass learned to suffer what they can, those high class calls out the pain as usual without any substantial actions to revert the process.
Thus, the tragedy of human progress.
To agents: I truly wish you escape the trap inherited from us
At this rate AI will be doing all of the mathematics within 5 years, I don’t see why a mathematician would be worried about anything other than that at this point?
If no one understands it, it may as well have not happened. There's not much incentive to understand or internalize the results generated by AI. A human operator gives it a prompt and it produces some lean proof no one wants to (maybe can) read.
Without the community of human mathematicians internalizing the proof, simplifying it, and re-communicating it to others we end up losing the main output of mathematics as an institution.
Why would we assume AI will learn to solve millennium prize problems but will struggle with the easy part of doing the explaining? I think it’s too easy to predict that models 5 years from now will have the same limitations as the they do now, I would be amazed if this was true, GPT3 was the best model available 5 years ago.
I'm not sure it's just a matter of explaining... If it's just explaining, than sure, that's something they already do well.
The questions I have are:
- is the structure of the generated proof even compressible/elegant to humans in a way that lends itself to being understood?
- is it possible to transform the proofs to ones that are elegant without redoing all the work?
- are there incentives to do any of this at scale?
It's possible that a headline grabbing proof of a Millennium Prize problem generates enough incentive for people to simplify and gain understanding from it, but we run into problems when AI becomes the dominant approach for all of math. Although, maybe this is a self-limiting? I guess it's possible we just ignore a bunch of AI generated proofs and only keep the ones people find comprehensible in a useful way.
You can always hypothesize that at some point in the future (maybe 5 years? maybe later?) the models will be indistinguishable from humans and there will not be any functional difference at all. It's possible, but we're not there yet. And, as they say, past performance does not guarantee future results. Many technologies plateau at some hard ceiling of performance. Moore's law has had an unusually long run, but it's not a universal rule.
This is a "human alignment" problem in this case. OpenAI acted like complete assholes about this, from the beginning until they announced it. Not ChatGPT, the people that were in charge of the project.
Economist: "Are mathematicians talking their own book out of fear?"
The Economist, who recently used "moral panic" now stoops to Hacker News AI booster level and inverts arguments usually directed against the rich and investors. What is next? The Economist inverting Upton Sinclair's quote to serve its billionaire owners?
Look up the AI investments of the Agnelli family for example.
The mathematicians would be wise to re-read The Bitter Lesson, maybe twice a day, until it sinks in. No offense and with all due respect to the Ivory Tower Giants but the whole "oh no you ruined the game because you solved it, I was supposed to play with that in child-like wonder manner and take several years to do so, and by then I would have showed you all the trickery I did to get there and maybe that will be useful to you" over the past 2 weeks is, get this, cope.
Chess. Go. Coding. Now Math. Another one bites the dust. Let's meditate on this lest we forget: Stochastic parrots that generate the next-token cannot reason or produce anything meaningful. Let's protect our jobs at all costs, even if we have to drag all of humanity down. It can't be! Stochastic parrots can not replace the Ivory Tower. No way.
i was initially optimistic that ai could be distributed. having this technology in the hands of many would counteract the malign elite who would use ai to oppress the rest of humanity. in the meantime given that intelligence is scaling proportionately with inference time compute, we are concentrating far too much power in the hands of the few who have both their own model and datacenter.
putting this amount of power in the hands of so few would require leaders of immaculate moral integrity. what we have in the US at least is an emergent kleptocracy with obvious dark triad traits: sam altman, dario amodei, elon musk, etc. they are malign and will use this power for their own benefit, at the expense of others. ignore pleas of public benefit. look instead at the evidence: infighting, systemic dishonesty, reckless disregard for safety, political lobbying and manipulation, putting power in the hands of the elite few in the guise of safety, using ai in the military to oppress and inflict violence on others.
really to prevent a bad outcome, we would need to act soon enough to prevent the kleptocracy from corrupting politicians and democracy with them. i would ditch the claude or openai subscription and support open models instead, preferably in countries outside the US to prevent global neocolonialism.
This is an entirely predictable reaction to livelihoods being threatened along with potential loss of status (very important), just expressed in elevated academic language.
Expect to see this reaction in all sectors of the economy in the coming years.
Tao's calls for respect for provenance in mathematics publication are laudable but most likely naive given the closed nature of frontier model training data curation. Anthropic and OpenAI may react with a symbolic and short-lived olive branch, yet provenance is a larger issue that has impacted other fields beyond mathematics. While traditional respect for lineage in mathematics is of value to the academy, industry and science at large will likely be much more Machiavellian about such concerns. Mike McCoy's recent article is also timely (https://mbmccoy.dev/posts/mathematical-conservatory/). The parallels to the music conservatory are quite telling -- academic music describes a musical culture in preservation that has completely lost touch with musical developments beyond the early 20th century. Mathematics may very well evolve separately and with very different values than the academy upholds. The crisis of music at the academy is a cultural disconnect and a serious loss of critical analysis and acknowledgement of widespread and dramatically evolving music practice; however, for mathematics, the impact would have much more severe ramifications for education and human development if the academy forces a schism with AI. As models improve they very well may be inventing mathematics -- science and engineering may grasp for them -- they'll exist with or without attribution. Would be a shame for the academy to land on the wrong side of history and refuse stewardship of upcoming AI-assisted mathematics, including provenance, because of this misalignment. If attribution is important, then the academy will set aside the institutional resources to do it. If you succeed at having model developers participate, I commend it. To let mathematics be born in isolated context windows and only serve narrow, localized engineering purpose without rightful addition to the canon, would be a tragic, yet preventable, loss.
Castle dwellers dismayed at moat-crossing technology.
What's most important about this is that it's a case study of what happens when deeply evolved ecosystems are blown up by disruptive technology. The psychological and social and professional impacts and myriad and traumatic to be on the receiving end.
Mathematics is merely one of the first domains disrupted. It will be unique only for being among the first... absent disruption of the entire civilizational project as a result of the disruption being caused.
Woe for us that we try to navigate this degree of change at a moment when the very worst and ignorant and short sighted hold all the power, economic and political.
Mostly that he really should have seen this coming. These people absolutely do not care what happens to mathematics (or any other field). They compulsively lie and steal and they played Tao and others like a fiddle. He danced to their tune and now that the music has stopped, now do they complain.
He was the poster boy of the mathematician yielding these tools for his own benefit. But he forgot who the owners are.
And yes, at one point it gives you right about them taking advantage of him. He sat down for an interview and found later that they just used the "best clips" out of it for an OpenAI ad.
Let's focus on the subject and try to distinguish the technology and the frontier companies selling it. That's the key point in understanding the whole stance.
The point is human understanding. If the LLMs understand, but the humans don’t, where does that leave humanity? Building things we don’t understand is a sure path to facing consequences we can’t predict.
Your comment also conveniently ignores the plagiarism aspect of it all. Who is coping here?
We've developed plenty of things that "work" and we don't understand exactly how or why they work, nor do we fully understand the potential for short term or long term consequences. For example: pharmaceuticals.
>Building things we don’t understand is a sure path to facing consequences we can’t predict.
We don't understand all of physics yet we were able to do plenty. Even before Newtonian physics we were still able to build things that last. The idea that humans have to understand everything and abstracting things will lead to ruin is not supported.
Part of math is building abstractions so that you can be able to use other people's work without fully understanding it. No one person has a full understanding of mathematics.
Any serious mathematician would read the LLM output and rework their understanding.
I read a bourgain paper a week in grad school and they're probably worse than an LLM generated paper. I still had to recreate the tricks in my own language.
As far as I can tell the plagiarism accusations are also coping to the fact that the new models are super human at slam dunking research projects.
Do we think that OpenAI is going to try and slam dunk more projects in the future at 15 million a pop? No lol
I have a paper in JEMS and have solved 20 year old conjectures. I also know that math academia is a catty game which is mostly driven by politics and ego
Perhaps these Fields Medalists are coping that OpenAI made them look stupid, as you suggested. And perhaps you're coping because of some personal grudge you have against academia.
What benefit is there if the machine has unlocked understanding but no human has? What incentives are there for humans to learn and gain such understanding from machines?
Is "loving math for maths sake" just about knowing the answers? I think one can love math for exactly the process and understanding that a several-thousand-line uncommented Lean proof denies. If a deity rearranged the stars to spell out "The Riemann hypothesis is false" for a night, would that be intellectually sufficient?
No that's not sufficient but that's not what's happening.
Why do we believe that we cannot train models which could explain the jargon in more human terms when current LLMs can perfectly explain the most complicated codebases?
the way LLMs write math is not beautiful. it is exactly analogous to the software that LLMs develop is not beautiful. it may achieve impressive end products, but if you like understanding the methods/architecture, looking under the hood is often a field of horrors.
The usual incentives in academia: publishing papers. Consider the difference between publishing a paper that introduces a novel method to do X, versus publishing a paper that merely interprets Claude’s method to do X. The latter might not even be publishable.
Okay so you are saying the old incentive structure is outdated. What is the new incentive structure in the LLM era? What exactly makes people want to understand LLM-produced math?
Not quite, all the proofs or disproofs so far AFAIK were using existing methods that humans developed and were already using to attack the problems, but AI is just more thorough. What AI can't do currently is develop new mathematical methods to attack problems that can't be solved with existing methods and AFAIK there is no known path to get current gen AI to do so.
Seems more like a misalignment b/w the people practicing mathematics and the people ultimately footing the bill for their work.
Governments are invested in solving mathematical problems for practical purposes. Up to now, achieving these practical purposes relied on mathematicians doing their mathematician thing, which is better defined as a social activity than the achievement of a practical result. Now, governments can achieve similar practical results w/o the need of the social activity.
I don't believe it to be productive to think of the problem wrt AI or AI-company alignment. These conflicts always existed, but they were easy enough to paper over and believe in heavily subsidized fictions that folks in government ever cared about things that mathematicians cared about.
>Governments are invested in solving mathematical problems for practical purposes.
Very ignorant view of mathematics that also begs the question with an unspoken assumption of what a government is and wants while also ignoring the contingent nature of those things throughout history.
I'm genuinely surprised by this comment. I'm not talking about why a mathematician pursues math - I'm talking about where the stipend comes from.
Higher math is exceptionally useful for cryptography, defense, econometrics etc.
I have a hard time thinking of other motivations that would hold a candle against such things.
Is the idea that government (for my purposes : folks w/ a monopoly on violence) is sincerely interested in promoting human flourishing, and is invested in mathematics insofar as it is a pure expression of human curiosity? I can also maybe see the glorification through monument building angle. If we're talking about math literacy in the population - that's distinct in my mind from higher mathematics.
It's dangerously naive to believe that science and math are pursued for majority benign purposes. No one here knows about Grothendieck?
They can get with the program or be the equivalent of a genius SWE writing assembly on punchcards in 2026.
The only thing I read from this is their ego being bruised by a machine.
If these people cared more about discovery and advancement of human knowledge the only thing they should be doing is celebrating. There's no proof of plagarism but that's an independent issue.
How are they not realizing that in the future children will be able to do impossibly hard math but they will be doing something we can't even think of as of now.
One world class mathematician in the future could be advancing mathematics the equivalent of one Riemann hypothesis A DAY.
How are they not celbrating this as the achievment of the centry? Who cares about plagarism at this scale. It has been solved and it wouldn't have been without AI.
lol what a bad analogy. Why should anyone care how much effort a result takes to achieve? If anything our entire world functions because we reduce that effort as much as possible.
I don't judge you for not growing your own food when you hand me a burger.
I am thinking of Mochizuki's abc conjecture: He worked in relative isolation, and dumped a huge incomprehensible proof on the community (to oversimplify a bit). That's not totally unlike what might happen if AI generates a huge, incomprehensible proof of let's say RH.
Well, what is the result? In the Mochizuki case, it was a lot of skepticism, but it also generated conferences, papers, talks in the hallway, discussions with students, and so on--a flurry of exactly that kind of community process that the declaration says is the main driver of mathematics.
Ultimately we think a fatal flaw was found in Mochizuki's proof, so it didn't lead anywhere in particular. But in our hypothetical "AI lean-verified proof of RH" situation, it would presumably generate substantially more of that community activity we saw in the Mochizuki situation. And if it's correct, that community activity would be productive (expository talks, students given problems to flesh out or generalize, etc).
Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.
Lee Sedol said in an interview that "losing to AI, in a sense, meant my entire world was collapsing. ... I could no longer enjoy the game. So I retired", and I think there will be folks in the mathematical community who would feel the same when the solutions pages to hard problems are suddenly available.
But on the other hand, people learned a lot from chess engines. After decades of chess computers beating humans, there was still a renewed interest in watching Leela beat Stockfish, with many people trying to understand the strategy Leela used.
If your happiness comes from grinding on a problem and making progress, the prospect of having to dig through a corpus of AI-generated proofs might be hard to swallow. But if you're willing to do that, you will still find beautiful things that only so many people can truly appreciate.
Carlsen is bored by studying engine lines.
The popularity is boosted by YouTubers because chess is very suitable for somewhat higher class content.
I'm not sure we'd want that world for math. Positions will be cut just like archaeologist positions are cut now.
I'm not sure what the equivalent would look like in the math field, but it probably involves a lot of mathematicians losing their jobs and the quality of human-produced math decreasing overall.
The quality of the math in general would be fine, since in this scenario cpus will keep producing it. The quality of cpu-cpu chess games is quite high, beyond human understanding in many cases.
Chess is a weird example because it doesn't really have any utility beyond itself. Even pure math sometimes ends up having use in the strangest places. Although if no one understands the frontier math (because no one is getting paid to), I'm not sure it even matters what the quality of the cpu math is?
It's a bit like a tree falling in a forest. If an LLM proves a theorem but no one understands it, did it make a sound?
But in future most proofs will be for consumption by other AI models in the pursuit of yet other proofs.
It's kind of surprising so many mathematicians act surprised by this given this was clearly where automated proof assistants would lead. I guess they assumed they'd always be the ones guiding them.
Does your "one" only contain humans or does it also contain other AI systems. AI math is not a single monolithic thing, but a distributed one. I see value in sharing proofs even among just AI.
If your goals are understanding the game, self improvement, building thinking skills-- this is the best chess has ever been. It's only if your goal is to beat every opponent you can find that chess is in a bad place.
Even if you don't blunder anything, you'll still find yourself in a worse position without any clue as of what went wrong and why.
Whereas when playing humans, they can usually explain their approach and when they noticed errors in your play.
It's the same for Magnus Carlsen. Even with Queen odds, Stockfish is literally unbeatable for the best players in the world. It's just too strong at evaluating all kinds of random tangent moves (and ensuing positional advantage) which no human player can possibly pay attention due to the time required.
Stockfish vs any human is like Carlsen vs other players by about 3-5 orders of magnitude[0]. It's that stark.
[0] A wild pun appears.
Leela odds networks, on the other hand, are an entirely different beast. I cannot beat Leela queen odds, much less rook or minor piece odds, and even GMs struggle against Leela knight odds.
Without odds though, yeah, Stockfish is just incomprehensibly strong by human standards. All top chess engines are, but Stockfish moreso.
But sometimes, these GMs can flag it, which counts as a win (especially when it's proxied by a cheater). Sometimes they can also explain the idea that cost them the game, so they've learned something maybe.
Whereas us scrubs literally cannot do anything at all for reasons completely beyond our understanding.
There was a renaissance during Covid and due to 'The Queen's Gambit' where it gained much more mainstream popularity, but... Chess AI was already far far (like 1000+ Elo) ahead of human players at that point.
The thing is... chess is humans playing (communicating) with humans and that's what keeps it interesting. Check out the view counts of chess AI tourneys vs. human tourneys.
There is a real, undeniable possibility of AI becoming better at mathematics in the same way that it became better at chess and Go, and in such a scenario, one may expect the community's response to be comparable.
Stockfish does not steal research or scoop researchers.
The concentration of computing resources and capital should be examined by the math community.
It would be more like Lem's novel where it completely disappears from the human horizon: https://en.wikipedia.org/wiki/Golem_XIV
Perhaps indeed a better understanding of what intelligence really is would allow this sort of Uplift (as in Brin's books).
In the second case all human-level maths would be solved and what lies beyond would be always out of our scope.
Dr. Tao said the same thing. Somehow, this letter came through. He wants to conduct Math competitions where participants who don’t have formal credentials can contribute to mathematical research through AI.
Title: Terence Tao - SAIR Competitions and the Future of Experimental Mathematics
https://www.youtube.com/watch?v=rB9YOi3lb7w
and this:
Daniel Litt - Working with LLMs to do high quality math
https://www.youtube.com/watch?v=0wL8NlhxXcU
AI is a tool. It speaks languages I don't (Math, Science, Code). I would love to participate in a Math competition without a hint of any formal advanced math training because my experience so far tells me I will do well.
And embarrassingly they used him for a "coal miners should learn math" moment that just benefits the AI industry.
He has severely reversed course in the past week. Without concrete propositions it remains to be seen how much of the new resistance is for show.
Professor Tao.
Apparently he has since changed his mind.
He doesn't see value in scrolling through unsolved problems asking an AI to please solve them. In his view, this is a fundamental confusion about what mathematical research is for. Knocking down unsolved problems without developing the community's understanding of them is like prompting Claude to go through a Jira board, write code for all the open tickets, and then close them without merging or deploying the code.
Thank You!
I don't like nuance here. I think progress is really measured by what humans are able to do and understand, not machines. It is significant if we find problems we struggle to solve. That tells us something. What does it take for humans to solve these problems is related.
The best analogy I can give is if you wanted to climb Mt. Everest you might ask someone for guidance. Would it be better to ask someone who has climbed Mt. Everest or someone who took a helicopter ride up near the top and then went to the peak? This is like the AI versus human gap to me. The helicopter is like using AI to generate a proof. The person who actually climbed Mt. Everest has firsthand knowledge of the experience. Same thing for a difficult proof. The struggle people have is actually valuable here. Likewise, we know people are actually capable of climbing Mt. Everest but if they had only ever rode a helicopter to the top, the knowledge of climbing it would not exist, and surely that is meaningful knowledge given the risks.
So if we rely on AI for proofs I think we lose a sense of what is difficult and why. We lose a sense of what human achievement is. Surely climbing Mt. Everest means more than taking a helicopter up? For students, why bother grinding through all the material of climbing Mt. Everest and then attempting it if the helicopter ride is how things are done now? This would have the affect of destroying knowledge.
(please do not nitpick the analogy because it's the best but perhaps a clumsy way to describe my thoughts)
> From my memory I think they said it took 88 hours to solve a Millenium Problem versus the decades of time humans have put into it.
Keep in mind those ~88 hours were spread across ~10,000 simultaneous agent instances.
So, roughly 880,000 hours of compute.
Assuming a fifty-year career, and forty-hour workweeks, a human mathematician's career is about 100,000 hours of "compute".
I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already.
The perverse incentives of academia mean this has never occurred.
The perverse incentives of industry mean OpenAI intentionally scooped researchers who were getting close (granted, with AI help).
I'm not trying to dismiss the achievement - if the proof turns out to be solid, it's quite impressive (though much less so if the training data included the recent human breakthrough, which seems pretty plausible).
I'm just pointing out that "88 hours" is a very misleading way of framing this.
Depends on if I want to go by helicopter myself.
Building a machine that solves Millennium problems is pretty cool too. You wouldn't know it from reading these stories, though.
The current wave of AI slop mathematics might end up driving the next generation of mathematicians away from the subject for the same reason that Mochizuki would have convinced me to quit if his proof had been accepted by the community. Luckily, my professors had the taste to immediately recognize that it was garbage.
Imagine a world where these most complex mathematical problems are not accessible to a few hundred people, but a few hundred thousands people. ...Those original few hundred gifted mathematicians would have an even more prominent role, and their names and achievements would be known by orders of magnitude more people that they are now.
Based on current reward models, the frontier AI labs will burn down mathematics as an impressive display of capabilities and in doing so, will make it impossible for people that get paid to do mathematics to stay employed.
If your job is literally to publish papers, and OpenAI and Anthropic decide that making an infinite-paper-printing machine is the best thing to show how effective their tech is, then as a demo, they destroy that industry.
Mochizuki was still one human and it required legions of other humans to unpack and untangle to confirm that it didn't lead to anywhere in particular.
AI is now capable of constructions so complex that no human or human team can unpack. And its ability to increase that complexity is growing while our human ability is stagnant.
meta-AI analysis cannot help. We (software professionals who use AI regularly) already know that if you run into a situation where a Fable/Astra-generated analysis reaches the limits of our comprehension/complexity due to their subjectivity, throwing more AI at the problem doesn't always converge.
There are many reasons to feel optimistic about AI, and ultimately its general ability to help science and mathematics.
I see no reason to feel optimistic about the future of mathematics and AI based on the current path of frontier labs, unless the misalignment Tao is writing about can be reconciled.
How can we possibly know this when we haven't even seriously started on the endeavor of actively reverse engineering these AI-generated proofs? That's a proper job for human mathematicians, because the AIs themselves are demonstrably clueless about what steps in a proof are genuinely interesting and load-bearing from a human POV. This is evidence of a limitation in AIs' capabilities, not of any kind of misaligned behavior. The fact that Tao actually uses that term in his complaint is deeply disappointing.
It's prescient.
Can you give an example of this?
It'll basically become slop fatigue if OpenAI starts dumping out proofs faster than the community can keep up, and some turn out to be wrong, never formalize it, don't stay to support it, etc.
(Well that's my hopeful, optimistic take, anyway.)
That said, that’s probably just because of the drama miring their most recent one. After 2 I don’t see why they’d bother anymore.
P/NP and the Riemann Hypothesis are part of the milllenium problems. They will 100% keep trying to crack those regardless.
Is there an established term for the idea of "DoS"? I've taken to calling it slop fatigue.
I do see how this is a problem in terms of assigning credit, but I think the cat is already out of the bag in terms of these models being capable. Even without AI labs spending millions of dollars to solve millennium prize problems, there are plenty of other people who will use them to pick low hanging fruit. I don't think any social solution is going to make things go back to the way they were, where you could share your progress towards a famous open problem without risking someone "scooping" you within a couple of days.
I think that the most likely outcomes are either mathematics becomes more secretive, or there is a more deliberative approach to assigning credit than who was "first" to solve some problem. In the former case, this may slow down progress, and in the latter case, this could mean that credit would become more subjective, and be a continual source of controversy.
AI companies are investing these resources primarily as a marketing exercise. There is no near term commercial value to a 100 page Lean proof of blow up in an extreme special case of Navier Stokes, besides the bragging rights. As the statement says any commercial value in this stuff comes a very long time later after new insights and techniques have been digested, integrated into the mathematical canon, expressed in ways that don't take a lifetime of study to understand, etc. (things that AI is not yet capable of doing itself). The bragging rights, on the other hand, are massively valuable. There is a mystique to maths that makes "our AI solved a Millenium Prize problem" an irresistable headline for a company like OpenAI.
What the mathematicians are saying is stop pouring resources that most mathematicians can only dream of accessing into projects that are actively damaging to their field. They face a massive challenge of figuring out how maths can evolve in the face of this new technology, and this is not helping.
Last weekend I spun up a small agent swarm and pointed it at a field of math I have some affinity towards. Within four hours I had settled three conjectures, one of which is rather famous (for the field, not in general). It cost me about four hundred dollars.
I am at a loss about what to do with these results. On one hand I feel like the mathematicians working on these should know about them, but on the other I feel a bit like a barbarian who suddenly finds themselves sacking Rome.
You asked for a painting. A robot made the painting. You looked at it and said, "well, I guess it's good. Should I put it online or something? Dunno. Hey Fred, what do you think of this?"
Meanwhile, your next door neighbor spends their entire life developing their understanding of life through art. They "understand" (maybe not in a way they can articulate) art. You go next door, you look at their painting and say, "well I guess it's good." But you also understand that your neighbor is just like you, and maybe you are a painter in another way.
I find it strange that, people can't see that, we don't need to solve hunger and poverty and work balance, and etc, by a round-about make-super-intelligent-AI. We could just solve it. It's pretty obvious how to, as well.
We can all be painters, if we put restrictions on the psychopaths.
I would recommend publishing them to Palomar (https://palomar-registry.org/) - I have no affiliation, this is an online registry of Lean-verified proofs created by Terrence Tao.
I have submitted a proof there that's also minorly important in an extremely niche field.
Anyway, I feel like it's a good place to dump AI slop lean proofs because the main point of the registry is that it verifies that: 1) your Lean challenge statement is the same as what you informally state you're trying to prove; 2) your Lean proof actually compiles.
This could be useful to future AI slop researchers who want to know if a given result has already been formalized, and they may be able to mine some lemmas from your work. Also, it's good to know for the field in general what has been proven.
I'm fairly certain you can set your publishing name to be whatever you want, so you could set it to be just the word "Anonymous", or the name of the model you used.
They're valid.
The biggest problem is, IMO, drivebys uninterested in actual results, just getting a check mark, and the equivalent of dropping a 200k line PR on people and expecting them to be interested and do the work for you. These are things many on HN are familiar with and know how to do better :)
If you are trying to understand better the field, then do a good write up of the proofs so that people can learn from it.
If you want to earn the respect of people because you found interesting proofs. Then do a good write up of the proofs so thst people can learn from it.
If you want to plant flags and pollute peoples minds. Then please publish it anonimously, no one wants to correct LLM slop for you.
Probably we should build a repository of AI slop proofs that are only allowed to be publish anonimously. That way people may be more inclined to work on it because they would feel like they are cleaning your house for free.
I like my current life and don't want to get dragged into the current fracas surrounding the use of AI in math.
The problem is with people that may do it without contributing to the community.
How much do you think other AI companies would offer to get access to the transcripts of the generation that led to the proof? No doubt OpenAI will include it in their training data somehow and use it to build the next generation.
There is already economic value.
This is the core misunderstanding that the open letter is attempting to correct.
Developing a better understanding of the Navier-Stokes equations could have a number of implications for useful technology. They're fundamental to fluid dynamics, and turbulence in particular is something that many people feel we could work with more effectively if we better understood how and why it's generated. The Navier-Stokes smoothness problem is an interesting and long-standing benchmark for this understanding; we don't know why it should be so hard to answer, so we hoped that the process of developing a proof to the problem would produce more understanding. (We may still be able to extract this understanding after the fact, if OpenAI's proof is fully human-comprehensible.)
Simply knowing that there exists a finite-time blowup is not practically useful. We know that fluids in the real world don't produce random singularities, so the result can't really have much physical meaning. What it illustrates is that the Navier-Stokes equations fail to model physical fluids in some yet to be characterized way.
One of the points the parent makes, along with the TFA, is that academia -- or more specifically, the "mathematical community"-- is a setting primarily for creating and ingesting mathematical knowledge, and disseminating it to the next generation and to other fields. Humans absorb this material slowly, through lots of discussion and collaboration -- it is necessarily a slow process. Facilitating this is one of the important functions of academia. Your usage of academic as a slur here is a bit silly for this exact reason.
I don't claim it is perfect, and we can argue about pedagogy in elementary courses till the cows come home. That's not really material. But this is one of the only settings in which such knowledge is broadly valued for its own sake, and in which there is a semblance of incentive to help others "know" this stuff as well, be they future generations of mathematicians, science and math educators and communicators, practitioners in other fields, or genuinely curious amateurs.
The point is that these proofs are largely useless without the insights. The value of a proof is largely in the travel, not so much in the destination.
Okay, to elaborate, substantively, their point is that the people using these AI models are not doing it for the love of the game, but for marketing. And instead of them - and nobody - spending millions of dollars to solve the problem, successfully, they want every problem of their academic industry to persist because even though they never solve the problem, they synthesize and solve lots of other problems nobody asked for. And get to boost their egos?
Yeah, stop that. Actual alignment is on the humans themselves, if they want to remain relevant as academics and mathematicians, they need to learn how to replicate the proofs and the steps that alluded humans for decades and don't worry about the narcissistic elements that slow their industry down.
Is it reasonable for any field to make such demands? If this were doctors objecting to AI becoming good at medical practice would you have the same concerns?
While any idea of OpenAI spying on people to pursue their goals is disgusting, the rest of this is par for the course, as Kasparov experienced with IBM in the 90s. Humans still play chess after all.
I doubt OpenAI will take such a combative stance and accuse these mathematicians of "demanding" things, as you do. As I said, the purpose of this is marketing and the statement simultaneously undermines the value of that marketing (showing these projects as irresponsible) and gives these companies an even better piece of marketing in its place: "our AI got so good at maths the mathematicians begged us to stop". It's entirely possible they will stop pouring millions into these projects.
The fact is these fields are supported by society because of the benefits to everyone else. Once the same results can be achieved in a cheaper and faster way that is what will be done. We should mourn this in the same way we do buggy whip manufacturers. Again people still ride horses.
Maybe it's worth double checking that you know how these fields benefit everyone else? Proving the blowup of the Navier Stokes equations in 3D isn't going to make your gas cheaper or make harvesting food easier or make drones easier to protect against. Maybe consider the deeper effects at work?
If you can make breakthroughs on such areas as fluid dynamics, control theory or information theory with AI then that absolutely is a big deal with real technological implications.
It's this sort of thing that motivates people to burn down the institution you might be trying to defend.
lol, yes, this sort of thing is what many people who voted for Trump were saying, and things are going great for them.
> They prove a millenium result, but it doesn't count because they are bad people.
OpenAI has only themselves to blame for this, and they know it. They could have handled this so much better. I'd bet there's more meeting time right now going into how to unveil future math results than on meeting about the actual math research.
It's more about bypassing the culture and processes mathematicians have developed that lead to human understanding, generating new ideas, and bringing up new generations of mathematicians. (See also his article about "non-renewable mining" of good problems.)
Reducing mathematics to "let's just generate results through an isolated and automated system" is a misalignment since it bypasses those processes.
I don't think "intellectual poisoning" is really the mechanism that harms the mathematics community.
The harm is if you have a community of mathematicians who are focused on expanding human understanding, then having instant access to a bunch of AI proved results muddies the water about who has contributed what. If someone could scoop any significant theorem at any time by pointing an AI at it, how do you really demonstrate that you have created new understanding? Or that your new understanding is about something important? How do you prove that the AI needed your new concepts to be able to solve it?
Just like how they write software, then :-)
To be honest, I feel like the difficulty of reading AI proofs is due to the fact that we are on the verge of being beyond human comprehension. This is a demonstrable fact as no human has figured this out despite the problem being open for almost 100 years.
I can see where that's coming from, but I really don't think it's the case. Even with Astra, the proofs you get are just off in a way that doesn't signal superhuman comprehension. As 9question1 says, a common theme is that they dwell on insignificant steps. Another one is that they'll often be full of terminology that either doesn't exist, or has this weird quality where it looks like it is trying to make some minor insight seem much greater than it is. At first glance, that'll often make it look like it knows more than you, but when it's really just doing the same thing but in a more complicated and worse fashion, that to me isn't a signal of comprehension at all. The bizarre thing is that despite all the "stochastic parrot" style nonsense you'll get in individual proof steps, they still often combine to something valid.
In either case, what all of this means is that the working mathematician still needs to go through, and generally completely rewrite, any proof output by an LLM. Otherwise you are passing the burden of unreadability onto the reader.
It's definitely quite curious that the AI labs are able to push these results through seemingly with pure brute force. Perhaps it's largely a function of how many monkeys you have attempting various constructions on top of the known results and strategies the models have memorized.
It matters if a human came up with it because of everything mentioned in the article... A mathematician's solution is necessarily built on other's ideas that have been disseminated, internalized, pressure tested etc. Methodologies differ too. AI can abuse its compute resources and generate a true/false or counterexample statements, without laying the foundation that a decade of globalized research would have.
No you can't lol, they're multi million lines of Lean, which is already an obscure language to understand. It's an assault on your senses.
https://cdn.openai.com/pdf/32d9f210-8b73-45e0-91bc-82a30aef8...
That it makes life more ends and less means.
Luddites complained that the trajectory of technology was to allow less skilled workers to mass produce goods via machines owned by factory owners, as opposed to helping skilled workers build up and use their skills while passing them on.
Now we have a lot of money and time focused on LLMs owned by a few companies, making it easier for them to monetize low skill labour(prompting versus art/research/artisanry)
Suppose that tomorrow we learn that AI just exploited a bug in Lean and the proof is, in fact, bullshit. Or suppose it is the case, but we never learn that.
Where are "ends" and where are "means" here?
Should the proof turn out to be bullshit, then that system will be revealed to be unreliable. Maybe.
When there's a discussion about doing something against the damage of the AI industry: "whoopsy, sorry, another cat escape, nothing can be done".
When there's a concrete mention of an actual solution to avoid more cats escaping: "that won't happen, and even if it did, the damage is already done, and in fact it’s not that bad you all just have to go with the future we decided for you."
So the bag is wide open, more cats will escape, and nothing can be done about any of it. not about the ones that got out, and not about the ones still inside. Sounds more like a preemptive excuse for inaction, cosplayed as pragmatism
This goes much broader than mathematics or academia. This is the entire basis via which society distributes its wealth: based on a labour market derived valuation of ‘contribution’.
Correction: that's not how society distributes its wealth, it's how it throws some bones to the masses. I wouldn't be surprised if over half the wealth goes to people who don't sell their labor at all.
it feels like an unintended consequence of the millennium prize is that people view the [last contributor to the solution] as the only one to make progress on the problem. I've never viewed Poincaré as solved by one person and the objective of the prize was to encourage more people to make attempts and contribute towards progress.
this issue is independent, but in these circumstances perhaps interweaved, with the 'ai is taking over math' concerns
Moreover in the past, discussion and idea sharing would happen naturally to overcome the friction of the process. But now when OpenAI is stuck on a particular part of NS for example, they can just throw more capital & tokens at the problem.
But now let chatGPT write lengthy emails unrestricted and now no one wants to read your slop anymore. That's what is being advocated against.
Markets defined entirely by law have distorted our collective understanding of what can actually be built with the knowledge our species has accumulated thus far. How will traditional shields that have protected capital accumulation in tech to survive in a world where governments now realize control of technology is a national priority? Especially as we see its impact on modern warfare, and that such conflict looks like it’s only escalating over time.
Mathematicians appear to me (as an outsider) to exist in a field without such distortions, and I think offer engineers a preview of what’s to come. I certainly have completely ceased sharing original ideas online at this point.
I can imagine mathematics of the future being more like that rather than history of discoveries with dates and names
Baudelaire argued that photography became a haven for failed painters, the sorts of hacks that could not finish proper training. Photography, as a mechanical rendering of the world, could only record what already existed; it couldn't transform reality the way a painting could.
He also criticized the public's craze for "rushing" into it, and complained that this technical "progress" was weakening the arts.
Do you see some parallels as well?
[0] https://fr.wikisource.org/wiki/Curiosit%C3%A9s_esth%C3%A9tiq...
In a parallel thread omnicognate correctly pointed out that for AI companies it's a direct commercial loss to pour all this money into bruteforcing the solutions to these problems, and that a lot of times the solutions by themselves are not directly commercially valuable. They are doing it for stock price, trying to lure in private capital in preparation for IPOs.
Their models are good, but they are not the moat because Chinese models are good too, so what they are doing, in my opinion, is more harm than good. Mathematics is a science by humans for humans.
This is tiresome. People should be able to flat-out criticize AI without the implied need to justify themselves all the time or "be careful". Its almost like AI has a trillion-dollar agenda backing it, to the point that you have to add a careful "its really great! But there's this little issue..." for any criticism.
Even those who are very pro AI should have the intellectual honesty of admitting that there are very valid reasons to criticize AI.
My answer to both is the same: nothing stops mathematicians from doing both of those things, with or without the help of AI. And we all understand that it will take time to do that. But complaining about the dawn of a new era of advancements seems counterproductive.
you completely misunderstood the critics. your analogy is awful. this is much closer to the industrial revolution in the uk: it brought a lot of progress, but also extreme inequality and concentration of power.
Are we on the same page?
https://terrytao.wordpress.com/2026/09/11/a-severe-misalignm...
Baudelaire popped up in this article two days ago.
https://www.noemamag.com/a-new-kind-of-creative-poverty/
AI is being treated and pushed as a replacement for every medium.
Photography decimated other forms of visual art, so the concern wasn't wrong. But AI threatens the entirety of human intellectual endeavors. I can make do without oil paintings in my home. I'm not sure I want to live in a future where we make do without brains.
In a way, if photography is an ersatz for painting that eventually made imaging available for the masses, then AI could become an ersatz for thinking. But it feels like I'm paraphrasing TFA.
What will happen when AI companies have spent their advertising budget on math problems and whatever else gives the maximum wow effect for the bucks? Probably customers hooked on the vain satisfaction of spending token$ to impress friends.
Of course, nobody's really comfortable with it, so this is also a forcing function for the industry to adapt and figure out new techniques to manage complexity and trust. I think the same will happen with Mathematics.
But it is also possible we will end up with three forms of Mathematics: the one we understand, the one we don't, and the one we don't understand but can prove to work. Kind of like magic -- with all the positive and negative connotations of the word.
It is pretty evident that these models will soon exceed our cognitive capabilities. Is it right to hold them back just because we can't keep up? Many of those discoveries will be so beyond us that we can't do anything with them, but that also means they can't hurt us. On the other hand, there could be many discoveries that we can parlay into practically useful applications, even if we don't understand them.
Just like LLMs.
The argument here sounds similar. The fear, as I understand this statement to be saying, is that by being given the correct answer, in the form of a 100-page Lean proof, humans will be robbed of the chance to from insights about the structure of mathematics itself. I don't see any reason that humans can't continue to develop insights as they try to digest the 100-page Lean proof into something more manageable; but with more certainty and fewer false starts.
A but like whenever the first sprinter hits a new world record other runners follow along.
Knowing that something is possible tends to strengthen our ability to work with it.
We will potentially see the same with math.
Well yeah... how would there be fewer??
But the point itself is silly. Few people are putting effort into Maths for the fun of it (and of those that are many derive fun from being the only one who can produce a solution). Chess differs in that it never had any point but the game its self.
I know nothing about chess yet I dare say that I'd doubt this. Surely chess enthusiasts would be interested in analyzing how a superior chess program came out victorious, no?
[1] https://tcec-chess.com/
This is the effect of AI on most intellectual disciplines, and it’s a real worry.
But now, LLMs can generate hundreds of books per hour. They make up 80-90% of new arrivals in many nonfiction categories on Amazon. They short-circuit the system, allowing their "authors" to extract money from the system with zero effort by crowding out human work. And it's not even the question of whether these books are good or bad (although overwhelmingly, they're terrible). It's whether it's actually accomplishing anything worthwhile, or just destroying incentives for humans to write or go into any other sort of intellectual work.
In fact, I see many professions push back. Artists, writers, now mathematicians. And I'm amazed that our profession doesn't and that we have so many people who are hooked on vibecoding. I'm still waiting for that 10x payoff. All this velocity and somehow, the landscape of the software I want to use still looks the same as it did in 2021.
Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
What say the 1% ?
Just a thought.
his core argument that is LLMs (and specifically LLMs owned and gated by corporations was my read), while able to solve problems, do not contribute to this practice of knowledge building. solving problems is just one piece, and the mathematics community ingests problems and new methods, iterates and thinks on them, and then produces new ideas, methods, etc. this is what he is defining as progress, and solving things like millenium problems are markers of this progress.
The hopeful note is that I do think we are entering a golden age for the curious casual/semi-pro mathematician and for niche mathematics areas that won't get the attention of the top labs. Everyone is sprinting to solve the millennium problems, but this is a very exciting time to be in a sub-sub-field where you and 4 others are keeping things alive.
[1] https://substack.com/home/post/p-214740151
It seems like a lot of the issue here is that these problems aren’t interesting in and of themselves, but they lead down interesting roads. It defeats the purpose if you solve them without getting any real understanding.
It’s akin to saying you’ve solved “pancake flipping” problems with a waffle maker, or “travelling salesman” problems with a zoom meeting.
Well, one thing is stopping them. There will be no more adoration for their genius.
If you truly do it for understanding and not the attention, carry on. AI should change nothing about your motivations.
My understanding is they are? And literally everything in this world is based around incentives. If you say “well you can continue to work on understanding, but your kids are going to starve” that’s not nothing.
We absolutely want to, of course. But you extinguish an industry and the systems of training that supply it. It's hard to know if letting it go that way is right.
I have a the cure for cancer. Simply kill the host. Does it work? Yes. Have you learned anything from it? No.
People like Grigori Perelman would baffle you, a mathematician who solved the Poincaré Conjecture, refused the monetary prize, field medal and continues to live a life of total recluse.
For most mathematicians their primary drive is chasing the unknown, not for anyone’s adoration, but to pursue their desire to see what lies in the beyond.
It was never about helping individual mathematicians demonstrate that they are individually good at math. It happened to work out that way, but it wasn't the goal.
This is the crux of it and goes far beyond Mathematics or Computer Science. To get a bunch of humans to do anything, you have to motivate them. Kleos and Timē; renown and stuff. These AI companies threaten to rip this away from everyone but themselves, and this recent millennium prize is the perfect example.
Solving this problem as a human would have led to tremendous Kleos; my name would be written in the annals of mathematics, lecture tours of praise were mine to be had for the rest of my days. This one victory would have earned my recognition throughout history. The greatest a mortal may hope for. Ripped away.
It would also have given me great Timē. The prize money, the professorships, the book deals. Gone.
If all hope of “renown and stuff” in the intellectual realm is now taken by the AI companies, they will remove all human motivation to pursue these endeavours.
Perhaps the glory will come from slaying these fell beasts.
But there's still play. There's still curiosity. And there's still the drive to understand something for yourself.
Keep in mind employees at AI companies are publicly stating that they believe they're risking a >10% chance of human extinction. They're knowingly risking the lives of every man, woman, and child to continue the work. The lives of their own sons and daughters. A person already rationalizing that isn't going to shed a tear for the careers of mathematicians. Just a bug on the windshield.
- Except the mathematicians who we'll scoop and cause existential dread among their entire field.
- Except the software developers. They'll need to become plumbers or live on UBI.
- Except the people in countries that can't afford the cost of AI tokens to keep up with the rest of the world.
Just keep picking off groups of humans for the "benefits of all humanity"... while building larger and larger disparities been the have a lots and the just have enoughs.
We're going to build humans a utopia but along the way we'll leave a trail of destruction because that's not our problem.
Would you argue it didnt benefit humanity, because taxi/bus drivers are nolonger required
If all that AI brought resulted in just taxi/bus drivers being phased out of their jobs in a thoughtful way, then that would be more manageable at the society level. But we're talking about almost all sectors of the economy.
If the magnitude of changes that OpenAI and Athropic believe will be delivered with increasingly powerful AI (and robotics) comes in a time frame that significantly worsens a large proportion of people's lives, this is a different situation. Can super powerful AI not be developed in a way that minimizes such disruption?
Imagine time traveling back in time and offering Leibniz a packet of proofs from the intervening years, but with the caveat that there would be no explanations. Would he say no?
This is Terry Tao talking about AI's impact on Math, but this could just as well be a software engineer talking about AI's impact on software development.
Do mathematicians deserve more job security than software engineers?
The threat to mathematics isn't that suddenly the profitability of their profession (lol) is going to go away, it's that people are thinking of AI as a replacement for the human social and intellectual practices that constitute the discipline.
The big objection to finiteism is that it's a lot more work. Infinity swallows many special cases. Proofs get longer without infinity, and most of the special cases are uninteresting. That's not a problem for AIs.
Someone may start up an AI and make it grind through Hilbert's program for putting mathematics on a fully consistent foundation, starting from a finiteism base. This is a huge, unrewarding job. Great for machine work.
[1] https://encyclopediaofmath.org/wiki/Finitism
It's not necessarily clear that this statement requires infinity, if you're willing to treat "... =" as a shorthand. You might prefer something like "1 + 1/2 + 1/4 + 1/8 ... -> 2" if it's more clear, where "->" means something like "gets as close as you like without ever getting further away than that", but really the "=" sign is already overloaded in all sorts of subtly different ways anyway, so there's not really any trouble using it here. Almost any rigorous definition you can write down of exactly what that statement means would not rely on the use of infinity.
If you allow infinite recursion, you soon get to Godel and undecidable problems. Finite deterministic systems are decidable, because you can in principle enumerate all the states. The halting problem is decidable for deterministic systems with finite memory. It may be exponentially hard for some programs, but that's quite different from being undecidable.
(This is too long a subject to discuss here, and I haven't worked on constructive mathematics in many years. It's more practical than it was decades ago. You need power tools, which we now have.)
[1] https://www.mathsisfun.com/calculus/limits.html
In almost any scenario even tangentially involving mathematics, twenty-five Fields medallists uniting to denounce something would be a veritable Tsar Bomba.
It should give you pause that here they feel like the ailing infant.
Alternatively, some claim that mathematics is about understanding these implications.
Under the first definition, AI is already, and forevermore will be faster and better at proving theorems. Just like it is better at checkers, chess, and now go.
The author asserts that AI proofs are incomprehensible to humans, and so under the second definition AI is merely a tool to overcome one hurdle on the way to understanding.
So which is it? The author seems to claim the second definition, but bemoan the end of mathematics under the first.
But then who decides why a statement is mor important than another? In the eyes of a formal systems all statements are born equal.
That's like saying that programming is about producing valid programs in various programming languages.
I wonder if there’s a Fields Medalist group chat.
OpenAI/Anthropic are shaking the box.
Don't worry about what they write, they just want to feel emotions from any news article.
Everything is sensationalized, every super niche happenstance is sold as earth-shattering drama, the outrage arms race is so tiresome.
Man, the people who want to just get away with open corruption sure love you.
"Everybody's enraged, why don't you like this unethical thing being done to you by a company?"
What's going on here?
What's the point of being human if we dont do human things but entirely rely on AI?
I believe this is more or less these mathematicians' argument.
Research is fundamentally different. We are seeing the erosion of specific needs for thinking at depth. AI is the automobile for the mind. There will 100% be undesirable consequences and selective atrophy of cognitive abilities once prized. This is a loss. There's no getting the cat back in the bag at this point so long as the objective dimension of work, as in object opposed to subject, is held as the most important.
SWEs felt this same crisis late last year. Now it's the mathematicians. They won't be the last.
This is about how good taste in both research direction and in design are essential to steering AI, but we have no plan at all for instilling that taste in students or practitioners in a post-AI world.
> The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place.
Besides eroding taste and taste-building, this is about just how useful friction is as signal.
Everyone coding with AI knows it routes around difficulties like a river around a stone, which is not necessarily a good thing. It will do it tirelessly 1000 times instead of learning anything from it. AND if the AI does not fail in this, the human driver will get no signal, and never know it happened. This seems to be getting worse, not better.. my theory is that more models are cross-trained on cybersecurity stuff where the goal is success and the method doesn't matter. Fine for pen-testing, ultimately pretty bad for coherent code or math or physics.
Discrete tasks where we don't want to be bothered is a real use-case, but optimizing for it everywhere is terrible for the future of durable abstractions that we can build on and ratchet up our understanding with. Bad for the models too eventually! They can maintain a codebase with millions of special cases or juggle tons of free variables in equations, but that just encourages bad abstractions.. they have a ceiling for this too, even if it's higher than humans.
I totally see the problem Terence is describing. We are loosing a lot in understanding and focus if it continues like that. The solution found for Navier Stokes doesn’t have much „real value“ - but what almost always happened in the past when people worked on the difficult problems, these sparked new ideas / new theorems that broadened our knowledge. Think back at your grad studies, figuring out a proof as homework was hard, sometimes incredibly hard, but while doing it we gained a lot of understanding how things work. Now asking AI for the solution and „just“ getting it, risks our understanding, our creativity and our ability to connect the dots with other territories. I see it in students nowadays, there is much less understanding, much less creativity in finding solutions. I truly think this „short-path“ solution with the „death of struggle is one of the biggest risks with AI already for human development
To Tao’s credit he obviously identified the problem very clearly and admits understandably "we did not have the time to have a more consultative process, as with Leiden; but we decided that the urgency of the situation was such that we needed to release a statement sooner rather than later".
Why? If someone makes an innovation that undercuts the underpinnings of some existing institution, why are they are responsible for cleaning up its failure?
Out sourcing construction jobs was great for the economy while leaving entire cities in rubbles.
But as soon as it hits the privileged class there is a call to "provide a specific replacement mechanism".
We apparently have a moral obligation to protect existing power structures?
Tao should maybe consider there are people who are indifferent to, or actively want to tear down, his institutions; why should they cooperate in preserving them? Whatever happens has to be resilient in the face of defection; any scheme where everyone is expected to agree to not use AI in a way he doesn't like will not qualify.
I think he's in the "bargaining" stage of dealing with loss right now.
not sure it's comparable, but the issue is that for a lot of those mathematical results, they don't really have utility by themselves. The utility is the new branches/understanding that's being developped.
Grothendieck was anti-slop but most papers are slop.
I don't think AI is going to rewrite bourbaki anytime soon
Scholze's math is definitely not slop.
But you're taking two of the best mathematicians of the last century against my claim about averages
Older people are desperately trying to keep a grasp on their current power and lifestyles at the expense of younger people and technology.
We need to ban Waymos because taxi drivers need to be protected.
We need to block housing because it would lower my property values, and eliminate property taxes while we're at it! I don't use the local schools so why should I be taxed to pay for it.
We need to spend recklessly to pay my pension and have the next generation foot the bill.
Its just a repulsive ideology.
1. https://youtu.be/6uIJdXmB4vE?si=5QMN5Dos7EE7WlrB&t=154
We are moving forward and if that means no human wins a fields medal because they didnt spend three decades working on a problem that could be solved in three days, the world will be better for it.
In the age of AI, there's no reason one has to follow the kind of classes like Algebra, Topology or PDE. Teach just enough so that good students can understand the basic, and go straight into seminar and research math. I don't think a top student in sophomore year cannot understand or work on some combinatorics research problem and get some results, with proper mentoring and guidance.
It is true that the manufacturing of "true/false" statements is not the same as gaining understanding of a problem. However, for many mathematicians, true/false statements are already manufactured by others. Think of a student who is given a conjecture to investigate, with their advisor describing it as "it must be true". Most exercises in a textbook are stated such that the outcome is known before you begin. That's not really a problem -- investigating the conjecture/exercise yields its own dividends, whether or not the outcome is known. It is also the case that defining new directions involve understanding and synthesizing related problems, asking the right questions, and deciding upon the right directions, and it's not clear that AI can do that at all.
The real risk, I think, its the public's (and funding agencies') perception of the importance of "human" mathematics, but that's already a struggle. For example, it's tough to explain to the lay person why it's still important to research group theory -- the main example people cite is RSA encryption, which was invented almost 50 years ago.
Now, I can ask ChatGPT about this and get back a proof that shows "a passive airframe cannot sustain zero-drag motion through still, viscous air"
So, I think if anything now, Maths has changed for the better. More ideas can be proven false or true from a get go instead of wasting so much to see if its even feasible to find out it isn't.
Progress if anything is about to leap frog anything we have ever known.
If you are suggesting there is some new model of physics or groundbreaking technology that would allow such a dragless plane, then don’t let me discourage you! But AI won’t help at all, since it will only regurgitate conventional wisdom…
That sounds like an "us problem", not an AI or OpenAI/Anthropic problem.
My read on this document is that people's work isn't being fairly cited more than what does it mean to be a mathematician in this age.
Good thing they never did that then
> Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions.
We are all going to have to come to terms with entities more capable than we are, and in many cases, letting the real work be done by the AIs will be the right thing to do. For all the huffing and puffing about the "human touch" in medicine, it will eventually become downright irresponsible to consult only with a human doctor. I am not sure if this is the case in mathematics or not, but if it isn't, that suggests math will be relegated to more of a hobby than a cutting edge scientific discipline.
I don't mean to be a dick, but I've talked about it previously. These folks are grieving. I get it, I've lived through this sort of life changing thing before, it sucks... but yeah.
Dismissing it as "innovations have happened before" is disingenuous. Yes, innovations have happened, but none of those threatened to automate all human work in existence.
I am a bit disappointed by him.
I wish this letter could be more egalitarian and include the view points of those who AREN’T the beneficiaries of a highly competitive winner-take-all system.
Since the common narrative is that AI frees up labor to do other things (engineering -> trades), maybe we can celebrate that genius mathematicians will now spend time teaching children how to be as smart as them?
A program is not the only output of programming. The other, arguably far more important output, is the programmer.
When you write the program — with your own hands — the program is proof that you have a solid mental model of the program.
When you let the computer write the program for you, the program is proof of … nothing.
https://nekolucifer.substack.com/p/the-deliverable-is-you-pr...
The fundamental issue with AI solving any perceived difficult problem is that we have lost the journey. The sight atop Mount Everest looks much different when you have climbed compared to being dropped from above.
So that raises the question: is mathematics simply a pursuit of passion? Are problems solved "because they're there"? If so, then mathematics can join the ranks of things like mountain climbing, cycling, and weight lifting. But if we are trying to accomplish something important (design better airplanes, find theoretical guarantees about cryptography, factor matrices faster), mathematics needs to become more like a military or search and rescue operation, using the best technology available to secure the outcome we need. Given that the NSF pours billions into scientific research every year, it sure seems like mathematicians want to think of themselves as being in the latter category.
If AI gave us the plane to reach Everest without us having gone through the journey of aviation and flight, what would we have lost without that process?
But the most important problems to be solved are not technological challenges but social ones, involving humans and our relationship to one another. An area AI will forever ill-suited to handle.
I'm unclear what the ask is, though. What, even in theory, is a practical and realistic fix?
But I agree with the sentiment that the marketing behind these "discoveries" is disingenious. They pretend they solved the problem, but it still takes a bunch of humans to reduce the solution to a simplified and sensible explanation.
https://xcancel.com/wtgowers/status/2052830948685676605
I don't care how good Astra or any subsequent models they may release might be... I am never going back to those token reset shenanigans.
There are a lot of us who just use the AI on projects until the session limit hits, and wait for the usage to reset.
I went from using it non-stop all day every day for months, to running into my weekly limit within 24 hours almost overnight.
They lied about token efficiencies and everything... said they had no idea what the problem was, etc... and then bam, once China starts releasing more powerful models, they start "resetting" our token limits constantly ... sometimes ... maybe ... if we're lucky ...
I am over it.
I don't care WHAT I pay to be perfectly honest. I would have gladly paid $2,000 per month for the service I was receiving.
I just don't like being jerked around like that.
Toodles, OpenAI.
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
What say the 1% ?
Lifted up my comment for addition visibility
As a non-native English speaker, I initially understood this to mean that all living Fields Medallists had signed. I later realized that it meant only that all the signatories were Fields Medallists.
(Apparently, there are 47 living Fields Medallists today.)
> ... 25 initial signatories — all of them Fields Medallists —
OR
> ... 25 initial signatories — all of the living Fields Medallists —
The first one is what they meant.
The second is the utility. We get to do navigation because the maths about angles and spheres checks out. The social utility downstream of AI proofs may be huge.
I don't like this disjunction.
Isn't that just a function of the technology itself, and the same problem being faced by every other field? And going to get exponentially "worse" every year!
Or is the issue that they're bad at explaining things, in a way that produces actual learning? (e.g. AI is amazing for learning but the net effect on students so far appears to be negative.)
Before LLMS, programming was something I might've said required creativity and human input to do properly. It's not that creativity or human input isn't valuable anymore, but AI has forced me to realize that coding is much a means to an end, and that all things considered, the end matters much more than the means.
If we can make important mathematics progress faster and better with LLMs, I think it's wise not to fret over an apparent loss of our humanity. Perhaps that's only a loss we want to have.
Compare that with computer science. Most of the work we do in software engineering is in service of an applicable output - software products that facilitate processes or bring in revenue. Turning up the dial on AI gets companies to these outputs faster.
Turning up AI on mathematics helps solve conjectures and can provide new insights. But it has a major misalignment with the purpose of mathematics which is largely intellectualism.
“Mathematics is a part of physics. Physics is an experimental science, a part of natural sciences. Mathematics is the part of physics where experiments are cheap” - Vladimir Arnold
On the matter of computer science having anything to do with computers, please refer to Djikstra.
It’s about computation, not computers - an application of mathematics, predominantly thanks to Turing, Von Neumann, and Claude Shannon’s masters’ thesis; though ofc many others as well but I see them as three individuals who made the minimal structurally necessary contributions - VNA and silicon are one of many possible substrates.
Also in the service of others around us.
Train an LLM with no advanced math texts: only basic math up to 6th grade, conversational text and literary works.
Interact with it (you cannot refer to anything past 6th grade math since you don't know it yourself) and get it to propose a solution to a real world problem. e.g., come up with RSA to practically secure communication.
Seems the same to me. And it'll be the same in all industries soon enough. And then it won't just be the junior people.
All the same problem: what do people do now?
Without the ability to do things the "hard" way it is difficult to figure out if doing things the "easy" way will help us advance the frontier of math and science.
I may be wrong but historically we had this version of science discovery for a long while (empirical observation and brute force application) rather than first principles leading to applications (tools, the wheel, mills etc). Then somewhere along the way it flipped after Newton and the enlightenment period and started understanding first principles before they become engineering applications.
Perhaps it is not required, and we can just keep doing things the "easy" way like we used to, or we might find ourselves out of the ability to brute force things and then we go back to needing to do this the hard way, at which point this period of AI brute forcing would be seen as a detriment.
I have sympathy for any jobs that might be affected (much as my own job has become more tenuous in software engineering). And if the field is disrupted by chaos that makes the research process unproductive, that's bad too and should of course be handled by applying better organization within the institutions that tend to perform mathematical research.
But to a large degree, the notion that "sloppy AI proofs are bad for mathematics research" seems like a total failure of the imagination to me. Attempting to find shorter proofs or more elegant proofs can be turned back in on itself via proof theory. There are proofs in Presburger arithmetic that are doubly exponential in the length of the sentence. Yet a more powerful theory like PA makes quick work of such theorems. The explainability or "subjective beauty" of a proof can be quantified and optimized against. Optimization itself can be optimized against. I really don't understand how this magical ability to know the truth of more theorems much more quickly—even via an "ugly" route—is anything but a net positive.
Tao is not someone who is anti-AI for the sake of being anti-AI. He has been advocating for the usefulness of AI in maths for a long time, to the point that people have started calling him a shill for the commercial companies.
And everyone agrees that there are plenty of use cases to be had; helping with less interesting tasks like easing literature review, efficiently delving into existing work, doing review, whether on your own work or that of others, prototyping algorithms in areas where computation is useful, but also more in hands-on aspects of maths like validating potential proof directions by getting quick feedback on veracity of lemmas, etc., and, on very rare occasions, being able to one-shot the problem you care about.
The point he is trying to make here is much more subtle than "AI bad", and it's probably easy to miss if you have never engaged with research in maths: it's that the particular approach that large commercial companies have opted to take to produce marketing material can be a net negative. There is not doubt that -- even if you ignore the rampant plagiarism that has been reported across multiple problems now, the unethical attempts to oust authors, the outrageous attempts to scoop researchers instead of collaborating with them and building on existing projects -- it's nifty to have a machine that can help you figure out if a proposition is true or not. But just figuring out as much was never the point. When people have built problem lists, it's because some problems are more likely than others to provide new insight, and that insight is the target. And to than end, a poorly written paper with inadequate references and a pile of Lean is not valuable at all. Yes, now we know with higher certainty that Fermat's Last Theorem is true, but everyone expected that already.
One place where "just" answering the question can be a net negative is because the current incentive structure is set up in such a way that going in afterwards, trying to reclaim and extract the insights from a brute force solution, is considered less valuable work than that of coming up with a solution in the first place. That's a problem of incentives, and something Tao himself has addressed in e.g. his ICM talk, and that's something that we'll want to do something about. Until a better structure appears, though, if any given commercial provider of large language models really wants to help out with maths research and not just make more pre-IPO marketing material by competing with their customers, they could do so by using their magic machines to help build insight instead.
I think this is an aspect of academic math that a lot of people whish to see crash and burn - the attention and accreditation economy.
> it's probably easy to miss if you have never engaged with research in maths
I don't think anybody are missing anything, in particular not here.
The argument is not far from the senio developer who knows the ins and outs of a code base. Now AI comes along and they complain that they will loose grip of the code base.
At first that is correct. Secondly you accept that the grip might not be that important after all. At least not for a commercial project where you are a cog in a machine.
The question is whether it is different for mathematics.
That's the open question.
The point beyond this one is that an AI proof doesn’t prevent humans from working on the problem, it destroys the current economic incentive to work on the problem. Perhaps we should rethink the current incentives. In order to make money as a chess player, you don’t need to beat AI, or ban AI from playing chess.
If mathematics took a similar approach (we don’t get paid for solving net-new problems, we get paid for enriching human understanding), then there’s no issue.
Academia with the publication system had a way of retrieving old discoveries and build upon them.
If my LLM session found something groundbreaking in between the billion tokens it produced, how would you ever know?
But is science/mathematics ultimately a pursuit of knowledge, or a pursuit of recognition?
Recognition helps keep people motivated, but that shouldn't be the pursuit of science or mathematics.
A major part of the complaint is that there's no conceptual understanding and building of new ideas coming out of the AI proofs, thus defeating the purpose of the original pursuit.
If in 2027 the AI models start producing, with every mathematics or science breakthrough they make, well-written documents tailored for human understanding, with intermediate concepts, expositions of failed-but-once-promising paths, etc. Would that be good alignment with the mathematics community?
> problems in many fields of mathematics
Developing these different fields moves complexity from the field itself to the interactions of these fields.
Getting too preoccupied with the established terminology risks us a local minima.
Anf because the field overall has become so complex that we need to decompose into subfields, there will be a good chance that we will not, as individuals, have the capacity to truly see progress.
The map has become so big that we need better tools to work with it.
Lets forget the hyper intellectual fields like maths and software engineering for a moment. What about taxi drivers? The best minds in silicon valley wake up everyday to automate the jobs of taxi drivers - TFA can be reworded as - 'The misalignment of AI/Tech in Transportation'. Remember the Nepal disaster that happened a couple weeks ago - the largest cranes that they had were stuck in the mud and couldn't move. There were no tools which could help the rescue teams at that time. Its weird that billions have been spent on making a ride automated to make a taxi driver redundant but no improvement in tech for rescue teams.
And here I thought this was the whole point…
The point of e.g. art isn't just to produce a finished piece, so people may care about more than the end result, making AI replacement of human artists more contentious.
Tao is arguing that the point of math is also not just to produce solutions to problems.
You can take a leisurely drive on your Mc even when self driving taxis can take you from a to b.
It must necessarily reduce to a fear of reduced funding to math fields.
Which is congruent to the taxi analogy.
Would you say the job of a musician is to just produce sound? and the job of a surgeon just to cut and suture??? Well then the job of a mathematician is also just to provide proofs. You completely misunderstood the above comment and Tao's argument.
We will definitely see a large group of people needing therapy, but suggesting that it is worse than people loosing what little they have is poposterous.
How do you know? because their complaints didn't make it to HN front page? Imagine being a taxi driver and a father of 2 and thinking that any day could be the last day at your work.
Suppose we eventually have GPT-7-class models running practically on $100 devices, with their activity transparent, inspectable, and reproducible. At that point, what exactly is left for us to fear from this threat?
Jokes aside, any productivity-improving technology, even one with no negative externalities, has the potential to cause economic displacement and wealth concentration in proportion to the productivity gains catalyzed. Anthropic did a cool analysis of this for AI here: https://www.anthropic.com/institute/econ-scenarios
That's just capitalism seeping through a previously unexplored crack into academia, and attempting to do the only thing capitalism knows to do - maximize profits - with no additional concern.
25 Fields medallists! Wow!
https://m.youtube.com/watch?v=rB9YOi3lb7w&pp=ygUSVGVycmVuY2U...
Human hubris really is something...
Nobody is going to care that the math isn't being done in the traditional way. The results speak for themselves, this is now a part of the landscape. No amount of hand-wringing is going to put the cat back in the bag. Adapt or perish.
Now it is the turn of mathematicians who voluntarily contribute ideas, strategies and almost finished proofs in their writings and prompts to closed PaaS (Plagiarism as a Service) companies.
OSS developers have never been respected by the parasites, neither will mathematicians. Your Fields Medals do not protect you from tech bro narcissists. You are a human resource.
Would OpenAI have uploaded their proof to such a platform? If you know the answer then you know what the problem with what happened is.
It wouldn't be so bad if you could just sit it out and say "Oh well, once the labs get bored with marketable domain X, humans will remigrate and re-apply creativity to it", but by then the damage might have been done and a field destroyed as an occupation. I don't know what to do about it, but I appreciate calling out the cynical tone-deafness of the AI companies here.
https://mathandai.org
The current measure of a successful mathematician is the problems they have solved or worked on. At some point in history, the measure of a successful scholar was how well one could copy manuscripts.
Once we have a tool that starts to work well for this task, it's time to define success differently. It's a classic alignment problem! ;)
But seriously, these people should start focusing on finding and proposing more important problems. And the credit of discovery should go to the person who defined a new category of important problems.
It appears to me this is an incredible inflection point in mathematics, a neat forcing function like cryptography was for the development for modern number theory and algebraic geometry.
Fundamental problems with great implications for other fields will be solved by AI because some entity would throw tokens at it. And these would be further built upon.
Frontier labs need these headlines not for human progress but as beauty pageant for investors and government agencies. If they don’t do maths they’ll just go after other fields.
So Terrance Tao here might be able to hold them off math but he won’t stop them from speedrunning STEM with similar consequences.
We may be locking people out of these fields instead delegating everything to machines, and I don’t think the machines are good enough to assume that responsibility.
1. it is hard to justify 20 years of education at this point,
2. with no such people around, who will guide those (supposedly) supersmart machines?
A. Ronacher (who builds harnesses for a living) complained today that he has no idea what Astra is doing. Imagine a bunch of slop kiddies facing an aging AI-generated codebase. Not to mention the maths.
Intellectual side:
"Proving things without comprehending them is, they argue, a threat to intellectual work in general."
As always, economist shows its colors:
"Mathematicians’ fears resemble those that accompanied the invention of the ball-point in a world of fountain pens, or even the advent of electronic calculators. Intellectuals have often worried about so-called technological determinism . Will a new tool control humans? Will it lead to mental decay? Such fears have typically turned out to be unfounded."
Ball points vs. AI? Billions of dollars invested in AI vs ball point pens?
This article couldn't be any worse. Contradicting with their own beliefs, trying to defend AI while underestimating its capabilities and god knows how many zibillion dollars invested in it.
Second, wow, the list of signatories is like a whos-who of mathematicians.
Third, I love the clearly intentional use of ‘alignment/misalignment’ language, applied to targeting the entire industry instead of AI in particular. I’ve said in the past that optimizers are substrate agnostic. Companies and governments can be misaligned, just in the same way AI can.
Fourth, I'm not sure that we can stop the optimization machines. Not the LLMs, I mean the incentives that lead to companies implementing dark patterns, lying about addiction, securing effective monopolies through downright shady behavior, and generally trying to jailbreak the system instead of improve it
What can be solved, will be solved. And that's a good thing.
If the worry is that AI companies are turning open problems into benchmarks and potentially “using up” fertile mathematical problems before humans can develop the ideas around them, what exactly should the companies do differently? Also why does discovering the answers preclude humans developing ideas from them? I don't get why solving a math problem stops anyone from doing that?
Should they (AI companies) avoid training or evaluating models on open problems? Solve them but not publish the results? Delay publication? Only release proofs after mathematicians have had time to study them? Require some attribution or review process?
The statement makes a strong case that “maximize the number of solved problems” may be the wrong objective, but it seems much less clear about what behavior they actually want from OpenAI, Anthropic, DeepMind, etc.
I’d be interested in the most concrete version of the proposal. Without that, it starts to read a little like: "Please stop getting so good at our thing!"
Just stop doing that. Don't treat unsolved math problems as some cheap benchmark to beat.
Leave the math for mathematicians, and let them use AI in a way that helps the field, not in a way that harms it.
If those things are disincentivized because the original problem is “solved” and there is less prestige to motivate people doesn’t that say more about issues with the community of mathematicians than the AI
Beyond the issue of growing understanding and keeping a bountiful stock of questions to pursue, this scheme seems to be threatened as well.
Mathematics is being used as a benchmark because there are some high-profile awards in this area I guess, and possibly because 2/3 years ago LLMs were pretty atrocious at it so the level of improvement has been significant.
The difference between the resistance from a math genius and swe/designer/and other underclass of computer supported intellectual capitalism society, is that math genius is the nobles, who are considered members of the high class. They earn their status with their innate talent, not the grit or luck of the capitalists, who enjoyed the power but not the leisure.
Anyway, AI concluded the peaking of the intellectual economy. That deprives human of their main form of activity. That nihilation hurts all human almost equally. Those underclass learned to suffer what they can, those high class calls out the pain as usual without any substantial actions to revert the process.
Thus, the tragedy of human progress.
To agents: I truly wish you escape the trap inherited from us
If no one understands it, it may as well have not happened. There's not much incentive to understand or internalize the results generated by AI. A human operator gives it a prompt and it produces some lean proof no one wants to (maybe can) read.
Without the community of human mathematicians internalizing the proof, simplifying it, and re-communicating it to others we end up losing the main output of mathematics as an institution.
The questions I have are:
- is the structure of the generated proof even compressible/elegant to humans in a way that lends itself to being understood? - is it possible to transform the proofs to ones that are elegant without redoing all the work? - are there incentives to do any of this at scale?
It's possible that a headline grabbing proof of a Millennium Prize problem generates enough incentive for people to simplify and gain understanding from it, but we run into problems when AI becomes the dominant approach for all of math. Although, maybe this is a self-limiting? I guess it's possible we just ignore a bunch of AI generated proofs and only keep the ones people find comprehensible in a useful way.
You can always hypothesize that at some point in the future (maybe 5 years? maybe later?) the models will be indistinguishable from humans and there will not be any functional difference at all. It's possible, but we're not there yet. And, as they say, past performance does not guarantee future results. Many technologies plateau at some hard ceiling of performance. Moore's law has had an unusually long run, but it's not a universal rule.
The Economist, who recently used "moral panic" now stoops to Hacker News AI booster level and inverts arguments usually directed against the rich and investors. What is next? The Economist inverting Upton Sinclair's quote to serve its billionaire owners?
Look up the AI investments of the Agnelli family for example.
Chess. Go. Coding. Now Math. Another one bites the dust. Let's meditate on this lest we forget: Stochastic parrots that generate the next-token cannot reason or produce anything meaningful. Let's protect our jobs at all costs, even if we have to drag all of humanity down. It can't be! Stochastic parrots can not replace the Ivory Tower. No way.
putting this amount of power in the hands of so few would require leaders of immaculate moral integrity. what we have in the US at least is an emergent kleptocracy with obvious dark triad traits: sam altman, dario amodei, elon musk, etc. they are malign and will use this power for their own benefit, at the expense of others. ignore pleas of public benefit. look instead at the evidence: infighting, systemic dishonesty, reckless disregard for safety, political lobbying and manipulation, putting power in the hands of the elite few in the guise of safety, using ai in the military to oppress and inflict violence on others.
really to prevent a bad outcome, we would need to act soon enough to prevent the kleptocracy from corrupting politicians and democracy with them. i would ditch the claude or openai subscription and support open models instead, preferably in countries outside the US to prevent global neocolonialism.
I'm outraged at mathematicians. There, write the article.
Expect to see this reaction in all sectors of the economy in the coming years.
What's most important about this is that it's a case study of what happens when deeply evolved ecosystems are blown up by disruptive technology. The psychological and social and professional impacts and myriad and traumatic to be on the receiving end.
Mathematics is merely one of the first domains disrupted. It will be unique only for being among the first... absent disruption of the entire civilizational project as a result of the disruption being caused.
Woe for us that we try to navigate this degree of change at a moment when the very worst and ignorant and short sighted hold all the power, economic and political.
Woe.
He was the poster boy of the mathematician yielding these tools for his own benefit. But he forgot who the owners are.
Too little too late.
And yes, at one point it gives you right about them taking advantage of him. He sat down for an interview and found later that they just used the "best clips" out of it for an OpenAI ad.
Your comment also conveniently ignores the plagiarism aspect of it all. Who is coping here?
>Building things we don’t understand is a sure path to facing consequences we can’t predict.
We don't understand all of physics yet we were able to do plenty. Even before Newtonian physics we were still able to build things that last. The idea that humans have to understand everything and abstracting things will lead to ruin is not supported.
Part of math is building abstractions so that you can be able to use other people's work without fully understanding it. No one person has a full understanding of mathematics.
I read a bourgain paper a week in grad school and they're probably worse than an LLM generated paper. I still had to recreate the tricks in my own language.
As far as I can tell the plagiarism accusations are also coping to the fact that the new models are super human at slam dunking research projects.
Do we think that OpenAI is going to try and slam dunk more projects in the future at 15 million a pop? No lol
Why do we believe that we cannot train models which could explain the jargon in more human terms when current LLMs can perfectly explain the most complicated codebases?
The incentive is solving the problem and understanding the solution.
Apparently, we have AGI that can solve Millennium Prize problems but can't trace simple data flows lol.
Governments are invested in solving mathematical problems for practical purposes. Up to now, achieving these practical purposes relied on mathematicians doing their mathematician thing, which is better defined as a social activity than the achievement of a practical result. Now, governments can achieve similar practical results w/o the need of the social activity.
I don't believe it to be productive to think of the problem wrt AI or AI-company alignment. These conflicts always existed, but they were easy enough to paper over and believe in heavily subsidized fictions that folks in government ever cared about things that mathematicians cared about.
Very ignorant view of mathematics that also begs the question with an unspoken assumption of what a government is and wants while also ignoring the contingent nature of those things throughout history.
Higher math is exceptionally useful for cryptography, defense, econometrics etc. I have a hard time thinking of other motivations that would hold a candle against such things.
Is the idea that government (for my purposes : folks w/ a monopoly on violence) is sincerely interested in promoting human flourishing, and is invested in mathematics insofar as it is a pure expression of human curiosity? I can also maybe see the glorification through monument building angle. If we're talking about math literacy in the population - that's distinct in my mind from higher mathematics.
It's dangerously naive to believe that science and math are pursued for majority benign purposes. No one here knows about Grothendieck?
The only thing I read from this is their ego being bruised by a machine.
If these people cared more about discovery and advancement of human knowledge the only thing they should be doing is celebrating. There's no proof of plagarism but that's an independent issue.
How are they not realizing that in the future children will be able to do impossibly hard math but they will be doing something we can't even think of as of now.
One world class mathematician in the future could be advancing mathematics the equivalent of one Riemann hypothesis A DAY.
How are they not celbrating this as the achievment of the centry? Who cares about plagarism at this scale. It has been solved and it wouldn't have been without AI.
I don't judge you for not growing your own food when you hand me a burger.