#draft/aitp_thoughts
It is hard to write about the broader impact of AI, without just going back-and-forth about the usual talking points: (1) Technology can be used for good. (2) Technology can be used for evil. (3) Statistical models trained on real human gathered data, will (without a lot of intervention) assume the biases and social prejudices in that data.
Now, many likely are drawn to formal mathematics because it is pure and doesn’t seem evil at all. Moreover, to the extent that there are biases in mathematics, they seem less socially harmful. I once thought this, but now I have a more nuanced viewpoint.
First, again, going back to the usual talking points, there isn’t really much separation in the AI used to solve the next great math theorem and the AI used to track a dissident as they travel around a country. The beauty of AI is its generality. It is also its curse. Even if one is only “consuming” AI technology and applying it to specific (not evil) domains, one is still reinforcing that technology. By being a customer of AI technology (whether GPUs, ML libraries, or ideas), you are reinforcing and supporting those technologies. Again, technology can be both good and evil, but it isn’t enough to say that you are only using it for good.
Now, why can general AI technology be risky? One main idea in my mind is that it allows fewer people to do more. It used to be impossible for a small government team or an invasive company to track every move you make. It is now getting more and more doable. It is also possible for a single actor to do more with less. Fake videos, fake articles, fake identities, fake academic papers are all now real technologies usable by individuals with know-how for both political and personal gain.
Now, let’s consider mathematics specifically. The greatest human inventions are built on mathematical understanding, and like it or not, pure mathematics often leads to practical technologies. Some of the most influential technologies of WWII, including the Manhattan project, employed some of the greatest mathematical minds of the time. While these folks were not perfect, and likely had a variety of mixed motives, any project that requires many extremely talented people is much more likely to be successful if those people have bought into the technology and its outcomes. But as the number of extremely talented people needed to build the next great questionable super-technology gets smaller due to AI advances, there may be less oversight and less introspection on the small team that builds it.
There is also a silver lining to mathematics and AI. One of AI’s biggest current issues is that it greatly amplifies bias in natural language and other data sources. Part of this problem (certainly not all of it) is that these AIs are trained to say or do what seems likely in the data. There is no real grounding. There is no introspection that says, is this actually true? good? unbiased? However, to build next-generation mathematical AI, one does have to at least address truth and correctness. As someone who has graded many undergraduate proofs, I can tell you that a correct proof and mathematical gibberish look pretty similar, but when you understand what is going on, they are worlds apart. What can we do to get machines to say “is this actually a true answer?” And if we can do that, can we then get them say, “is this actually a good answer?” for many definitions of “good”?