Disadvantages of Designating Mathematicians for AI Output Releases

In some past releases an AI company has designated mathematicians to receive the output on a problem, often before anyone else. Those mathematicians then worked to uncover the ideas in the output, check them, develop them, and present them to the wider community. The appeal of this release model is easy to see: it puts expert eyes on the output early, can lead to a better initial public writeup, and gives each result a knowledgeable point of contact.

After consideration of many negative effects that this release plan has, however, we have come to the conclusion that on balance the disadvantages outweigh the advantages. This should in no way be construed as criticism of the mathematicians who have taken on such a role: they have acted in good faith and done valuable work. Nonetheless, we recommend that AI companies do not adopt this practice for future releases, for the reasons below.

Delayed or unequal access. In cases where designated mathematicians see material before everyone else, they get a head start on problems others may have worked on for years. Other mathematicians may have valuable contributions to make based on the material but may not have access to it, and may be dissuaded from working on it.

The remaining reasons apply even if all the material is released publicly at once.

Denied agency. The choice of who works to understand, check, and write up an important AI-generated result is consequential, both for those chosen and for the field, especially in the current period of transition when norms around credit for mathematical activities are in flux. If an AI company designates a particular mathematician to undertake or lead this task, it is to an extent making this decision for the community and denying it agency over its field of study. Further, this choice may discourage other mathematicians from contributing to the understanding of the results.

Biased selection. Any choice of designated mathematicians by the AI companies reflects their networks and priorities, which are different from those of the mathematical community. However carefully the choice is made, it is opaque and unaccountable. Moreover, this practice could incentivize mathematicians to align with whatever they perceive the AI companies’ priorities to be.

Borrowed reputation. In practice, designated mathematicians’ names end up vouching for the release and shielding the AI company from criticism. Given the wide range of views on the release of results by companies in the mathematical community, that puts the designated mathematicians in a difficult position that could compromise their role of helping with understanding.

It is unnecessary. Recent experience has shown that human understanding can develop organically and efficiently when AI-generated material is made available. In many cases recently, mathematicians have independently read released AI output, worked to understand it, and written useful papers providing clear arguments that solve the problems. We are also aware of several reading groups, workshops and websites being organized to understand other AI-generated output, and there are new funding opportunities for such activities.