返回资讯中心
外部精选
软件工程
#开发者实时流

Adapting Our Processes to the Changing World of Scientific Publication

15 October 2026 · by TMLR Editors-in-Chief · 4 min read

Hacker News 最新theanonymousone2026/10/16 · 2026/10/1614 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

15 October 2026 · by TMLR Editors-in-Chief · 4 min read

At TMLR, we have witnessed a 300% growth in the number of submissions in the past year. While AI has led to various benefits to the research process, not all of this growth is necessarily healthy. This post outlines a number of steps we have taken at TMLR to maintain review quality without overwhelming our reviewers and action editors (AEs) amidst this surge.

We first investigated the sources of this growth. We examined the authorship patterns in submissions to TMLR. We found that the number of solo-authored papers has risen 1300% in the past year. Quite frequently, we were receiving four to eight submissions by the same person in the span of a week or two, often of solo-authored papers. We further did an exercise to understand what is going on behind the scenes, where we invited authors of ten submissions slated for desk rejection for interviews. The results were sobering. These observations have been helpful in designing our policies.

We have found authors trying to misuse the system and make many low-quality submissions. In ML/AI conferences and journals, there is hardly any downside to submitting anything and everything. At TMLR, we have taken stronger actions: if any authors violate the journal’s policies or submit multiple very low-quality papers, then we have informed them that we will not be sending any of their papers out for the full review process for one year. We have taken this action for 159 authors in the last 12 months.

We instituted quotas on authors with respect to the number of papers they can submit. Quota rules deployed in other venues have used a fixed rule of the form “no author can submit more than k papers.” That has two problems: first, a small fixed quota bottlenecks traditional collaborations and advisor-student groups whereas a larger fixed quota offers little protection against the pattern of solo-authored lower-quality submissions. Second, a fixed rule assumes that an individual author places the same review load on the system whether they submit a solo-authored paper or a paper in a large collaboration. We thus instead use a quota rule that is based on coauthorships. Our rule is carefully designed to constrain the authorship patterns disproportionately associated with low-quality submissions, while mitigating incentives to add spurious coauthors. More details are available here https://blog.tmlr.org/2026/annual-author-submission-quotas-for-tmlr/, and a tool for authors is here https://www.cs.cmu.edu/~nihars/quota/author.html?rule=generalized&N1=2&A=9&NA=9.

Since we introduced the submission quota policy in August 2026, we have detected several cases where authors attempted to bypass the policy. One method of circumvention we have seen is where an author who has exhausted their quota creates another OpenReview account and attempts to submit additional papers using that account. A second method of circumvention is by omitting some authors: once we reject a paper for an author going over their quota, the remaining authors submit it again by dropping the over-quota author from the list of authors. Any such unethical academic conduct or violations of OpenReview’s terms of use will be subject to strong actions.

There are a number of scientific integrity violations that have previously been discovered in the peer-review process in ML/AI such as identity theft, collusion rings, and many others. In ML/AI conferences and journals, it is usually the case that any scientific integrity violations discovered during the review process stay hidden. This absence of consequences unfortunately leads to repeat violations. We would like to change this culture. At TMLR, we will now deanonymize any case of major scientific integrity violation that has strong supporting evidence, after giving those involved a chance to respond. Such cases are far rarer at TMLR (handful per year) than at conferences, and we wish to ensure keeping it that way.

Our desk rejection rates have gone up from 6% in 2023 to about 53% now. To reduce burden on AEs, the Editors-in-Chief (EiCs) conduct an initial screening of all submissions, accounting for about 90% of these desk rejections; AE desk rejections account for the remaining 10%. We desk reject if either the claims or evidence are not clear from the writing (which is often the case for fully AI generated papers using current AI models), or if the evidence does not appear to meet the claims, or if the contents are deemed unlikely to be of interest to the TMLR audience. Given the large volume of submissions that we need to handle, we regrettably cannot provide individual feedback on most desk rejected submissions. That said, our exercise of interviewing authors of papers slated for desk rejection did give us more confidence in our desk rejection process.

Starting October 2026, each submission to TMLR also receives an AI review. The AI review evaluates whether the claims made in the paper are met with convincing evidence. It does not evaluate interestingness, which is left to human reviewers. The AI review can be used by the AE in their decision. It is also available to the AE for their desk rejection decisions. The AE is responsible for making the final decision, which is then reviewed and approved by the EiCs, with further discussion between the AE and EiCs when needed.

We have more initiatives planned in the upcoming months to navigate the rapidly changing scientific landscape. We will continue to explore ways to protect reviewer time, uphold scientific integrity, and maintain a high signal-to-noise ratio in the review process.

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →