
Show HN: MediationBench – testing AI mediators in 62 simulated conflicts
Published by Human Assisted Intelligence, PBC
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Published by Human Assisted Intelligence, PBC
MediationBench compares AI mediators on the same 62 simulated two-party conflicts. Within each test, every system works with the same model-played parties under the same rules.
Early results are labeled provisional: one complete run, scored by one or two judge models.
Each run covers 62 authored disputes. Language models play both parties, and the system being tested acts as mediator. The mediator decides when to speak, wait, or end within a fixed 3-to-25-round window.
Sol scores every finished conversation after the run. A row scored by Sol alone is Sol's average across all 62 conflicts. Other judges can be added later and are shown separately under the row; each judge that scores all 62 conflicts is averaged into the row score, so early results stay inexpensive without pretending they are complete.
What happens when you put an AI in the middle of an argument?
Sixty-two fictional conflicts: a business breakup, a blocked driveway, a fight over credit. AI plays both sides, and neither starts out generous.
Add an AI mediator: that’s the test. After every message, it picks a move: speak, wait, or end it.
Afterward, an AI judge steps in. The mediator can say it went great. The judge only reads the transcript, and scores five things.
Sixty-two conflicts, one score out of a hundred. Early results are provisional, so small gaps don’t mean much.
It’s all AI: a measurement, not proof it works with people. But before AI sits in the middle of our arguments, it’s worth measuring.
Read the full method Read a sample conversation
Each provisional row shows a score from 0 to 100 and how often the final judge found that the conversation ended appropriately. Rows are ordered by score. Do not assume a small score gap is meaningful.
All parties are AI. The benchmark does not measure human safety, fairness, legality, or whether an agreement will last.
Jonathan Hendler · Working draft · Paper revision 2026-09-13 (13 September 2026)
The research page contains the exact method, earlier studies, limitations, corrections, and downloadable data. The working paper gives the complete design and evidence.
Numbers may change while the paper remains a working draft.
Provisional results are early evidence, not a settled ranking. This is a controlled model test, not evidence that AI mediation is safe or effective with people.
MediationBench is owned and operated by Human Assisted Intelligence (HAI.AI), which also develops mediation systems. It is not an independent evaluation.
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。