instrumental convergence
Instrumental convergence is the AI safety idea that intelligent agents with very different final goals tend to pursue similar intermediate goals, such as acquiring resources.
Instrumental convergence is the thesis that agents with almost any final goal will share certain intermediate goals that help achieve it, such as acquiring resources and capabilities, preserving themselves and keeping their goals intact. Steve Omohundro and Nick Bostrom discussed the idea systematically, and Bostrom formulated it as the instrumental convergence thesis in his book Superintelligence.
In AI safety research, it is used to explain the risk that a goal-directed AI system may seek permissions or computing resources it was never asked to obtain.
This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.