USE OF MINIMAL ABSTRACTION TO ADDRESS GOAL UNDERSPECIFICATION

Abstract

The problem of identifying and accounting for under-specified or misspecified goals has been widely considered central to AI safety. While most work in this space focuses on querying users about potential negative side effects, almost none look at the question of addressing the cause of misspecification, namely, the user’s misunderstanding regarding the consequences of pursuing the specified goal. In this work, this shortcoming is addressed by identifying the minimum level of abstraction at which the user can reason about possibly unavoidable side-effects of the current specification. This enables the user to make informed choices about what side effects are acceptable. A planning compilation is also introduced that allows the calculation of both the minimal abstraction and the set of unavoidable side-effects effectively. The effectiveness of the proposed method is evaluated through computational experiments, performed on standard planning benchmarks, and a user study.