How to Evaluate Controlled Natural Languages

Abstract: This paper presents a general framework how controlled natural languages can be evaluated and compared on the basis of user experiments. The subjects are asked to classify given statements (in the language to be tested) as either true or false with respect to a certain situation that is shown in a graphical notation called "ontographs". A first experiment has been conducted that applies this framework to the language Attempto Controlled English (ACE).