How ready are robots for the real world? A test for ethical performance

Jonathan Diller, David Barnes, Rebekah Bogdanoff, Rhett Collier, Roddy Collins, Keith Fieldhouse, Yonatan Gefen, Cameron Johnson, Anuriha Kodali, Brad Kriel, Varun Murali, James Niehaus, Mish Sukharev, Joseph VanPelt, Anthony Hoogs, Vijay Kumar, Arslan Basharat

Autonomous systems today are either blocked from unsafe actions or allowed to proceed—but users get no transparency about limitations or reasoning. REBAR introduces a quantitative benchmark that tests robotic systems against ethically difficult scenarios in photorealistic simulation, scoring them on an Autonomy Readiness Level (ARL) scale. The framework uses LLMs to generate diverse test cases and explain ethical difficulty, then maps performance metrics into computable, interpretable scores that tell users whether a system is ready for a specific task.