Benchmark Concept for Industrial Pick&Place Applications

Robotic grasping and manipulation is a highly active research field. Typical solutions are usually composed of several modules, e.g. object detection, grasp selection and motion planning. However, from an industrial point of view, it is not clear which solutions can be readily used and how individual components affect each other. Benchmarks used in research are often designed with simplified settings in a very specific scenario, disregarding the peculiarities of the industrial environment. Performance in real-world applications is therefore likely to differ from benchmark results. In this paper, we present a concept for the design of general Pick&Place benchmarks, which help practitioners to evaluate the system and its components for an industrial scenario. The user specifies the workspace (obstacles, movable objects), the robot (kinematics, etc.) and chooses from a set of methods to realize a desired task. Our proposed framework executes the workflow in a physics simulation to determine a range of system-level performance measures. Furthermore, it provides introspective insights for the performance of individual components.