SIMULATION-BASED VALIDATION OF ROBOT COMMANDS

ABSTRACT

Speech-based robot instruction is a promising field in private households and in small and medium-sized enterprises. It facilitates the use of robot systems for experts as well as non-experts, especially while the user executes other tasks. Besides possible verbal ambiguities and uncertainties it has to be considered that the user may have no knowledge about the robot's capabilities. This can lead to faulty performances or even damage beyond repair which leads to a loss of trust in the robot. We present a framework, which validates verbally instructed, force-based robot motions using a physics simulation. This prevents faulty performances and allows a generation of motions even with exceptional outcomes. As a proof of concept the framework is applied to a household use-case and the results are discussed.

FRAMEWORK

Parameters of Interest

A Parameters of Interest Map (POI-Map) holds the desired object parameters for the robot, the workpiece and the remaining objects in a scene for a robot motion. For each object state s_i, its parameters can either be limited via thresholds, e.g. f_min < f ≤ f_max, or set as insignificant if they are of no interest in the current setting. The POI-Map then serves as a look-up-table for the interpreter component in a later step.

Simulation

To gain information about the dynamics while a motion is simulated, we implemented a simulation based on the Bullet Physics Engine. Inputs are a workspace configuration, i.e. the set of objects, a data base storing the static object information, e.g. inertia tensors, and a set of motion commands. Controlling the robot is implemented via a torque control. The necessary joint torques are computed at each simulation step via extracting the position and force information for the robots Tool Center Point (TCP) from the current MP and transforming them to appropriate joint torques. The complete simulation is then run several times with varying initial object transformations to simulate the inaccuracy of the system. After every simulation step the world state w(t) is logged in a parameter log L_P. An alternative would be to log state parameters only if a deviation in accompanying POI values take place. In most cases, this would result in a smaller amount of data, but it can also lead to data loss if the threshold for identifying a deviation is set too high. A maximum duration for the motion is also passed to the simulation in order to prevent infinite loops.

Interpreter

The interpreter evaluates L_P regarding the POI-Map to find critical parameter values, i.e. values which extend the thresholds. For each entry in L_P, its validity is checked regarding the POI-Map thresholds. If thresholds are exceeded, the simulation run is tagged as unsuccessful and critical values are stored in a result log L_R along with their timestamps and object IDs. The overall probability of success p_s ∈ [0,1] is then computed as the rate of successful to unsuccessful simulation runs. If p_s = 1 the instruction is accepted and the motion parameters are sent to the real world robot. If p_min < p_s < 1, p_min ∈ [0,1], the user is informed that the logged deviation may occur. If p_s < p_min the user is informed that there is a high risk that the logged deviation occurs. In both cases the user is asked whether the motion should still be performed. Thus, even motions with exceptional goals can be instructed as well.

Experiment

As a proof of concept, the framework is evaluated by the household use-case shoving with two objects. The framework is used with varying sensor accuracies to gain information about their impact on the resulting motion, and the results are compared to their real-word motions to investigate the simulation's validity.

A user plans to make a cake and is currently mixing ingredients in front of the robot. While stirring, the user instructs the robot: 'Can you shove the obj_i towards me?'. Where obj_i = {cocoabox, milkbox}. A shoving motion is automatically initialized, consisting of approaching the object beneath the z-coordinate of its centroid, which usually results in a robust shoving motion. The motion is simulated n = 9 times and the minimum success rate is set to p_min = 80%. The robot arm used in this example is a Franka Emika Panda. The POI-Map is set up indirectly through the shoving motion, i.e. translation into the y-direction, limited by a force that prevents damage of the box. The work-piece orientation is restricted by a threshold that ensures that it does not fall over during the manipulation. The remaining object states are tagged as insignificant. The sensor inaccuracy is modelled by varying the initial box centroid z- and y-coordinates by adding combinations of the accuracy ε ∈ [.1 mm, ..., 1 mm] in the y- and z-direction to it. The experiment was performed on a PC with the following specification: Intel Core i7-6700HQ processor and an NVIDIA GeForce GTX 960M graphics card.

Insights on the computational effort are gained by measuring the time for each simulation step for a step size d_step and the overall duration of each simulation run. Overall a single simulation step with 1 ms without sub steps took 0.086 ms and a whole simulation run (simulation + torque computation + logging + interpreting) took about 909 ms for a motion that takes 13.39 s in real life. Thus, an acceleration by a factor of about 14 is possible in this use-case, which is an adequate time to inform the user about the outcome of the motion. In the milk box case an accuracy of 1 mm leads to a successful execution in all runs. The success rate decreases to by 12% for 2 - 8 mm and drops to 77% for 9 and 10 mm. In this case the user would be informed that something may go wrong if the given accuracy is below 1 mm. In the cocoa box case a success rate of 55% is reached for all accuracies. We also executed the two motions on a real life robot arm (shown in this video ). The results are similar to our simulations, which leads us to the conclusion that our system is able to estimate the outcome of a robot motion.

CONCLUSION

In this paper an approach is presented which deals with the scientific question if and to what extent is it possible to validate force-based robot motions using a physics-based simulation. Software components for simulating, logging and interpreting logged object states are discussed and the system is evaluated by pushing manipulations in a household use case. Thus, the scientific contribution is an approach that allows a risk assessment of force-based motions as well as the execution of motions with exceptional outcomes.