Stereo 3D camera helps robots get to grips with bin picking
3D machine vision is the future for robot vision. Many applications in robotics and automated serial production can only be satisfactorily implemented using three-dimensional data. This applies equally to challenging assembly processes such as bonding and welding, and to the notoriously tricky process of bin picking.
The ability to accurately detect the precise position of the unsorted parts is essential for reliable bin picking by robots. The Ensenso stereo 3D camera from IDS represents a solution for 3D image capture that is impressively precise, cost-efficient and fast. The German company bsAutomatisierung GmbH is using the new options available with this camera to achieve some seriously speedy bin-picking solutions.
Bin picking remains one of the most difficult tasks to execute in robot-supported production. In order to be able to pick unsorted parts out of a wire-mesh pallet or stacking container, it is necessary for the robot to be able to accurately detect the shape, size, position and alignment of the objects. All of this information is essential to the ad-hoc generation of collision-free robot paths.
However, challenges arise as soon as you begin to capture 3D data because short cycle times and process stability are required in addition to accuracy and completeness. The small number of solutions that have been available up to now are either too slow or fail to achieve the required level of process stability to be suitable for use in serial production. Alternatively, they are simply too expensive.
The Ensenso stereo 3D camera from IDS successfully covers all bases in terms of the requirements for cycle times, availability and cost efficiency that apply to robot vision applications. The industrial camera with a USB connection is the first of its kind on the market, integrating, as it does, not only two global shutter CMOS sensors with WVGA resolution and powerful software in a very compact housing, but also an infrared pattern projector. It projects a random pattern of dots onto the object to be captured, allowing structures that are not visible or only faintly visible on the surface to be enhanced or highlighted. This is necessary because stereo matching requires the identification of “interest points” in an image. The object is then captured by the two image sensors in accordance with the stereo vision principle. Finally, 3D coordinates are reconstructed or calculated for each and every pixel using geometric relations based on the triangulation principle. This means that, even if parts with a relatively monotone surface are placed in the bin, a virtually seamless and detailed 3D image of the entire surface can be generated without additional technical effort. All of this happens in just a few milliseconds.
The speed offered by Ensenso has benefits for bsAutomatisierung when applied to its bin picking robot cells. The company, which has its headquarters in Rosenfeld,Germany, specializes in the development and construction of systems for fast and precise loading and unloading of production machinery and parts handling. Its bin-picking cells can automatically pick individual, randomly aligned parts out of a container and pass them on to downstream production processes. The cells achieve cycle times of less than 10 seconds, which would not be possible with conventional solutions such as the light section method. Depending on the application and on customer requirements, the cells can be supplied with a classic multi-axis industrial robot or with a linear portal robot. They have a modular structure, can be adjusted to suit various container sizes, and are designed for use with either a single bin or up to four bins simultaneously.
Each bin is monitored by two stationary mounted Ensenso cameras. This offers two key benefits over a camera installed directly on the robot arm. First, the system can achieve shorter cycle times. While the robot is busy picking parts out of one bin, the process of object detection can already be started in another bin. Even if parts only need to be picked from a single bin, the process of searching for the next part can begin while the robot is placing the most recently picked part elsewhere.
As an additional benefit, the images from different Ensenso cameras can be very easily calibrated with each other, as the camera software has been specially designed to handle multi-camera applications. When two or more cameras are used simultaneously, a scene can be captured from different sides at the same time, thus reducing shadows and extending the picture field. When working with multi-ple cameras, the N10 software similarly delivers a single 3D point cloud contain-ing the data from all cameras used. The resulting precision can also be scaled to meet virtually any requirements by adjusting the number of cameras installed. In addition, the Ensenso software not only allows you to combine several stereo cameras, but also enables the use of these cameras in conjunction with conven-tional industrial cameras, for example, to capture color information or barcodes.
Finally, the Ensenso software assumes control of the two CMOS sensors and pattern projector, and takes care of the capture and pre-processing of the 3D data. As a result, both frame rate and image quality are optimized, and so significantly fewer resources are required on the PC used for analysis. The camera is designed for working distances of 260 mm to 1,400 mm and for variable picture fields. The available focal lengths of 3.6 to 16 mm can cover a wide range of distances and sizes. Unlike other 3D imaging processes, this camera can capture both stationary and moving objects with a frame rate of up to 30 fps. Despite having two sensors and an integrated projector, the camera only measures approx. 150 x 45 x 45 mm, and weighs just 400 g. Thanks to its robust aluminum housing and GPIO connector for 12-24 V hardware triggers, input and output, the Ensenso is perfectly equipped for industrial applications. Accordingly, the 3-pin M8 sensor/actuator connector and USB cable have a screw-mountable design.
However, the Ensenso offers OEMs and system integrators another key benefit. It is ready for immediate use and delivers 3D data out of the box. This means it can be installed straight away. All that remains is for the robots to be calibrated for working with the camera. This is very easily done using a calibration plate mounted on the robot gripper. The software uses this plate to calculate the mounting position of the camera, and the 3D data is immediately represented in the robot’s coordination system.
The images captured are analyzed using Halcon 11. The interface required is included in the Ensenso’s comprehensive software package, alongside an API for C, C++ and C#. The target coordinates of the found parts and a map of the remaining contents of the container as a dynamic obstacle is then forwarded to a software module. The software uses this information, together with the CAD data from the cell, the robot and the robot gripper to generate a collision-free robot path. The path is transferred to robot control, where it is then executed. The at-tempt at bin picking may fail due to inaccuracies, incorrect identification or if parts slip out of position. The robot detects this failure thanks to various sensors in the gripper and collision monitoring between the gripper and the flange. The robot then navigates independently out of the container and tries to pick the part in a different location. No human intervention is required. The entire process is monitored and controlled by a PLC. The PLC is also responsible for informing the ma-chine vision system which type of part is to be picked, when, and from which bin.
Until now, the potential offered by 3D machine vision has been hardly tapped due to the complexity and relatively high costs of the technology. However, these bin-picking cells from bsAutomatisierung demonstrate how this process can be implemented with guaranteed cost efficiency and process reliability thanks to the use of the Ensenso stereo 3D camera. With its USB 2.0 port, the camera is easily integrated, while its technological design also makes it attractive from a price standpoint. And, in terms of speed, the Ensenso is streets ahead of other comparable solutions. In addition to the Ensenso with a USB interface, a model will soon be available with a GigE connection and two 1.3MP CMOS sensors from e2v.
