OpenNI Application "Bouncing Ball 3"

in Japanese

Introduction

In the previous page, I showed the application that is using the OpenNI and the Open Dynamics Engine libraries with the Xtion Pro Live. Recently I have introduced the SceneAnalyzer node, which is supported by the OpenNI library, to the application. In this page, I provide you with an explanation on the new application. (Here is my first introduction to the ODE.)

Demo

I start by showing a demo movie.

Developement Environment

1. Mac OS X 10.8.2
2. Processor：3.06 GHz Intel Core 2 Duo
3. Memory：4GB
4. Xcode4.5.1

Primary Libraries

1. boost-1.51.0
2. pcl-1.6.0
3. OpenNI-Bin-Dev-MacOSX-v1.5.4.0
4. ode-0.12

My Source Code

Here is my source code. I do not confirm the behavior of the application on environments other than mine.

Compile Options

Header Search Path:

Path	Purpose
/usr/include/ni/	to include the OpenNI headers
/opt/local/include/	to include the Boost and the ODE headers
/usr/local/include/pcl-1.6/	to include the PCL headers
/opt/local/include/eigen3/	to include the Eigen headers on which the PCL depends
/opt/local/include/vtk-5.10/	to include the VTK headers on which PCL dependes

Library Search Path:

Path	Purpose
/opt/local/lib	to link the Boost library
/usr/local/lib	to link the PCL library

Other Linker Flags:

-lboost_thread-mt -lboost_filesystem-mt -lboost_unit_test_framework-mt -lboost_chrono-mt -lboost_program_options-mt -lboost_system-mt /usr/lib/libOpenNI.dylib -framework OpenGL -framework GLUT -lpcl_filters -lpcl_kdtree -lpcl_segmentation -lpcl_surface -lpcl_search -lpcl_features -lpcl_common -lode -lpcl_io

Usage

An execution file name is InteractiveObjects. Running it with an argument --help, it writes the following usage statements on the standard output. As shown above, the default window size is 320x240. I encountered the following error when running it with arguments --width 640 --height 480: The 21th-22th lines are output from the OpenNI library. I may be mistaken, but I think that the SceneAnalyzer node does not support the resolution 640x480.
Running it without any arguments (i.e. default size), it opens a viewer and begins to display frames captured by the Xtion Pro Live. Typing "m" on the keyboard under the situation where the application focuses on the viewer, a triangular mesh is constructed from a point cloud. The point cloud is calculated from the depth data in the frame at the moment when typing "m." The triangular mesh is used to create the background objects which consist of a floor, a wall, and so on. Then typing "r" brings virtual rigid bodies (balls and cylinders) into the captured scenes. The cylinders are fastened on to the floor. In the viewer, you can see that the balls collide with the cylinders, the background objects, and the other balls. Typing "o" enables an object detection, and the viewer begins to draw the collision between the detected object (human body) and the balls. To quit the application, type "q."

Brief Overview Of Source Code

I will show you different code from the previous one.

Transforming Depth into Point Cloud

In the method Device::create_cloud, a depth data captured by the Xtion Pro Live is transformed into three point clouds:

1. a point cloud which is used to draw RGB images (cloud_)
2. a point cloud which is used to calculate a triangular mesh which describes the background objects (background_cloud_)
3. a point cloud which is used to calculate a triangular mesh which describes the objects detected by the SceneAnalyzer node (object_cloud_)

The 11th-13th lines and the 39th-44th lines in the code shown below are related to the SceneAnalyzer node. The point cloud object_cloud_ is captured per frame, while background_cloud_ is captured at the moment when typing "m" and it is fixed until typing "m" again.

Rigid Bodies

I use the ODE library to describe the rigid bodies. The ODE objects in the application are defined as follows: The prefix character "d" indicates the class provided by the ODE library. The object which moves based on the rigid body dynamics is represented by the dBody object. The object which requires the collision detection is represented by the dGeom object. In the application, a ball moves under the rigid body dynamics and collides with each other and other objects. Therefore, it is defined as the 5th-9th lines, where dSphere is the derived class from dGeom.
On the other hand, a cylinder is fastened on to the floor, so it does not move, but only collides with the balls. For that reason, as shown in the 18th line, I defined it as the object of the class dCylinder which is the derived class from dGeom.

Additional details (e.g. the transformation of a point cloud to a triangular mesh, registration of a mesh to the ODE, a collision detection, and so on) are seen in the previous page or my source code.