Combining ARToolkit & OGRE

AR and OGRE

It’s wonderful to see AR Applications, running within our physical environment. For an average guy it really looks like magic. But creating that magic was never easy. A professional magician always has to acquire a proven set of tools and techniques in order to deliver a realistic illusion. So as with the developer! For years ARToolkit served as the fundamental set of software package for crafting AR applications. However it becomes more rewarding to combine ARToolkit with a 3D Graphics Rendering Engine so that a one can create his own 3D graphics or even an animation and turn them into Augmented Reality. Therefore the objective of this article is to share my own experience in combing an AR application with a 3D rendering engine.

This application was tested in a Win32 platform. Practically I faced two challenges. The first one was aligning the pose of the marker accurately with the graphic object. The second one was to set the background of the OGRE’s render window into the camera view. It’s in the OGRE’s render window that we display both the camera background and the 3D graphic.

OGRE is a 3D rendering engine. It’s capable of drawing 3D graphics as well as animations in a window. Generally, OGRE provides two options for rendering graphics – i.e. Direct3D and OpenGL. Both OpenGL and Direct3D communicate closely with the underlying video graphics drivers in order to render the output. You can find a file called ogre.cfg getting created when you run an OGRE application for the first time. Usually this file resides in the bin\debug directory of OGRE installation-in my case it’s C:\OgreSDK\OgreSDK_vc10_v1-7-4\bin\debug. Modify this file to set the desired rendering method. It’s up to the developer to choose which rendering method to go with.

I know it’s quite lame (for pro’s ) but let me introduce the ARToolkit a little bit. It’s a set of C++ libraries and header files that facilitates in creating marker-based AR applications. It’s capable of tracking a predefined marker at real-time and calculate the pose at frame rate. By combining ARToolkit with OGRE we can track a marker real-time and apply its pose (position & orientation) to a graphics object, which is being rendered by OGRE.

Let’s see the steps involved in ARToolkit & OGRE briefly.

ARToolkit_OGRE

I started out by first downloading one of the tutorial applications in OGRE. Basically http://www.ogre3d.org/tikiwiki/tiki-index.php is good starting point. The challenge here is to supply the ARToolkit’s transformation matrix to OGRE’s graphics object. This can be achieved by running OGRE and ARToolkit in two separate threads. I used Pthreads from http://sourceware.org/pthreads-win32 .However you may stick to your own flavor perhaps… TBB (http://threadingbuildingblocks.org/) from Intel.

You’ll find out that in most of the OGRE’s tutorial applications, there’s a function named frameRenderingQueued(). We can update the scene for each frame through this function. So that we use the transformation matrix which is a 3 X 4 matrix calculated by ARToolkit and set the graphic object’s position & orientation according to following order.

Matrix

As with the above manner, we can apply the marker’s pose into the graphic object. Sometimes the pose of the graphic object might not coincide with the marker pattern. For example, when we rotate marker clockwise, the graphic object turns anti-clockwise. This happens due to the misalignments in the axis directions. We can overcome this by inverting such axis. (Probably this would take some time to realize which axis to be inverted – since it’s trial & error.)

The next big challenge is to set the OGRE’s background into the camera view. In OGRE it’s not obvious to directly capture the camera view and set it as background.

First we have to create a Texture through TextureManager class in OGRE. After that we have to create a Material and bind the texture to that Material. Then a rectangle box should be created and attach the material with this Rectangular box. Finally this Rectangular box should be rendered as the background. The tutorial here http://www.ogre3d.org/tikiwiki/tiki-index.php?page=Creating+dynamic+textures&structure=Cookbook would be helpful for more details. I also went through that.

The logic behind the above process is that the data which is confined within this rectangle area need to be filled with the camera’s captured pixels. So we need to acquire the pixel buffer within this box and assign each pixel of this pixel buffer to the value of the data pointer (dereference the pointer) which points to the ARToolkit’s pixel buffer of the new video frame. Obviously this was done inside a loop. In this manner we can set the background of the OGRE render window to the camera’s view at frame rate.

In summary…mentioned here is just one way of creating an AR application with 3D graphics and animations. The libraries/frameworks used for this application are as below.

  • ARToolkit
  • OGRE 3D
  • Pthreads

Here’s an important note to the end. Camera connection was made by ARToolkit through DSVL – DirectShow Video Library. (see here http://www.icg.tugraz.at/Members/daniel/DSVL_doc). Therefore you have to download necessary DSVL dlls in order to function. The pixel format is 32 bit RGB.

In a later post I’ll show how to modify the DSVL connection so that we can capture video from camera in different pixel formats, resolution etc..

Advertisements

8 thoughts on “Combining ARToolkit & OGRE

      1. After several studies and research, I obtained the following result ARToolki integrating with Ogre3D. See this video on Youtube

    1. Hi Paulo

      Please find below url for my code example and I must apologize for my delayed response. Nonetheless it’s better late than never…

      https://github.com/eranda1985/ARToolkit

      It’s a visual studio solution and should configure the libraries and include paths particularly for OGRE and Pthreads before starting the build process. After a successful build, the exe requires additional dlls (pertaining to OGRE & Pthreads) to be present at the same location (alternative you can deploy them to GAC). These additional dlls are already packed inside the zip.

      This is just a test example and code clarity is a little bit dubious. I made it as a POC for my research hypothesis and didn’t worry too much about refactoring.

      Thanks and hope this helps.

      1. Hi Paulo

        Thanks a lot.. I’m doing some research with markerless AR as of now and may be I’ll post you a query in need of an advice.. By the way you maintain a nice blog.

        cheers!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s