Ixonos Goes "Imaging Tampere Get-Together"

Companies with a presence in Tampere, Finland have started movement towards making the region a center for imaging expertise, which means focusing efforts in pattern recognition, image enhancement, augmented reality etc. With this in mind, a get-together event was held in late November, and Ixonos with its bright and enthusiastic engineers had to be there too! Other participants included many participants from the Tampere University of Technology, Intel and several startups and older players in the fields of video surveillance etc.

Instead of just showing up with a stack of callcards, though, we decided to amuse the crowd by whipping up a special demonstration software running on the Intel MinnowBoard. It turned out well, and was much loved by the participants.

Ixonos Imaging Demo The system consist of a Playstation 3 camera attached to a MinnowBoard, along with a display for visualising the imaging algorithm results.  MinnowBoard is a small and low cost embedded platform using Intel® Atom™CPU. In addition, a racing track playset with two electric cars was used as the pattern recognition problem. The software consists of the Ixonos Embedded Linux BSP (base support package) , the OpenCV imaging library and a very simple application that tracks two cars on the racing track, calculating their lap times and counts.

Minnowboard (at the back), PS3 camera, racing track!

Car recognition is done by simple color segmentation. The colors are preset, and blobs of certain color are recognised with the OpenCV routine . The centroid of each blob is then visualised on the screen, and their passage over the "startline" is tracked. Very simple. Not a display of our pattern recognition algorithm abilities (call us if that is what you want), but rather of our ability to quickly integrate a complete system where we could later drop a specialised algorithm into. And fun. The purpose was to have fun!

More detailed image processing steps:

1. Capture image frames (640x480)
2. Resize frames down to 320x240
3. Blur to reduce noise
4. Convert from BGR to HSV color space
5. Apply filtering thresholds and create binary image
6. Use moments to calculate the position of the center of the object
7. Use coordinates to track the object and apply tracking visualizations on top of the image
8. Display frames with tracking visualizations

The proud author (Ilkka Aulomaa) of the playset car recognition system

About the authors Ilkka Aulomaa, M.Sc. - author of the car recognition system software and setup Mikael Laine, M.Sc. - author of this blogpost, and participator "in spirit" in creating the demonstrator (which means sitting on a sofa and making smart ass comments). He has has written his Master's thesis under the title "On Optical Character Recognition on Mobile Devices" (later published as "A Standalone OCR System for Mobile Cameraphones" in the proceedings of 2006 IEEE 17th International Symposium on Personal, Indoor and Mobile Radio Communications. He has also participated in research in the field of pattern recognition.

Intel Perceptual Computing

Do you remember Tom Cruise's Minority Report directed by Steven Spielberg? With that fancy user interface Tom used when searching people from crime database?

Well, it's here now. Not 100%, but getting closer to that.

Intel published Perceptual Computing SDK 2012. SDK is free, all you need is a 149$ camera provided by Creative Technology Ltd, a development environment like Visual Studio and a bit passion to create cool software for creating greatest user experiences ever.

http://software.intel.com/en-us/vcsource/tools/perceptual-computing-sdk

With the Intel Perceptual SDK, you can detect few hand gestures like "peace" sign, hand movements, fingers, swipes, it has depth information that tells how far your hand is from the camera. It detects faces, recognizes voice commands etc. The most used development environment is Visual Studio C++, but you can do your things also with C# or Unity game development tool.

Detecting gestures and face

Some common questions I've been asked about this:

1. Is is stable?

-Pretty much, but I would not attend as a patient to a surgical operation, if the doctor is using this remotely.

And the license strongly advised not to use it in any critical systems, like car driving, controlling aeroplanes etc. 

Damn - I was just about to connect this with F-18C Hornet!

2. How much it costs?

The sdk is free, you need a 149 USD camera manufactured by Creative Labs and development environment. And some time. Not that much, if you’re familiar with Microsoft Visual Studio tools, but you’ll get started pretty fast. The cam itself looks pretty ok, it’s a lot heavier than they usually are. Maybe it tells about the quality, or just because the heavier cam stays easily at the top of the monitor(!)

3. Is there any useful apps developed for this?

Check out Intel's Perceptual Computing Challenge results from

http://software.intel.com/sites/campaigns/perceptualshowcase/

4. What kind of data you can get from this camera?

You get actual image frame, recognized gestures, depth data, hand coordinates from high level services provided by intel SDK etc. Also you’ll get also the raw data, if you wish to do some image and gesture processing by your self. And there are some voice recognition stuff.

The camera at the top of the monitor

Here is some C# code for gesture detection. The cam recognizes few gestures like hand waving, “peace”-sign, etc. I used it to control Windows 8 desktop.

public MyPipeline(Form1 parent, PictureBox recipient)

{

lastProcessedBitmap = new Bitmap(640, 480);

this.recipient = recipient;

this.parent = parent;

// setting up some features

attributeProfile = new PXCMFaceAnalysis.Attribute.ProfileInfo();

EnableImage(PXCMImage.ColorFormat.COLOR_FORMAT_RGB24);

EnableFaceLocation();

EnableFaceLandmark();

EnableGesture();

}

// when there will be a gesture, this is called

public override void OnGesture(ref PXCMGesture.Gesture gesture)

{

switch (gesture.label)

{

case (PXCMGesture.Gesture.Label.LABEL_POSE_BIG5):

if (sameCommandDelay != null && sameCommandDelay.AddSeconds(COMMANDELAYINSECONDS) < DateTime.Now)

{ // avoid too many commands -problem…

sameCommandDelay = DateTime.Now;

InputSimulator.SimulateKeyPress(VirtualKeyCode.LWIN);

}

break;

case (PXCMGesture.Gesture.Label.LABEL_HAND_CIRCLE):

base.Dispose();

//parent.Close();

//Application.ExitThread();

break;

case (PXCMGesture.Gesture.Label.LABEL_POSE_THUMB_UP):

if (sameCommandDelay != null && sameCommandDelay.AddSeconds(COMMANDELAYINSECONDS) < DateTime.Now)

{

sameCommandDelay = DateTime.Now;

VirtualMouse.LeftClick();

}

break;

Depth data, c++ demo from Intel

Sweet and tasty approach to OpenCV and MinnowBoard

PC-esque cheap hardware is booming, and there seems to be no limit to the cool apps you can create on boards like Beaglebone, Rasperry Pi, or Minnowboard.

This obvious trend has had our attention for a long time now, plus we've got some customer cases going with the basic idea of migrating from expensive legacy systems to cheap off-the-shelf processing boards with huge capabilities in a meak form-factor.

Recently, some of our clients have expressed their interest in imaging systems, so we decided to whip up a small demo involving our "Ixonos BSP" small-footprint Linux distro and the industry standard OpenCV imaging library.

In this demo we used the MinnowBoard, Intel's small and low cost board which is based on Atom processor. The camera we used is a basic USB webcam from Logitech. Pictures below:

The Minnowboard with webcam watching candy drops

The camera setup allows the system to see some candy drops in this rather trivial pattern recognition demonstrator. The system acquires image rasters of the scene using v4l2 and OpenCV. Circle shaped patterns are detected using opencv function "HoughCircles", based on Hough Circle Transform. Code snippet below demonstrates simple circle detection using HoughCircles:

//circle detection
vector<vec3f> circles;
HoughCircles(detected_edges, circles, CV_HOUGH_GRADIENT,
             1, minSizeThreshold,  lowThreshold, lowThreshold/2,
             minSizeThreshold, minSizeThreshold + minSizeThreshold / 2);

printf("total circle count: %d\n",  circles.size());

After detecting all the circles, they are categorized according to color and statistics are printed to the screen.

Candy drops detected

  Another picture below illustrates a situation with some more candy drops.

More candy drops detected

Ilkka Aulomaa, SW Engineer - Ixonos
Kalle Lampila, SW Engineer - Ixonos