Intel Perceptual Computing

Do you remember Tom Cruise's Minority Report directed by Steven Spielberg? With that fancy user interface Tom used when searching people from crime database?

Well, it's here now. Not 100%, but getting closer to that.

Intel published Perceptual Computing SDK 2012. SDK is free, all you need is a 149$ camera provided by Creative Technology Ltd, a development environment like Visual Studio and a bit passion to create cool software for creating greatest user experiences ever.

http://software.intel.com/en-us/vcsource/tools/perceptual-computing-sdk

With the Intel Perceptual SDK, you can detect few hand gestures like "peace" sign, hand movements, fingers, swipes, it has depth information that tells how far your hand is from the camera. It detects faces, recognizes voice commands etc. The most used development environment is Visual Studio C++, but you can do your things also with C# or Unity game development tool.

Detecting gestures and face

Some common questions I've been asked about this:

1. Is is stable?

-Pretty much, but I would not attend as a patient to a surgical operation, if the doctor is using this remotely.

And the license strongly advised not to use it in any critical systems, like car driving, controlling aeroplanes etc. 

Damn - I was just about to connect this with F-18C Hornet!

2. How much it costs?

The sdk is free, you need a 149 USD camera manufactured by Creative Labs and development environment. And some time. Not that much, if you’re familiar with Microsoft Visual Studio tools, but you’ll get started pretty fast. The cam itself looks pretty ok, it’s a lot heavier than they usually are. Maybe it tells about the quality, or just because the heavier cam stays easily at the top of the monitor(!)

3. Is there any useful apps developed for this?

Check out Intel's Perceptual Computing Challenge results from

http://software.intel.com/sites/campaigns/perceptualshowcase/

4. What kind of data you can get from this camera?

You get actual image frame, recognized gestures, depth data, hand coordinates from high level services provided by intel SDK etc. Also you’ll get also the raw data, if you wish to do some image and gesture processing by your self. And there are some voice recognition stuff.

The camera at the top of the monitor

Here is some C# code for gesture detection. The cam recognizes few gestures like hand waving, “peace”-sign, etc. I used it to control Windows 8 desktop.

public MyPipeline(Form1 parent, PictureBox recipient)

{

lastProcessedBitmap = new Bitmap(640, 480);

this.recipient = recipient;

this.parent = parent;

// setting up some features

attributeProfile = new PXCMFaceAnalysis.Attribute.ProfileInfo();

EnableImage(PXCMImage.ColorFormat.COLOR_FORMAT_RGB24);

EnableFaceLocation();

EnableFaceLandmark();

EnableGesture();

}

// when there will be a gesture, this is called

public override void OnGesture(ref PXCMGesture.Gesture gesture)

{

switch (gesture.label)

{

case (PXCMGesture.Gesture.Label.LABEL_POSE_BIG5):

if (sameCommandDelay != null && sameCommandDelay.AddSeconds(COMMANDELAYINSECONDS) < DateTime.Now)

{ // avoid too many commands -problem…

sameCommandDelay = DateTime.Now;

InputSimulator.SimulateKeyPress(VirtualKeyCode.LWIN);

}

break;

case (PXCMGesture.Gesture.Label.LABEL_HAND_CIRCLE):

base.Dispose();

//parent.Close();

//Application.ExitThread();

break;

case (PXCMGesture.Gesture.Label.LABEL_POSE_THUMB_UP):

if (sameCommandDelay != null && sameCommandDelay.AddSeconds(COMMANDELAYINSECONDS) < DateTime.Now)

{

sameCommandDelay = DateTime.Now;

VirtualMouse.LeftClick();

}

break;

Depth data, c++ demo from Intel