Using Contiki with linux CCS

Creating Contiki/CETIC based IPv6 border router solution (6LBR) with BeagleBone Black and using TI SensorTag2 with SLIP to connect 6LoWPAN mesh network is one thing. Debugging node from linux is another. Let's see how it works :)

Getting Contiki git repo and compiling web-demo for TI Sensortag in linux works quite well. Flashing it not so. Recommended way is to
transfer image to windows and flash from there, but it's possible to flash from linux commandline also. Using Code Composer Studio (CCS) makes job much easier, and you can also debug your code, at least in theory.

CCS can import project with makefile-style compiling (File -> Import -> C/C++ -> Existing Code as Makefile Project). Unfortunately it only can understand simple Makefile and Contiki uses include etc. Even when you can compile image, CCS will complain lot's of errors and warnings. It's quite bizarre situation. One way to 'fix' it, is defining things in makefiles to headers so CCS known what is actually defined when compiling. Like adding to project-conf.h:

/* for CCS as it can't detect this from Makefile.include */
#define NETSTACK_CONF_WITH_IPV6 1
/* CCS uses wrong button-sensor.h */
#ifndef BUTTON_SENSOR_VALUE_DURATION
#define BUTTON_SENSOR_VALUE_DURATION 1
#endif

Also finding source files can be a bit tricky. Adding whole contiki would be simple way so solve this but CCS don't like adding parent of project. Way around is adding links (to project) to needed subdirectories:
project -> properties -> c/c++ general -> Paths.. -> Source Location -> Link Folder
apps, cpu/cc26xx-cc13xx, core, platform/stf06-cc26xx
Also in c/c++ build -> settings you can enable gnu-versions of parsers.

You can also add Makefile.include from top, but it only makes editing easier (project -> Add Files -> Link).

Also you need to get debug-symbols to elf, so add Makefile:
CFLAGS+=-g
LDFLAGS+=-g

Nice. Now it doesn't complain and (most) code it shows is actually right one. To debug you need to create .ccxml target configuration file (project -> New).

Note that when you connected Debug DevPack to PC, it shouldn't have SensorTag2 connected to it. After board have booted (green led on) connect SensorTag2. Also when debugging first time, Debug DevPack firmware is updated to newest one.

Start debugger. Change Debug -> .ccxml -> Edit.. -> Program -> Cortex_M3_0 Project to project root and Program to .elf file. Check that run to symbol main is set. Next start of debug should then automagically load & run to main(). If it doesn't automatically load image to SensorTag2, you can choose Load & select elf.

Unfortunately good things stops there. For some reason JTAG connection to SensorTag2 seems to disconnect really often. Even dropping it's speed from 2.5MHz to 100kHz doesn't help :( This is probably due eclipse blocking and emulation package then misses something from JTAG/Debug DevPack. Hopefully future update fixes (some of) this.

Debugging active and ready to run

Jari Vanhala, Senior SW Designer, Ixonos

Quick Visual Dashboard

When developing IoT systems you’re often dealing with a huge amount of time series data. Basically key-value pairs bound to a timestamp. Or maybe you are debugging some sensor output or feedback controller only using text output on your terminal. Getting a sense of what kind of data your sensors are outputting is not easy when you’re dealing with tens, hundreds or even thousands of sensors and measurements.

Various tools already exist to visualize time series data. They are very good at visualizing large amounts of data in multiple different ways. But setting these tools up often takes time and the learning curve to effectively configure the visualizations to view your data is often an unwelcome bump in the road when all you care about is getting a sense of the data you’re dealing with without bothering with the details.

Real-time data visualized using simple graphs with minimum lag.

When developing, testing and prototyping, it is essential that you get good understanding, how your system performs in real-time and also within a longer time-frame. You don’t want to spend time configuring and setting up visualizations tools. This is why we put together Quick Visual Dashboard. A local or cloud deployed visualization tool where you can input data using standard TCP or MQTT protocols. With simple script templates done in Python, uploading data to the tool can be done in no time.

Built-in geospatial data support to see how and where your data is on the map in real-time.

The data is preprocessed and backed by InfluxDB. This means that after you’re done with your initial prototyping and you stop caring about real time data - you can go back and use industry proven tools like Grafana to further investigate all the details of your time series data at any point in time.

Using Grafana it is possible to dive deeper into historical time-series data.

Christian Talmo, Full-stack Developer, Ixonos

Stanislav Radomskiy, Senior SW Designer, Ixonos

NFC as an Enabler for Efficient Asset Management

When it comes to managing assets in industrial environments there are a few use cases that come up frequently. And naturally also a few problems relating to each. In this post an asset can be pretty much any device that monitors or controls an industrial manufacturing process. Of course assets also exist in environments with moving machines and logistics; like harbor areas.

 When a service person is visiting a site (either during installation, as a routine check-up, or responding to a service request) the first thing to overcome is usually identifying the asset he or she wants to operate. For example there can be various injection moulding machines operating in a factory floor. The machines may be from different suppliers, of different generation/version, running different software and/or configurations, and so on.

 After the asset has been identified, it usually needs to be accessed to retrieve logs, adjust parameters and so on. For this purpose there are various local control panels. These often don’t have the most modern and easy-to-operate user interfaces. Also the panels may be physically located in a hard-to-reach place (e.g. behind a corner, between a machine and a wall, etc.).

 In some cases you can or even have to connect an external control device (like a laptop) to the asset to control, reconfigure or even reprogram it. This is traditionally done with a laptop and some kind of serial cable. In more modern solutions (like Ixonos Internet Suite) it may also be possible to do the connecting and configuration wirelessly e.g. with a tablet or other mobile device.

 To solve the issues above and make the service person’s work as efficient and ergonomic as possible, there are several ways where modern wireless technology can help. You could use RFID tags or barcodes to identify the assets and setting up the connectivity.

 We decided to take a closer look at NFC. By using simple, inexpensive passive NFC tags we were able to significantly speed up identifying, accessing and connecting to assets in the field.

 Check the video below for details and examples:

All-in-all NFC is a modern and reliable technology for improving the efficiency of service personnels work in the field. Of course this is only a small part of a comprehensive solution which requires lot of intelligent software solutions both on site and in the cloud. Check our previous blog posts and stay tuned for more relating to those.

Manish Kumar, Senior SW Engineer, Ixonos
Mikko Mäkinen, SW Engineer, Ixonos

Mesh networks for industrial internet

Industrial internet is mostly about collecting, analyzing and visualizing data to enable new use cases or to enhance operations. One of the challenges in getting industrial internet systems up and running is how to efficiently set up and manage networks consisting of dozens or even hundreds of sensors. Wirepas have developed an interesting wireless mesh network solution they call Pino™ to solve this issue.

To test the usability of the Wirepas mesh with our existing data collector, Sensor Hub, we added a Wirepas plugin to the Hub, and designed a set of simple sensor boards that measure temperature, absolute air pressure and battery voltage. The readings were then transferred via UART to a reference HW board developed by Nordic Semiconductor running Wirepas Pino™ stack. These boards automatically organized themselves to a mesh and relayed the data to a gateway, a simple USB dongle with a virtual serial port. Users were able to view data collected from the sensors locally, and the Hub also pushed the data to the cloud using either ethernet, WiFi or cellular network.

See video of the solution in use:


Wirepas Pino™ mesh is not aimed at low latency real-time data, but rather at giving possibility to easy deployment of very low power battery operated mesh network with zero configuration and minimal installation.

Just drop the wireless sensor nodes where needed and the mesh automatically configures itself. If the signal range has to be extended, simply drop another node on the way and it starts relaying the data.

Most of the time there are multiple routes through the mesh for relaying data. The sender can set preference for either low latency or energy. The mesh automatically optimizes routing based on the energy reserves available in the nodes, or by fastest route. If a node is low on battery, the mesh automatically starts avoiding that node to further save its battery.

In case of deploying large amount of nodes, it is possible to add more gateway nodes to distribute the traffic load and also to avoid single point of failure.

Based on our experience the Wirepas Pino™ does it's job in providing a wireless sensor network that is easy to set up and administer. This quite nicely complements Ixonos' solutions that focus on data collection, cloud storage, and visualization of data quite nicely.  We look forward to creating solutions for our Industrial customers using Wirepas technologies.

Teemu Rasi, Embedded Engineer, Ixonos Plc

Linux Flashing for Freescale i.MX6

We often have customer and R&D projects where we build embedded SW solutions on the Freescale i.MX6 platform. As our OS of choice usually is Linux, developing those solutions has been a bit problematic since Freescale's own tool for flashing (MFGTool) is only available for Windows. Our developers finally got bored of having a separate Windows -workstation for flashing the stuff they had developed on Linux. Therefore they decided to create a Linux tool for flashing to speed up the
development process.

The result of this work was a tool called utp_com which can be used with imx_usb to flash i.MX6 based hardware from Linux environment. As we are talking about Linux we also decided to make the tool publicly available.

Development started from point where we were able to flash "flashing OS" into the device with imx_usb tool. After this a SD card device (/dev/sd* ) is available from the target device to the host machine via an USB connection. The SD card device is not a conventional one, but communication is done with the SCSI commands. On top of them Freescale has defined an Update Transport Protocol ( UTP ) which is used for communication. UTP message data is sent in the vendor specific CDB field of the SCSI message. Return values come as part of the SCSI sense data.

Example commands how a file is copied to the target device and then written to a flash device below:

utp_com -d /dev/sdb -c "send" -f u-boot.bin
utp_com -d /dev/sdb -c "$ dd if=\$FILE of=/dev/mtd0 bs=512"

Code is under GNU GPL v2.0 license, and can be found from: http://github.com/ixonos/utp_com.

We hope you find it as useful as we do!

Teemu Piiroinen, SW Specialist, Ixonos Plc

Fast Piloting of Industrial Internet

The buzz and fuzz around Industrial Internet seems to accelerate almost daily. While it is clear that the possibilities and potential benefits are endless, it is quite difficult to decide how to reach those in practice. We wanted to create a solution that makes it easy to setup the “basic infrastructure” to enable piloting different applications of Industrial Internet in practice.

We have seen that often a lot of data is collected from Industrial automation systems through different sensoring and other solutions. If that data could be easily moved to cloud, analysed and presented to users in smart ways, it would be easier to experiment with ways to utilise that data to improve efficiency of operations or even as an enabler for new digital services.

With this in mind we took our Industrial Internet Suite -software solution and installed it on Industrial grade reference HW with 3G connectivity. Our sensact -communication framework combined with Ixonos Cloud and Remote Dashboard make it easy to “plug in” your existing automation environment and start visualising your data. The HW/SW combination supports connections through common field-buses (like Modbus). Check out the more detailed architecture from a previous post.

After having the solution in place, we put it in a nice package. See the video below about what it looks like in practice.



We now have a set of these packages that we are offering to our customers for pilot use. This way it is very easy and risk-free to try out what you can get out of already existing data and automation by adding modern “Industrial Internet” -technologies.

Contact us for more info or a trial. There's also a white paper available concerning the topic on a bit wider scope.

Jukka Hornborg, Head of Offering Management, Ixonos Plc

Renew web & mobile automated testing with a new Page Model -based tool

Today more and more services are becoming web based. This means that modern user interfaces are built using HTML5 and other web technologies, even in embedded environments. At the same time applications and services are becoming more complex which means that testing is more crucial than ever in bringing high quality software to markets on time.

There are several existing tools to automate UI testing mostly based on different recording and playback methods. While it is easy to develop such tools, using them and especially maintaining test assets created with them is not very easy. Also it may be hard to match reported errors to actual web page elements (e.g. “element_id_zyz_2” is missing).

We decided to try a more innovative approach. We wanted to create a tool that is easy to use, while powerful enough for automating the testing of even the most complex and dynamic user interfaces. Also the solution should have long lasting support and have a wide cross-browser support across different desktop and mobile devices.

To meet the usability criteria we developed the Page Model approach. Here’s what it means in practice:

• Page Model is similar to the Page Object, which represents the screens of your web app as a series of objects.
• The difference is that the Page Model has more elements:
• Information of web app screen and model type (full screen, selected area of screen)
• Information of Page Objects (web elements and dynamic objects)
• Screenshot of full screen or selected area of your screen
• Methods containing the functionality of the web page (e.g. login method)
• Page Model transforms the information of a web page's elements automatically to textual format that can be used instantly in creating test scripts.
• Page Model screenshot visualises the web page and it is used for selecting objects in test creation.
• Page Model file also contains methods that can be used to execute actions present on the web page. It can be understood as a Page Model specific “function library”.
• After creating models, test scripts and methods can be easily constructed utilizing our graphical user interface

As a technology base we chose Selenium Webdriver, which has support of the largest browser vendors and is a widely used and well maintained Open Source tool. This is completed by the also Open Source Appium -framework to enable Selenium -based testing on mobile devices.

We combined these algorithms and technologies to Ixonos Visual Test(TM), which is a powerful set of tools for making testing easier both for test engineers and developers. With our tool you can plan, create, plan and maintain test scripts and test assets more visually. You can produce easily maintainable Page Models simply by browsing your web or mobile application. We have support for dynamic locators, javascript, AJAX and other special elements already implemented. We even have a solution for accessing HTML5 canvas elements from the test scripts; something at least we have not seen before. Everything is done with a modern powerfuI UI and the generated scripts are standard Selenium Webdriver format.

The best part is that the our tools also makes it possible to automatically detect changes to web pages. Detected changes (new and missing elements, broken methods) are visualized on top of the web page (see picture) and the tool even proposes fixes for those.

One of cool things is that you can generate tests graphically using a model graph UI that defines page model transitions and methods which are used there. The tool will go through the model graph and make test cases based on it.

To see introduction video, watch this:

And to witness the same approach applied to Android devices watch this:

Got interested? Visit our product pages for more information and a free trial Windows/Linux version.

And stay tuned for next innovations!

Anssi Pekkarinen, Solution Architect/Lead Test Automation Consultant - Ixonos Plc

Ixonos Industrial Internet Suite Goes Cloud

During the last months our R&D team has been hard at work creating a complete “data-pipe” from sensors to cloud. By combining some of our existing components (like Wireless sensor data collection with BTLE, Interactive embedded touch GUIs with HTML5 and Ixonos sensact library) and creating new ones we now have a solution in place, called Ixonos Industrial Internet Suite.

Basic architecture is illustrated in the picture below. Basically we utilize our sensact libraries running on Linux to collect the data, secure websocket connection to create local Human Machine Interface views on mobile devices and ship the data to Ixonos Elastic Cloud.

Our UI framework choice for everything is HTML5. This makes our solutions run on different platforms with minimal customisation. Of course we do some tweaking to make the HTML5 apps run smooth and stable on hardware with limited resources.

Meanwhile our fellows at Ixonos Design Studios have been working magic and creating a complete facelift for the UX of the different solutions. With these guys even industrial automation can be fun and easy to use.

Take a look the video below for visualisation of the system:



Got interested? Check out more info and contacts at: http://www.ixonos.com/business-areas/industrial-internet

Jukka Hornborg, Head of Offering Management, Ixonos Plc

Wireless Sensor Data Collection with BTLE

Bluetooth Low Energy protocol, also known as Bluetooth 4.0 or Bluetooth Smart, is a hot topic right now. At Ixonos, we have been working with it for a while and one of the examples is the addition of Texas Instruments SensorTag support to our libsensact library.

TI SensorTag is a small Bluetooth Low Energy device, which has 6 different sensors and runs on a coin cell battery with very small current consumption.

The applications, which use libsensact, are able to set TI SensorTag as one of the sensors where to connect and read the sensor data from it.

The code for connection to TI SensorTag is similar to the code needed for USB sensors in the earlier example. The difference is that you need to define the BTLE addresses of the devices instead of USB ids:

/* List of supported sensor/actuator devices */

struct ble_sensortag_config_t ble_sensortag0_config =
{
      .ble_address = "BC:6A:29:C3:3C:79",
};

struct ble_sensortag_config_t ble_sensortag1_config =
{
      .ble_address = "BC:6A:29:AB:41:36",
};

struct sa_device_t devices[] =
{
   {  .name = "ble_sensortag0",
      .description = "TI sensortag 0",
      .backend = "ble_sensortag",
      .config = &ble_sensortag0_config },

   {  .name = "ble_sensortag1",
      .description = "TI sensortag 1",
      .backend = "ble_sensortag",
      .config = &ble_sensortag1_config },

   { }
};

int main(void)
{
    int sensortag0;
    /* … */
    sensortag0 = sa_connect("ble_sensortag0");
    /* … */
}

The video shows the code and TI SensorTag in action on our HTML5 demonstrator prototype.

Now that the basic BTLE -support is in place we have an easy solution for bringing wireless sensors within our sensor framework. Adding support for new BTLE sensors is pretty straightforward with our scalable architecture.

Stay tuned for further updates as we are combining this and other components into the Ixonos Human Machine Interface -solution to be launched in near future!

Tero Koskinen, Senior SW Designer - Ixonos
Petteri Tikander, Senior SW Designer - Ixonos

Ixonos Multi-Display for Android 4.4.2 with Miracast

The Ixonos Multi-Display solution has, since the previous post, been ported to Android 4.4.2 and a few features has been added in the process. It is truly medium agnostic and works over 'whatever medium' supported by the DisplayManagerService (i.e. MHL/HDMI/MiraCast etc.) of the platform. Finally applications can be moved between displays through the 'recents' menu.

The solution addresses the limitations of the Android platform when it comes to multitasking and running several apps in parallel on different displays.

The below video shows a Nexus 10 tablet running Android 4.4.2 initially connected to a TV via HDMI playing an action flying game. New input methods like track pad and game controller has been added to the System UI to provide input for the external display. This provides mouse- and game controller input events, enabling all games that supports the standard Android game controller API to be controlled from the tablet.

At a later point in time, the tablet is connected wirelessly to the TV using Miracast via a Netgear Push2TV display adapter. This enables a true cordless Multi-Display experience where users can enjoy content on a secondary screen without the hazzle of cables.

Vasile Popescu, Chief Software Engineer - Ixonos
Mikkel Christensen, Chief Software Engineer - Ixonos
Martin Siegumfeldt, Chief Software Engineer - Ixonos
Jakob Jepsen, Chief Software Engineer - Ixonos

Ixonos Goes "Imaging Tampere Get-Together"

Companies with a presence in Tampere, Finland have started movement towards making the region a center for imaging expertise, which means focusing efforts in pattern recognition, image enhancement, augmented reality etc. With this in mind, a get-together event was held in late November, and Ixonos with its bright and enthusiastic engineers had to be there too! Other participants included many participants from the Tampere University of Technology, Intel and several startups and older players in the fields of video surveillance etc.

Instead of just showing up with a stack of callcards, though, we decided to amuse the crowd by whipping up a special demonstration software running on the Intel MinnowBoard. It turned out well, and was much loved by the participants.

Ixonos Imaging Demo The system consist of a Playstation 3 camera attached to a MinnowBoard, along with a display for visualising the imaging algorithm results.  MinnowBoard is a small and low cost embedded platform using Intel® Atom™CPU. In addition, a racing track playset with two electric cars was used as the pattern recognition problem. The software consists of the Ixonos Embedded Linux BSP (base support package) , the OpenCV imaging library and a very simple application that tracks two cars on the racing track, calculating their lap times and counts.

Minnowboard (at the back), PS3 camera, racing track!

Car recognition is done by simple color segmentation. The colors are preset, and blobs of certain color are recognised with the OpenCV routine . The centroid of each blob is then visualised on the screen, and their passage over the "startline" is tracked. Very simple. Not a display of our pattern recognition algorithm abilities (call us if that is what you want), but rather of our ability to quickly integrate a complete system where we could later drop a specialised algorithm into. And fun. The purpose was to have fun!

More detailed image processing steps:

1. Capture image frames (640x480)
2. Resize frames down to 320x240
3. Blur to reduce noise
4. Convert from BGR to HSV color space
5. Apply filtering thresholds and create binary image
6. Use moments to calculate the position of the center of the object
7. Use coordinates to track the object and apply tracking visualizations on top of the image
8. Display frames with tracking visualizations

The proud author (Ilkka Aulomaa) of the playset car recognition system

About the authors Ilkka Aulomaa, M.Sc. - author of the car recognition system software and setup Mikael Laine, M.Sc. - author of this blogpost, and participator "in spirit" in creating the demonstrator (which means sitting on a sofa and making smart ass comments). He has has written his Master's thesis under the title "On Optical Character Recognition on Mobile Devices" (later published as "A Standalone OCR System for Mobile Cameraphones" in the proceedings of 2006 IEEE 17th International Symposium on Personal, Indoor and Mobile Radio Communications. He has also participated in research in the field of pattern recognition.

Interactive Embedded Touch GUIs with HTML5

Recently we've been considering graphical user interfaces (GUIs) from the point of view of a systems integrator. There are several thing to consider, when creating a complete solution, such as: several different software platforms ( embedded devices, phones, tablets, desktop computers, ...), data network considerations and future proofing.

Several technical solutions come to the rescue here. Firstly, there are standards that span several (all involved) platforms and allow software development to be done once - with perhaps some adaptation for each platform. Secondly, networks of all sizes and shapes allow for powerful distributed systems, where data can be shared and interaction happens across the room or from the other side of the globe.

The Ixonos Embedded HTML5 library - ixgui.js - has proven to be a highly flexible and scalable platform for creating embedded GUIs. Recently, a number of system topologies have been explored using ixgui.js, involving running the GUI as detached from the embedded device. HTML5 obviously fits natively into this kind of distributed environment. The GUI can be hosted on the cloud, on an embedded device or basically anywhere.

Sensor data sharing in our demonstrator is fascilitated using the Ixonos sensact library, which you can read about in an earlier blog post.

The user interface for this demo is simple. It displays data coming in from the TI Sensor Hub Booster Pack. In addition, there is an RPM display and setting slider, but that is only for show: there is no motor in this version - but in later ones there will. below is a screenshot of the GUI:

Simple Touch Interface using ixgui.js

The below video illustrates using this GUI on the Texas Instruments AM3359 Evaluation Module with a separate, more elaborate, GUI running on a detached display.

ixgui.js is a HTML5-based GUI library, which allows performance optimized GUI creation by using the Canvas 2D interface for fast graphics and fine control over what is drawn at a given time. It is designed around the principles of simplicity, performance, standards compliance and programmer friendliness.

This article outlines some key methods for improving Canvas 2D performance. It has been extremely gratifying to fine-tune drawing for ixgui.js, and indeed we implement optimization on several levels.

On the top level, rendering is optimized by only drawing what needs to be redrawn. For most GUIs, only when items are interacted with, do they require to be redrawn.

Pre-rendering: often a large part of an item is static, and actually requires no update at all during the entire lifecycle of an application. In these cases, we can simply pre-render those areas that don't change onto a separate buffer, and reuse that for each redraw. As an example, see the below picture for how the vertical sliders in the demo are drawn:

Finally, at the lowest level possible (in JavaScript), some optimization is achieved by only feeding integer values to drawing routines. All coordinates and dimensions throughout the GUI are cast to integers.

Mikael Laine, SW Specialist - Ixonos

Ixonos Multi-Display for Android

Ixonos enables its Multi-Display feature for recent generation Android - making multi-tasking easy.

Have you ever tried using the Android secondary display API's (described here) that was introduced in Android Jellybean 4.2? Using the "Presentation" class to show content from your app is quite cool, but you are still limited to to run only one activity at a time. Basically you launch a Dialog (Presentation) to the secondary display from your activity running on the main display. This is useful for certain types of apps like e.g image and Powerpoint presenters, but what about running the stock Android Browser on one display and watching YouTube on the other?

Watch the video below and see what Ixonos has created to enable true multitasking for multiple displays.

This technology is a generic solution that enables the user to run existing Android applications on either display and also to map external input devices to the given display. Also, it utilizes the new Android display manager service and is thus display agnostic, meaning that we can use any type of display, eg. HDMI or Miracast. The Multi-Display feature can be integrated with recent generation Androids (4.2, 4.3, 4.4) by our engineers.


Vasile Popescu, Chief Software Engineer - Ixonos
Mikkel Christensen, Chief Software Engineer - Ixonos
Henrik Kai, Chief Software Engineer - Ixonos 

Build Gear version 0.9.19beta released!

A new version of Build Gear has recently been released.

Build Gear is the open source build tool that is used to build the Ixonos Embedded Linux BSP for various embedded boards based on a range of different chipsets including TI OMAP/AM/DM, Freescale IMX, Intel Atom/Haswell, etc.. This build tool allows us to very effectively create and maintain clean cut modern Linux BSP firmware tailored to fulfil the requirements of individual embedded customers.

This release includes a couple of new features and some bug fixes.

One of the new interesting features is the introduction of a new command to create a software manifest which provides a detailed list of the software components involved in a particular build. This is a quite useful feature in case you need an overview of the licenses of the components going into your firmware. Actually, for most this is an important feature so that the BSP firmware can be legally approved before going to production.

For more details see the release announcement here

The Build Gear tool has been in beta stage for quite some time but it has now stabilized to the point where it is ready to move out of beta. Thus, it will soon be labelled stable and a 1.0 release will mark the final transition out of beta.

Expect more posts from me on this build tool and on how and why we use it to create the Ixonos Embedded Linux BSP platform solution.

Keep it simple!

Intel Perceptual Computing

Do you remember Tom Cruise's Minority Report directed by Steven Spielberg? With that fancy user interface Tom used when searching people from crime database?

Well, it's here now. Not 100%, but getting closer to that.

Intel published Perceptual Computing SDK 2012. SDK is free, all you need is a 149$ camera provided by Creative Technology Ltd, a development environment like Visual Studio and a bit passion to create cool software for creating greatest user experiences ever.

http://software.intel.com/en-us/vcsource/tools/perceptual-computing-sdk

With the Intel Perceptual SDK, you can detect few hand gestures like "peace" sign, hand movements, fingers, swipes, it has depth information that tells how far your hand is from the camera. It detects faces, recognizes voice commands etc. The most used development environment is Visual Studio C++, but you can do your things also with C# or Unity game development tool.

Detecting gestures and face

Some common questions I've been asked about this:

1. Is is stable?

-Pretty much, but I would not attend as a patient to a surgical operation, if the doctor is using this remotely.

And the license strongly advised not to use it in any critical systems, like car driving, controlling aeroplanes etc. 

Damn - I was just about to connect this with F-18C Hornet!

2. How much it costs?

The sdk is free, you need a 149 USD camera manufactured by Creative Labs and development environment. And some time. Not that much, if you’re familiar with Microsoft Visual Studio tools, but you’ll get started pretty fast. The cam itself looks pretty ok, it’s a lot heavier than they usually are. Maybe it tells about the quality, or just because the heavier cam stays easily at the top of the monitor(!)

3. Is there any useful apps developed for this?

Check out Intel's Perceptual Computing Challenge results from

http://software.intel.com/sites/campaigns/perceptualshowcase/

4. What kind of data you can get from this camera?

You get actual image frame, recognized gestures, depth data, hand coordinates from high level services provided by intel SDK etc. Also you’ll get also the raw data, if you wish to do some image and gesture processing by your self. And there are some voice recognition stuff.

The camera at the top of the monitor

Here is some C# code for gesture detection. The cam recognizes few gestures like hand waving, “peace”-sign, etc. I used it to control Windows 8 desktop.

public MyPipeline(Form1 parent, PictureBox recipient)

{

lastProcessedBitmap = new Bitmap(640, 480);

this.recipient = recipient;

this.parent = parent;

// setting up some features

attributeProfile = new PXCMFaceAnalysis.Attribute.ProfileInfo();

EnableImage(PXCMImage.ColorFormat.COLOR_FORMAT_RGB24);

EnableFaceLocation();

EnableFaceLandmark();

EnableGesture();

}

// when there will be a gesture, this is called

public override void OnGesture(ref PXCMGesture.Gesture gesture)

{

switch (gesture.label)

{

case (PXCMGesture.Gesture.Label.LABEL_POSE_BIG5):

if (sameCommandDelay != null && sameCommandDelay.AddSeconds(COMMANDELAYINSECONDS) < DateTime.Now)

{ // avoid too many commands -problem…

sameCommandDelay = DateTime.Now;

InputSimulator.SimulateKeyPress(VirtualKeyCode.LWIN);

}

break;

case (PXCMGesture.Gesture.Label.LABEL_HAND_CIRCLE):

base.Dispose();

//parent.Close();

//Application.ExitThread();

break;

case (PXCMGesture.Gesture.Label.LABEL_POSE_THUMB_UP):

if (sameCommandDelay != null && sameCommandDelay.AddSeconds(COMMANDELAYINSECONDS) < DateTime.Now)

{

sameCommandDelay = DateTime.Now;

VirtualMouse.LeftClick();

}

break;

Depth data, c++ demo from Intel

Sweet and tasty approach to OpenCV and MinnowBoard

PC-esque cheap hardware is booming, and there seems to be no limit to the cool apps you can create on boards like Beaglebone, Rasperry Pi, or Minnowboard.

This obvious trend has had our attention for a long time now, plus we've got some customer cases going with the basic idea of migrating from expensive legacy systems to cheap off-the-shelf processing boards with huge capabilities in a meak form-factor.

Recently, some of our clients have expressed their interest in imaging systems, so we decided to whip up a small demo involving our "Ixonos BSP" small-footprint Linux distro and the industry standard OpenCV imaging library.

In this demo we used the MinnowBoard, Intel's small and low cost board which is based on Atom processor. The camera we used is a basic USB webcam from Logitech. Pictures below:

The Minnowboard with webcam watching candy drops

The camera setup allows the system to see some candy drops in this rather trivial pattern recognition demonstrator. The system acquires image rasters of the scene using v4l2 and OpenCV. Circle shaped patterns are detected using opencv function "HoughCircles", based on Hough Circle Transform. Code snippet below demonstrates simple circle detection using HoughCircles:

//circle detection
vector<vec3f> circles;
HoughCircles(detected_edges, circles, CV_HOUGH_GRADIENT,
             1, minSizeThreshold,  lowThreshold, lowThreshold/2,
             minSizeThreshold, minSizeThreshold + minSizeThreshold / 2);

printf("total circle count: %d\n",  circles.size());

After detecting all the circles, they are categorized according to color and statistics are printed to the screen.

Candy drops detected

  Another picture below illustrates a situation with some more candy drops.

More candy drops detected

Ilkka Aulomaa, SW Engineer - Ixonos
Kalle Lampila, SW Engineer - Ixonos

Ixonos Multi-Window - 2nd generation!

A couple of years ago Ixonos was the first to introduce a comprehensive solution for running multiple applications in multiple windows on Android.

Now, Ixonos engineers have been working tirelessly to make latest Android versions support this feature. The result of this work improves on the original implementation by introducing better performance and some exciting new features.

Ixonos Multi-Window demonstrated on the Nexus 10 tablet running Android 4.2.2.

Among the new features is a really cool "Super Window" mode which puts select applications in a grid layout which can be resized or moved around for easy window navigation.

Stay tuned for more blog entries on this cool technology...

Vasile Popescu, Chief Software Engineer - Ixonos
Mikkel Christensen, Chief Software Engineer - Ixonos
Henrik Kai, Chief Software Engineer - Ixonos

Ixonos sensact library – sensor/actuator communication made easy!

Fresh out of the Ixonos engineering labs comes the embedded HTML5 industrial prototype that Ixonos engineers are working on. The latest version of this prototype samples sensor data from a Texas Instruments multi-function sensor device http://www.ti.com/tool/boostxl-senshub which is connected via USB – this device includes various sensors (thermometer, compass, gyro, pressure, etc.).

To easily access these sensors we have created a library named “libsensact” which abstracts away the communication channel and input/output details of sensor or actuator devices – each device is simply characterized by it's name and the names of sensor/actuator variables that it provides. The communication channel which is abstracted can be any of USB, I2C, UART, Ethernet, EtherCAT, CAN, ProfiNET, etc.. Though, the first version of the library only supports USB.

The library offers simple connection handling functions and get/set functions for retrieving or setting sensor/actuator variable values of various types (char, int, float, etc.).

For example, retrieving the “temperature” and “pressure” values of the multi-function sensor device named “senshub0” is as simple as described by the following application code:

#include <stdio.h>
#include <unistd.h>
#include "sensact.h" 

#define TIMEOUT 100 // ms

int main(void)
{
  int device;
  float temp;
  float pressure;
  int status;

  device = connect("senshub0");

  status = get_float(device, "temperature", &temp, TIMEOUT);
  if (status < 0)
    printf("Error fetching temperature.");
  else
    printf("Sensor Temperature: %f\n", temp);

  status = get_float(device, "pressure", &pressure, TIMEOUT);
  if (status < 0)
    printf("Error fetching pressure.");
  else
    printf("Sensor Pressure: %f\n", pressure);

  disconnect(device);

  return 0;
}

Also, for each sensor/actuator device, the library contains an entry in a device configuration structure list which contains device specific configurations. For example, the “senshub0” device is configured by the following code:

#include "device.h"

/* List of supported sensor/actuator devices */ 

struct device_t device[] = 
{ 
  { .name = "senshub0", 
    .description = "TI Sensor Hub USB device", 
    .connection = USB, 
    .vid = 0x1CBE, 
    .pid = 0x0003, 
    .endpoint = 0x1 }, 
  { } 
}

For different connection types there will be different “.connection” definitions and subsequently different related dot configuration variables.

Future plans for this library are to add support for more connection types and extend the API with subscriber type functionality so that the user can register callback functions which will be called upon value changes for sensor/actuator devices which are able to support event-driven communication.

Watch out for more blog entries on this library as it evolves...

That's it – keep it simple!

Martin Lund, System Architect - Ixonos

HTML5 Canvas GUIs

The HTML5 Canvas is an excellent graphics platform for creating portable graphics. It supports hardware graphics acceleration, where available, and it doesn't even have to be hard to code.

Recently, we've been building GUIs for embedded devices, and with HTML5 being clearly a technology on the ascendancy, we wanted to use web technologies for our embedded demonstrator. Obviously performance was a question, so we decided to work entirely on the HTML5 Canvas element, as used in many graphics-intensive web games.

So how does a HTML5 Canvas approach compare to the more usual approach. Let me show you a short example, where first we create an app using HTML / CSS /JS - the traditional way. The other example is a similar app, built on the GUI library we created on HTML5, in pure JavaScript.

The app simply animates five icons across the screen, repeating infinitely.
The first part of any HTML app is the HTML. Here we declare the image items that we will animate

<body onload="someFunction();"> <img height="64" id="anima" src="png/a.png" width="64" /> <img height="64" id="animb" src="png/b.png" width="64" /> <img height="64" id="animc" src="png/c.png" width="64" /> <img height="64" id="animd" src="png/d.png" width="64" /> <img height="64" id="anime" src="png/e.png" width="64" /> </body>
Then the CSS. We define the animation that will be shared by all img elements

@-webkit-keyframes move
{
   0% { -webkit-transform: translateX(0px); }
   100% { -webkit-transform: translateX(500px); }
}

.anim
{
 -webkit-animation-name: move;
 -webkit-animation-duration: 5s;
 -webkit-animation-timing-function: linear;
 -webkit-animation-iteration-count: infinite;
 -webkit-animation-fill-mode: forwards;
 position: absolute;
}

Finally, the JavaScript part, where we first find the img elements we wish to animate, then assign the style we need.

function someFunction()
{
 items.push( document.getElementById('anima') );
 items.push( document.getElementById('animb') );
 items.push( document.getElementById('animc') );
 items.push( document.getElementById('animd') );
 items.push( document.getElementById('anime') );

 for( var i = 0; i < 5; ++i )
 {
  items[i].className = items[i].className + " anim";
  items[i].style.top = ""+(70 * i)+"px";
  items[i].style.left = "10px";
 }
}

As you can see, this involves three snippets of code. Now, HTML5 Canvas GUI coding doesn't have to be hard. Not when there is a bit background work done. Check this out.

function initApplication()
{
 var btns = [];
 var btn_paths = ["a.png", "b.png", "c.png", "d.png", "e.png" ];

 for( var i = 0; i < btn_paths.length; ++i )
 {
  var btn = new ItemDecorator( 0,i*70, 64, 64, btn_paths[i] );
  addItem( btn );
  var anim = new AnimMove( btn, 500, i*70, 5000 );
  anim.repeatForever();
 }
}

Here we've created the exact same animation using a decorator item in our widget library. As you can see, the amount of code is less than a third of the traditional way. And the code is very easy to read.

That was a quick look into full-canvas web GUIs. Stay tuned for examples and more.

Mikael Laine, SW Specialist - Ixonos

Ixonos Embedded HTML5 Demonstrator

Recently, Ixonos has been working to demonstrate the feasibility of using HTML5 as a graphical user interface (GUI) platform on embedded devices.

A GUI needs to not only look good, but also to work well. A button press and consequent feedback must accurately reflect the internal state, and latency should be kept to a minimum overall. Broken abstractions abound when graphics design and technology implementation exist in separate silos - that's why our designers work hand-in-hand with developers. You cannot create a successful GUI by just thinking of it as a set of screenshots - it's also how it works!

Our demonstrator prototype embodies this principle, and provides an excellent open standards based platform for developing and demonstrating our capabilities as a design and software development "one stop" house.

The prototype runs on the Texas Instruments AM3359 Evaluation Module (EVM) connected to a multi-function sensor device represented by a TI Tiva C Series LaunchPad Evaluation Kit in combination with the TI Sensor Hub Booster Pack. Basically this particular prototype involves an ARM Cortex-A8 processor running at 720MHz and the InvenSense MPU-9150 MEMS chip (gyro, accelerometer, compass) combined with a Cortex-M4 microcontroller with USB connectivity. The purpose is to demonstrate a complete embedded system, from sensor I/O, to middleware and GUI.

The complete software stack consists of the following:

• The Ixonos Embedded Linux BSP (Base Support Package)
• Qt 5.1 framework
• QWebView based web programming environment
• Ixonos HTML5 canvas GUI libraries and web app
• Ixonos data server, providing the web GUI with constant sensor updates via WebSockets
• Ixonos "senact" library which provides an API for handling sensor and actuator communication

In the video below you can see one view from the demonstrator, involving the accelerometer and compass attached to a HTML5 Canvas widget. Enjoy.

One view from our HTML5 Demonstrator Prototype - accelerometer attached to the embedded TI AM3359 board, with an HTML5 Canvas-based compass widget.

From a business perspective, one needs to understand how to harness the relevant aspects of hardware and software for the task at hand. This is where Ixonos comes into play. We’ve created a low-footprint Linux platform, which runs a Webkit-based HTML5 runtime, with a plugin architecture that provides native functionality to portable web apps. Our motto is: "dream design deliver", and we live up to it through our interdisciplinary working mode, where low-level technology implementation meets high-level graphics design vision.

This is the first installment of our HTML5 prototype demonstrating the basics. In the future, more sensors/actuators will be added and our UX designers will add beautiful graphics to make it really shine.

Stay tuned for more blog entries on future versions of this prototype and the technologies behind it...

Martin Lund, System Architect - Ixonos
Mikael Laine, SW Specialist - Ixonos