Tangible Interaction with Sifteo Cubes.

 Dynamic Fiducial Pyfos (DFPs)

During the last weeks I have started developing what I would like to call DFPs short for dynamic-fiducial-pyfos. With the help of some friends I have now a basic skeleton to extend my interactive experiments. Here is a video of the result:


Tangible Interaction and Pyfos

After I submitted my upgrade draft, I realised that I was going to encounter some issues when working with pyfos when using them as part of the fiducials for the TUIO system. Since users have to combine different concepts (e.g. Roman + pottery or painter + 1800) this will result in a numerous amount of pyfos. The interface already has several objects that can not be removed since they are part of the basic interactions such as: map navigation, box dragging, etc… Therefore, I decided to explore a little bit further. I need to find a way to extend the capabilities of pyfos.

Pyfos have three main states: token, constraint and token+constraint. The TAC (Token and Constraints) paradigm in Tangible User Interfaces (TUIs) offers a set of constructs of how these objects react. Nevertheless, it can be argued that technology can offer pyfos that can self adapt or expand the constraints that bound them. This due to the fact that physical objects such as pyfos cannot morph or change according to user needs.


Other researchers are exploring how these TAC approaches can be expanded. It is this search that directed me to explore with mini-displays and sensors. Since I had worked with some Internet of Things and Arduino, I thought of designing some display that detected two different RFIDs to make the combinations and display a specific result. That result display presents the final combination in a form of fiducial so the TUIO interface can detect it. This way, users can pre-design a concept combination thus integrating it in a final dynamic-fiducial-pyfo (DFP) that encompasses that prior combination. Most importantly, DFPs can also produce other display results on the table-top without clustering tools or options.


Producing DFPs

There are some alternatives to produce DFPs out there. Many of them require to be built from scratch, but there are some products that can be adapted to our needs without having to fiddle or hack any of the electronics. Although there is a wide variety of tools out there, these are the most ‘approachable’ tools that I encountered.


Since I had worked with Arduino and Internet of Things before, I thought of building a mini-display with either RFID or any other type of sensor. I found the Educubes project that presented a good opportunity to start developing for this idea. This might also prove to be beneficial since there are some TFT mini-displays that support touchscreen actions as well.

Educubes by Adafruit
Educubes by Adafruit

 Although this presented a good opportunity to develop, I needed to start producing tools that could work straight away instead of focusing on the electronics. Moreover I thought that the size of the electronics is still quite big for them to be used on the prototype. But it was mainly the issue of working with a wide range of electronics and hacking them so they can do what I wanted.

Sifteo Cubes

After searching I encountered Sifteo Cubes. These cubes already provide a very nice presentation that encompasses a wide range of electronics and a mini-display. Moreover these cubes can be programmed through an SDK provided by the same company. I decided to jump ahead and ordered a second generation Sifteo Cubes.

My surprise was at the moment of using the SDK. I was not the first user to be put down by its complexity. The Sifteo SDK works with C++ with other command tools to run installations and device management. Moreover, through the forums I encountered that the released SDK contained some bugs, which made some of the tutorials not to work.

Nevertheless, I encountered some compiled SDK in GitHub such as Investio and Sifteo Blickets. They provided me with some hints into how to actually start using and managing the cubes. I still had to learn how to program what I needed. So I started first learning how to make interactions. There is a base of few interactions that are supported by the sensors in the cube that include: tilt, pair, shake and press. 

Sifteo Actions
Sifteo Actions

Although it seems quite nice in pictures, the task was not so simple. Since I do not come from a programming background, working with C++ was a huge challenge. First I did a tutorial on neighbouring. Here is the video:

A few days after, I started working with other actions such as tilt and press. It is relevant to mention that I worked using the examples that came with the SDK so, the interactions were pretty much pre-designed and I was just learning a few basic commands that might be used. Here is a video of this stage.

The problem started when working with my specific requirements. I needed an array of options per cube that could be combined between them. This so the final combinations could be applied in the Europeana TUIO system. Using C++ this was not so straightforward. In a nutshell, this is what I needed to create:

Basic combination skeleton
Basic combination skeleton

It took me a lot of time and effort to find a way to program this interaction. I could program something like this with other languages but not with C++. Therefore I asked for some help to develop this. Kevin Lesur from One Life Remains gave me a hand with this. So this is was the basic skeleton built for the interactions:

Operations and interactions skeleton.
Operations and interactions skeleton.

Two cubes are required to make the combination through neighbouring. When they are combined a third cube presents its combination that will eventually show a fiducial making it a DFP. To navigate between the cubes options, users can tilt the cubes in either direction.

This way I am hoping to now carry on and go back to the TUIO experiments and see how these DFPs work on the tabletop system.



The God Collector

The God Collector. A tangible user interface for children.

Different cultures have embodied their fears of the unknown into supernatural beings. As a result different cultures have devoted themselves to worship these depictions of the unknown. Through time and space these elements tend to repeat themselves with different names and iconography. By understanding how different cultures have represented these elements, we can be able to understand better our own culture.

Based on these representations, there is an immense legacy of material culture distributed through various museums around the world. By understanding these representations, we can get familiarised with the various archaeological artefacts presented in museums.

I will start then, by presenting a list of different cultures and their respective depiction of such elements.

This is a starting point for selecting different characters for the didactic interface to be implemented.

TUI 01. Dice Version

God Collector die version

For this version the TUI utilises a two dice, one represents the elements and the second the culture. The uses can be able to mix both together and retrieve the desired god. The interface can display on a screen a map where the god will be visualised on top of the country where the culture belongs to. Once the character is visualised, the user can be able to print the PaperCraft model of the character.

TUI 02. Token Version

TUI token version

This version of the interface, the user can be able to explore the various elements/representations, by using tokens. These tokens can be placen on a physical map. This map may by also presented as a table-top system where the tokens can react. This may affect portability and acces so further research is required. Once the token is placed on the desired country, the interface reacts and displays on a secondary screen (unless using a table-top system). For this experiment, only the three selected countries will be highlighted. The displayed god then provides the opportunity to print the PaperCraft model of the character.

TUI 03. The Disc

Disc spinner for the God Collector

This interface can be presented along with a display where the results will be visualised. The disc will enable to see the markers that will make the interface react. The different rings will present the various elements required to display the desired character. One ring will represent the element or representation, another the culture and a third tentative ring can represent the gender of the character. On the center another AR marker can be utilised to display a 3D model of the result. Once the desired display is rendered, the user can be able to print the PaperCraft model.

Further implementation

There is still a post-print process where the user can be able to start organising the various built PaperCraft model. The template containing the model will be able to provide extra information like:

  • Name
  • Culture
  • Representation
  • Museum where related artefacts can be found
  • Relation to other gods or cultures


This entry focuses mainly on the proposed TUI/HCI methods to interact with the interface.