Tangible Interaction with Sifteo Cubes.

 Dynamic Fiducial Pyfos (DFPs)

During the last weeks I have started developing what I would like to call DFPs short for dynamic-fiducial-pyfos. With the help of some friends I have now a basic skeleton to extend my interactive experiments. Here is a video of the result:

 

Tangible Interaction and Pyfos

After I submitted my upgrade draft, I realised that I was going to encounter some issues when working with pyfos when using them as part of the fiducials for the TUIO system. Since users have to combine different concepts (e.g. Roman + pottery or painter + 1800) this will result in a numerous amount of pyfos. The interface already has several objects that can not be removed since they are part of the basic interactions such as: map navigation, box dragging, etc… Therefore, I decided to explore a little bit further. I need to find a way to extend the capabilities of pyfos.

Pyfos have three main states: token, constraint and token+constraint. The TAC (Token and Constraints) paradigm in Tangible User Interfaces (TUIs) offers a set of constructs of how these objects react. Nevertheless, it can be argued that technology can offer pyfos that can self adapt or expand the constraints that bound them. This due to the fact that physical objects such as pyfos cannot morph or change according to user needs.

TUIO Javs

Other researchers are exploring how these TAC approaches can be expanded. It is this search that directed me to explore with mini-displays and sensors. Since I had worked with some Internet of Things and Arduino, I thought of designing some display that detected two different RFIDs to make the combinations and display a specific result. That result display presents the final combination in a form of fiducial so the TUIO interface can detect it. This way, users can pre-design a concept combination thus integrating it in a final dynamic-fiducial-pyfo (DFP) that encompasses that prior combination. Most importantly, DFPs can also produce other display results on the table-top without clustering tools or options.

02

Producing DFPs

There are some alternatives to produce DFPs out there. Many of them require to be built from scratch, but there are some products that can be adapted to our needs without having to fiddle or hack any of the electronics. Although there is a wide variety of tools out there, these are the most ‘approachable’ tools that I encountered.

Educubes

Since I had worked with Arduino and Internet of Things before, I thought of building a mini-display with either RFID or any other type of sensor. I found the Educubes project that presented a good opportunity to start developing for this idea. This might also prove to be beneficial since there are some TFT mini-displays that support touchscreen actions as well.

Educubes by Adafruit
Educubes by Adafruit

 Although this presented a good opportunity to develop, I needed to start producing tools that could work straight away instead of focusing on the electronics. Moreover I thought that the size of the electronics is still quite big for them to be used on the prototype. But it was mainly the issue of working with a wide range of electronics and hacking them so they can do what I wanted.

Sifteo Cubes

After searching I encountered Sifteo Cubes. These cubes already provide a very nice presentation that encompasses a wide range of electronics and a mini-display. Moreover these cubes can be programmed through an SDK provided by the same company. I decided to jump ahead and ordered a second generation Sifteo Cubes.

My surprise was at the moment of using the SDK. I was not the first user to be put down by its complexity. The Sifteo SDK works with C++ with other command tools to run installations and device management. Moreover, through the forums I encountered that the released SDK contained some bugs, which made some of the tutorials not to work.

Nevertheless, I encountered some compiled SDK in GitHub such as Investio and Sifteo Blickets. They provided me with some hints into how to actually start using and managing the cubes. I still had to learn how to program what I needed. So I started first learning how to make interactions. There is a base of few interactions that are supported by the sensors in the cube that include: tilt, pair, shake and press. 

Sifteo Actions
Sifteo Actions

Although it seems quite nice in pictures, the task was not so simple. Since I do not come from a programming background, working with C++ was a huge challenge. First I did a tutorial on neighbouring. Here is the video:

A few days after, I started working with other actions such as tilt and press. It is relevant to mention that I worked using the examples that came with the SDK so, the interactions were pretty much pre-designed and I was just learning a few basic commands that might be used. Here is a video of this stage.

The problem started when working with my specific requirements. I needed an array of options per cube that could be combined between them. This so the final combinations could be applied in the Europeana TUIO system. Using C++ this was not so straightforward. In a nutshell, this is what I needed to create:

Basic combination skeleton
Basic combination skeleton

It took me a lot of time and effort to find a way to program this interaction. I could program something like this with other languages but not with C++. Therefore I asked for some help to develop this. Kevin Lesur from One Life Remains gave me a hand with this. So this is was the basic skeleton built for the interactions:

Operations and interactions skeleton.
Operations and interactions skeleton.

Two cubes are required to make the combination through neighbouring. When they are combined a third cube presents its combination that will eventually show a fiducial making it a DFP. To navigate between the cubes options, users can tilt the cubes in either direction.

This way I am hoping to now carry on and go back to the TUIO experiments and see how these DFPs work on the tabletop system.

 

Tangible User Interface for Europeana’s API

I have been working with TUIO for the main interface in between Europeana’s API and a user. One of my main research questions is how to can users make questions through tangible interfaces. In this case, users should be able to make questions about cultural heritage content from different organisations. In this case users can be to access data from over 30 million metadata records that include books, photos, art, audio and artefacts among others. On one hand having access to all this content may benefit users since they will have vast sets of information to answer their questions. On the other they might get lost with all the data that is available for them.

When working with vast sets of information, users can benefit by dissecting specific items of the information that they are looking for. Nevertheless, this process might prove difficult might require a lot of concentration. By offloading this mental process onto physical objects, users might pace their thinking process and assist themselves by using the physical objects as an aid.

My intention is to produce such objects that can help users solving questions and finding information from a data portal such as Europeana. For this I have created a starting skeleton of essential objects that can then be transformed to queries on the API.

Essential Queries

Who – What

Some of the most essential queries might include who or what are looking for. These will be represented individually.

who and what icons

query = who:"string"
query = what:"string"

Places

Geolocation

By positioning the fiducials on top of a map, users can be able to add geoCoordinates on to the query. This can be used by adding two values either by tapping or by using two fiducials.

pos

qf=pl_wgs84_pos_lat:3A%5B01+TO+02%5D & qf=pl_wgs84_pos_long:3A%5B03+TO+04%5D

Country

Countries are different to geoLocation. Countries are defined by name that are part of a human perception and not a geographic one. This way this fiducial will detect it’s x and y position and add the name of the closest place to the query. There will be several of these fiducials for adding more places to the query.

country

query = COUNTRY:string

Time

Another value that can be added to the query are the dates for the time period that you want to constrain the data to. Time being abstract, its very difficult to represent and manipulate. Europeana provides a starting year and an ending year. For this reason this will be using two ‘dials’ to forward or rewind starting or ending year.

year

query = YEAR: 0000 TO 0000

Boolean Operations

Basic boolean operations can be added to the query to produce a more specific question. Operations such as AND, NOT and OR can be used. This fiducial has to be used in addition of the fiducials previously mentioned otherwise there will be no values to compare to.

This marker commonly will require one of the three options. Nevertheless, users might benefit by looking at all the options to analyse what they are asking.

boolean

query  = where:(Paris NOT France)

Container

To put everything together, users might benefit of structuring their thoughts on a template where they can finalise their sentence or question. This does not mean that they must place artefacts in such order but is merely a starting point to give an approach of how they might structure questions.

console

There is still a wide variety of query options that can be added, but this is the essential information that users will require to input at a specific time in order to retrieve meaningful results.

From Emotion to Technology

From Emotion to Technology and Vice Versa

Engaging with museum content nowadays usually involves some sort of technological process. This does not mean that all exhibits must contain this technological element. When visiting the museum, there are six categories of experiences that happen when visiting a museumš:

  • Entertainment
  • Sociability
  • Learning
  • Aesthetic
  • Commemorative
  • Joy

Through my research, I am in the process of finding how the Web is capable of providing such experiences. The online museum should be able to provide the same or better experience as a traditional museum. The Web as a Social Machine can provide the tools for promoting and developing content for the Cultural Heritage sector. I will argue that areas such as Human Computer Interaction within Computer Science, are at the vanguard of technological innovation and it is until the developer community is contacted or until they develop for museum content that such technology is implemented. Many of these technologies are now being developed with different Open Source or Creative Commons licenses that many museum curators and managers can implement to promote their collections.

As an initial stage I started co-relating the experiences with the technologies I believe can enhance such emotions/experiences. This table of course will evolve through time and with the technologies available. I believe this is a good start to define the different technologies already implemented by museums and developers.

References

1. Kotler, N. and P. Kotler (2001). Estrategia y Marketing de Museos. San Francisco, Jossey-Bass Inc.

The Internet of Cultural Things [Web Science Research]

The Internet of Cultural Things [Web Science Research]

 

Research proposal (Abstract)

 

poster of internet of things
Research proposal poster

How can the ‘Internet of Things’ promote critical thinking in Cultural Heritage?

 

Museums are spaces where physical objects are presented in order for to teach about a specific event in history. Curators are the specialists in charge of learning all the facts about those objects and narrate a story through displays and exhibits. Every story is then contrasted with different tangent stories, depending on the perspective from where is told. As a result, there will be a debate and critique towards the most appropriate story to tell about such artefacts. For this, Critical Design can be utilised.
Critical Design has been in charge of opening dialogue and debate in different communities. It utilises everyday objects and utilises them to promote critical thinking. By embedding critical design through objects, dialogue between an audience and the cultural heritage sector can be promoted. By exploring different affordances of objects that can promote Critical Design dialogue, interdisciplinary connections can be created.
The Web has opened access to information. By embedding Web distributed information on objects, a new ‘industry’ or thinking current is being created. The ‘Internet of Things’, where everything connects and everything interacts. This concept will open a new line of services and products where the cultural heritage sector can benefit from. This research focuses on the impact of these thinking theories and disciplines and how can they be utilised to promote critical thinking in cultural heritage and their display in a public space such as a museum.

 

Contemporary Cultural Theory in Museum Studies

 

When dealing with cultural heritage we encounter representations of culture and society through tangible and intangible objects. These can include buildings, artefacts, texts and even landscapes, which are curated and preserved for further generations.
In the process of preservation of cultural ‘objects’, different cultural heritage groups intend to capture information that can be able to provide a context in history. Society became ‘object-centred’ by embedding value to material culture. In many cases, these values represent culture that portray national identity. Kreps (2003: 48) mentions that there are several biases when identity is constructed through cultural heritage groups encompassed under a specific ‘umbrella’; especially under a western colonialist umbrella that dominated the construction of historic documentation in specific contexts. Nevertheless, this is not the first time that society uses exhibited artefacts to produce national identity. Aztec culture presented Teotihuacan’s artefacts among other cultural artefacts to promote their national pride. This certainly puts the concept of museums being a western modern invention under scrutiny. By expanding the perspective of artefact curatorial processes, we can be able to understand better their relation in history. The same way that questioning the origin of specific institutions such as a modern western museum, many ideologies or historic events can be placed under scrutiny. James Deetz (1977: 7) indicated how material culture, has been shaped by man under a specific context, who was also shaped by his own culture. The context of the object that made it valuable sometimes becomes more important than the object itself. Meltzer (1981) mentions the Apollo 17 mission’s moon rock. The moon rock became important due to the fact that it came from the moon. Outside of this context, the ‘rock’ is just a piece of basaltic volcanic rock of around 4 million years old. In our planet we certainly have older rocks that can double its age. So why does everybody interacts with it? The ‘rock’ has been hand picked and displayed; as many objects in museums, they become important objects under the “world of human values” (Pearce, 2001: 10). Picking up from this example, it is evidenced how history is presented through material culture. For this reason, curators select specific objects so they can be able to narrate stories that audiences can ‘pick up’ according to their specific ‘cultural values’ thus appropriating that segment of history.

 

Critical Design and Contemporary Cultural Theory

 

As previously mentioned, artefacts are curated to narrate specific stories. The story presented, envisions a reality from either an individual or a group [e.g. museum, govt., cultural group]. For this reason there will be alternative stories to be told.
Contemporary cultural theory allows and interdisciplinary approach to study everyday life. Culture is becoming more accessible to different socio-economic groups especially in the Web. For this reason art and culture no longer ‘belongs’ to the educated or rich classes. Macdonald (2011) mentions how museums are starting to utilise this theoretical current. By allowing ‘alternative stories’ museums can be able to empower visitors and construct a more meaningful narrative. This becomes then, a way to criticise and communicate our own storyline.
In this case, Critical Design provides a channel where this dialogue can happen. It uses design to produce thinking and questioning from an audience. This does not mean that Critical Design is against everything, it “is simply a useful way of making this activity more visible and subject to discussion and debate” (Dunne and Raby, 2012). Anthony Dunne pinned down the term Critical Design in 1999; nevertheless, many other previous design currents can be attributed to it. Using design to question and critique an idea is nothing new (Drury and Farhoomand, 1999). Visual communication methods have been used before medieval times to convey messages. When the press was invented, these visual messages were easier to reproduce therefore, making them more accessible to the ‘masses’. For this reason, many cultures have been using design to raise awareness regarding specific topics. The Cuban poster dates back to the 1960s; they are famous for raising awareness within their population [and the world], focusing on Cuban international politics among other topics. In Mexico, muralists utilised art to convey their messages and criticise capitalism and religion among other topics. It was during the 1920s all the way to the 1950s that Mexican muralists succeeded re-shaping national identity thus promoting indigenous heritage. Such as these, many other examples can be raised to illustrate how design and art has been applied for questioning ideas. I will argue that Critical Design might me considered as the evolution of a ‘judgemental fine art’. Critical Design needs to be embedded with objects that are part of our everyday life (Dunne and Raby, 2012). By blending methods and techniques from art with a contemporary cultural theory, Critical Design can be able to produce aesthetic objects that make people think.

 

The Internet of Things and Critical Design

 

Critical Design and Contemporary Cultural Theory can open pathways for debating narratives presented by museums or cultural heritage organisations. As mentioned before, this is not to argue against everything but to open a channel for dialogue and critical thinking. Modern technology and social machines such as the Web can provide new and more powerful tools for debate. The objects utilised for Critical Design have the opportunity to be embedded with ‘visualised data’ to promote the dialogue. On the other hand human computer interaction [HCI] can also provide the transition between data, object and audience. The idea of embedding everyday objects with data and digital functionality, has recently been referred as an area of study called the Internet of Things [IoT](Xia et al., 2012). For now, the vision under the IoT encompasses the development of necessary technologies to support the connectivity in between artefacts. On the other hand, it can also study new industry opportunities produced as a result of all the new applications and services created and the new network of objects interconnected between each other (Miorandi et al., 2012). It has been argued how the ‘things’ or ‘smart objects’ under the IoT are based under their capacity to be identified by anything, to communicate and interact (Miorandi et al., 2012). All objects are interconnected to each other, therefore producing and consuming information. This is where the exchange of information can be able to promote the dialogue and new narratives for cultural heritage under the IoT.
Many countries are now adopting the concept of IoT. In Europe the “Internet of Things- An Action Plan” was produced to envision IoT as technology. In the US, the National Intelligence Council identified it as one of the ‘key technologies’ for the future. It has even been identified as a new industry in countries such as China (Ning and Hu, 2012).
An interdisciplinary approach toward these topics can be able to envision the pathway of the IoT and the future of the Web. Certainly it will open many dialogues in between society and technology. The ubiquitous quality of the IoT needs to add a socio-cultural field to the area of study, an area where objects and dialogue happen.

 

 

References

DEETZ, J. 1977. In small things forgotten : the archaeology of early American life, Garden City, N.Y, Anchor Press/Doubleday. DRURY, D. H. & FARHOOMAND, A. 1999. Information technology push pull reactions. Journal of Systems and Software, 47, 3-10.
DUNNE & RABY. 2012. Critical Desing FAQ [Online]. London. Available: http://www.dunneandraby.co.uk/content/bydandr/13/0 [Accessed 20 November 2012].
KREPS, C. F. 2003. Liberating culture : cross-cultural perspectives on museums, curation, and heritage preservation, London, Routledge.
MACDONALD, S. 2011. A companion to museum studies, Malden, MA ;
Oxford, Wiley-Blackwell.
MELTZER, D. J. 1981. Ideology and material culture. In: GOULD, R. A. & SCHIFFER, M. B. (eds.) Modern Material Culture: The Archaeology of Us. New York: Academic Press.
MIORANDI, D., SICARI, S., DE PELLEGRINI, F. & CHLAMTAC, I. 2012. Internet of things: Vision, applications and research challenges. Ad Hoc Networks, 10, 1497-1516.
NING, H. S. & HU, S. 2012. Technology classification, industry, and education for Future Internet of Things. International Journal of Communication Systems, 25, 1230-1241.
PEARCE, S. 2001. Interpreting Objects and Collections, Andover, Routledge, 2001.
XIA, F., YANG, L. T., WANG, L. & VINEL, A. 2012. Internet of Things. International Journal of Communication Systems, 25, 1101-1102.