Upgrade Process

I had my last meeting with my supervisors yesterday. At this stage I have managed to develop enough tools in the interface to be able to present the idea of my research.

The interface is a combination of interaction methods that include haptic and tangible interaction. The technology used in the interface includes computer vision and tabletop interaction based on Web technology such as JavaScript and Web Browser render.

The Upgrade Process

So I have this month to:

  • Define my contribution
  • Define methodology
  • Structure thesis

The thesis should include:

  • Problem / Question
  • Literature Review
  • Methodology
  • Discussion
  • Expected Results

I am currently doing a Web Science research. Web Science is an interdisciplinary research group. We are studying the Web from different perspectives. In the case of my research, I am trying to understand how can Cultural Heritage institutions can be enhance their engagement and pedagogic activities online. For this there is a wide range of disciplines that I have included in my research.

(Inter)Disciplinary approach

Human Computer Interaction

Through this approach I am intending to understand how people interact with the computer. Lets remember that on the Web, all interactions occur through an interface. For this reason it is important to study how these interactions might occur.

Human Information Interaction

Under this scope, I intend to produce a better understanding of how people/users relate, and process information. Usually when interfaces are developed designers focus merely on the interaction process without thinking beyond the tool itself. The HCI community has started adopting this process in their methodology but it has not yet been standardised as a design process in HCI.

Pedagogy / Psychology

Among many areas, I am focusing on Embodied Cognition. This thesis presents the idea that the mind is determined by the human body. This is to say that mental processes are not bounded to the mind itself. The body itself is a channel in which the human mind interacts and reasons about the world. This process is also highly linked to Constructivism, where people learn by experience. Constructivist learning experiences have commonly used cognitive processes as a pathway to empower learners. Even though constructivism is not limited to embodiment of activities, it might provide a positive pathway for pedagogic activities.

Enhancing Engagement with Online Museums

Museums as part of the Web are in need of producing meaningful pedagogic activities. The pedagogic element is essential to Cultural Heritage institutions. That is one of their main reasons of funding and even though organisations that are not in the education business can also benefit from such pedagogic activities. Nevertheless, this challenge is not easy to solve. For people to produce knowledge by themselves might not be as straightforward as one might think. There is a wide combination of how people might approach information spaces such as websites, databases, books, etc in order for them to extract knowledge.

By using the aforementioned disciplines in an interdisciplinary manner, I am attempting to produce an tangible user interface where people can be able to ask questions to a linked data system populated with cultural heritage data.



June 17 – Visualising // Managing Results

Visualising and Dealing with the Results.

It has been about three weeks since my last post. This is because I have been focusing more on specialising in using JavaScript and more specifically JQuery. JQuery has the potential of making it easier to exploit JavaScript on Web content.

Updates on the Interface

Displaying Results

When users make queries, they will be retrieving vast sets of information. This will prove to be a complex task. For this reason, I need to find a way on how they can be able to sort such content. One of the ways that I have envisioned is by providing an area where results can be sorted and then manipulated by assigning them to a fiducial item. Users should be able to offload that information thus simplifying the process.


The drop box will serve the function of holding the result that the user wants to select. When he or she puts it back into the original results panel, the item will be highlighted in a different colour, otherwise the user can clear the result and remove the item from the interface. By highlighting the returned item, users can be able to identify the ones that they found particularly relevant.


The returned element has a heart icon that shows that the element was kept from the drop box. The box on the other hand just contains a lightbulb showing that it is highlighted. Moreover, I have also added clear buttons so users can be able to remove unselected elements from the specific panel.

I should be able to create a fiducial  with a screen, or perhaps a frame where the visual can be contained. When the user drags the frame, the drop box moves along with it. The physical object should allow users to literally grasp specific datasets so they can be compared or used for further queries.

dropUIfidJQuery and JQuery UI

There is a huge potential that JQuery has to offer to these type of interfaces. The challenge will be on transforming common graphic user interface tools into the physical tools. My previous post indicates some of the many possibilities of making the queries and navigating once the result has been made. This drop box should be able to help the querying process along with the data navigation.

Here is a video of the table top system still using TUIO as the interactivity connection.

Tangible User Interface for Europeana’s API

I have been working with TUIO for the main interface in between Europeana’s API and a user. One of my main research questions is how to can users make questions through tangible interfaces. In this case, users should be able to make questions about cultural heritage content from different organisations. In this case users can be to access data from over 30 million metadata records that include books, photos, art, audio and artefacts among others. On one hand having access to all this content may benefit users since they will have vast sets of information to answer their questions. On the other they might get lost with all the data that is available for them.

When working with vast sets of information, users can benefit by dissecting specific items of the information that they are looking for. Nevertheless, this process might prove difficult might require a lot of concentration. By offloading this mental process onto physical objects, users might pace their thinking process and assist themselves by using the physical objects as an aid.

My intention is to produce such objects that can help users solving questions and finding information from a data portal such as Europeana. For this I have created a starting skeleton of essential objects that can then be transformed to queries on the API.

Essential Queries

Who – What

Some of the most essential queries might include who or what are looking for. These will be represented individually.

who and what icons

query = who:"string"
query = what:"string"



By positioning the fiducials on top of a map, users can be able to add geoCoordinates on to the query. This can be used by adding two values either by tapping or by using two fiducials.


qf=pl_wgs84_pos_lat:3A%5B01+TO+02%5D & qf=pl_wgs84_pos_long:3A%5B03+TO+04%5D


Countries are different to geoLocation. Countries are defined by name that are part of a human perception and not a geographic one. This way this fiducial will detect it’s x and y position and add the name of the closest place to the query. There will be several of these fiducials for adding more places to the query.


query = COUNTRY:string


Another value that can be added to the query are the dates for the time period that you want to constrain the data to. Time being abstract, its very difficult to represent and manipulate. Europeana provides a starting year and an ending year. For this reason this will be using two ‘dials’ to forward or rewind starting or ending year.


query = YEAR: 0000 TO 0000

Boolean Operations

Basic boolean operations can be added to the query to produce a more specific question. Operations such as AND, NOT and OR can be used. This fiducial has to be used in addition of the fiducials previously mentioned otherwise there will be no values to compare to.

This marker commonly will require one of the three options. Nevertheless, users might benefit by looking at all the options to analyse what they are asking.


query  = where:(Paris NOT France)


To put everything together, users might benefit of structuring their thoughts on a template where they can finalise their sentence or question. This does not mean that they must place artefacts in such order but is merely a starting point to give an approach of how they might structure questions.


There is still a wide variety of query options that can be added, but this is the essential information that users will require to input at a specific time in order to retrieve meaningful results.

Google Maps – TUIO.Js – Europeana Linked Data

Query and Display on Timeline Google Maps

The previous versions of the prototype managed to get the data into the query form so it could be retrieved from the Europeana database.

With the help of William Fyson the data is now displayed using the latitude and longitude (edmPlaceLatitude) metadata. By using the Timeline produced by SIMILE, those geolocations can be related to specific date  also provided in the metadata.

Using TUIO and Google Maps Api V3

Once the data is displayed, TUIO can be used not only to query but to manipulate the map. In this example, one of the fiducials is used to control the zoom of the map.

Google Maps API

Google Maps API


The Google Maps API allows users to populate data on maps powered by Google. Google provides an API where a wide variety of settings and technology implementations can be used.

This certainly brings a huge potential when using museum and archaeology data. For example Europeana provides latitude and longitude coordinates as part of their data model. In order for me to use this, I started playing with some visualisation tools in gMaps.

Styling the Map

The first thing I had to learn was to apply styles to the map. This is quite straightforward, especially using a styler assistant such as the Google Maps Style Wizard. This will generate the JSon  so you can just apply it on your map. Moreover, there are repositories where the community deposits their own designs and colour schemes so you can just implement them on your map. Snazzy Maps is a good place to start.

Generate Icons

I also had to learn how to place markers and icons on the map. For this, instead of using the classic gMaps marker, I generated my own set of 8bit icons. I used Illustrator and exported them as .png format.


Use Some GeoData

Finally I started using some geoData and placing the markers in the map. For this I used a map that contained information about textile designers of indigenous Mayan communities from Chiapas. I created arrays of info and coordinates according to their ethnic group.


Each one of the arrays contain each ethnic group. The rest was just repeating the same process until I had them all organised into colour-groups.

** Check the map in my website **

Next step

The next step is to “auto-populate” the gMaps coordinates automatically from the Europeana database.

March 24 – TUIO Js // Fill and put form data

TUIO//Js Put and send data in a form

Playing a little bit more with TUIO.Js I have now started to place some information in a input text form on HTML. The information is sent with all the marker number values and indicates if its higher than 10 or lower than 11. That information is placed on the input field which then can be used to populate the final form. There are two options: Add to existing text and Replace existing text. This method can be used to assist users with the creation of statements and queries.

Moreover, I have also used the values to place either a vase or the painting accordingly. Using the image placement on a canvas can help visualising the query results on the whole table-top system.