Learning with TUIs

A cube to learn: a tangible user interface for the design of a learning appliance.

Terrenghi, Lucia, Matthias Kranz, Paul Holleis, and Albrecht Schmidt. 2006. “A cube to learn: a tangible user interface for the design of a learning appliance.” Personal and Ubiquitous Computing no. 10 (2-3):153-158. doi: 10.1007/s00779-005-0025-8.

This paper describes work developed for a cube with six displays where information is displayed. It is combination of hardware and software. By the use of multiple-choice test system this application can be used for several tests or games. In this case it was used for a language translation test where on the top face the original word appears and on the remaining 5 faces there are multiple choices where one is the correct answer. When facing the correct side (answer) and shaking the cube, it provides the result.

The team argues that “breaking conventions of how a computer looks like and providing children a playful interface” it will be easier to introduce these type of interfaces into an everyday activity. Children do not see an apparatus like this like a traditional learning tool but rather as a toy.


Applying tangible story avatars to enhance children’s collaborative storytelling.

Liu, Chen-Chung, Kuo-Ping Liu, Pi-Hui Wang, Gwo-Dong Chen, and Mu-Chun Su. 2012. “Applying tangible story avatars to enhance children’s collaborative storytelling.” British Journal of Educational Technology no. 43 (1):39-51. doi: 10.1111/j.1467-8535.2010.01146.x.

This project describes the use of avatars to facilitate collaborative storytelling with children. In this case the avatars are physical marionettes.

They describe related works that include:

• Virtual storytelling environments.

StoryBuilder and FaTe2 where the use of the Web allows to share their stories with other users.

• Software avatars.

It is necessary to equip avatars with an intuitive manipulation interface that can facilitate behaviours and enhance social interaction to improve interpersonal communication skills.

• Tangible Storytelling Avatars (TSA)

TellTale, Storymat and Dolltalk use voice recording functions to assist children in the creation and sharing of their stories.

With TUIs it is important to analyse how is the social interaction going to take place to be able to provide an environment where they can help each other.

Storytelling systems: constructing the innerface of the interface

Umaschi, M., and J. Cassell. 1997. “Storytelling systems: constructing the innerface of the interface.” In Proceedings Second International Conference on Cognitive Technology Humanizing the Information Age, edited by J. P. Marsh, C. L. Nehaniv and B. Gorayska, 98-108108. IEEE Comput. Soc.

The researchers identify the necessity of computational tools that are specifically designed to encourage exploration and communication issues.

By building things, learners are capable of producing more and more different type of ideas.

This project SAGE, uses personal connecitons, where children program their interactive storyteller according to their own culture and interests. SAGE encourages constructivist learning.

Their embodied objects are cover stuffed animals with computational affordances. Psychologist Winnicot refers to this as a transitional object where it becomes the first object “not me” for children.


The CTI framework: informing the design of tangible systems for children

Antle, Alissa. 2007. The CTI framework: informing the design of tangible systems for children. Paper read at Proceedings of the 1st international conference on Tangible and embedded interaction, at Baton Rouge, Louisiana.

There are five themes analysed in this paper.

  1. Space for Action.
    1. Tangible interfaces must use space.
      • Design Concepts: Body-based interaction and control. Age appropriate repertoire of physical actions. Existing performative knowledge.
    2. Pragmatic actions involve manipulating physical entities to directly accomplish a task. Lego and Tetris provide understanding of how connecting systems work.
  • • Design Concepts: Epistemic actions and offloading difficult mental processes.
  1. Perceptual Mappings
    1. Perceptual Affordances

Ways that the children will activate the interface or actions required to implement

  • • Design Concepts: Perceptual affordances and Designed affordances.
  1. Scales of Experience and Representation

Relating real world objects and their scales in their interpretation of objects may be an issue.

  • • Design Concepts: Child-body scale as reference
  1. Behavioural Mapping
    1. Cognitive Mode Switching

The process of experimenting with concepts and structures through the interface relates to children’s metacognitive strategies. Their ability to self-regulate and reflect.

  • • Design Concepts: Experiental vs. reflective modes of cognition. Ready-to-hand vs. present-at-hand.
  1. Cause and Effect

Children as young as three can rely on temporal order when identifying causes. Several variables may influence simple casual judgement including: the degree of similarity between causes and effects and the inhibitory nature of the causal relations.

  • • Design Concepts: Temporal precedence. Covariation. Temporal and spatial contiguity.
  1. Semantic Mappings
    1. The Problem with Multiple Representations

Children under sever may have difficulty relating physical manipulatives to other forms of representation across context.

  • • Design Concepts: Reveal representational mappings. Exploration of relationships between entities and representations.
  1. Mapping between Representations

It is necessary to relate abstract ideas to concrete ideas. As children grow older, their spatial cognition changes. This improves their understanding of their own space and position.

  • • Design Concepts: Reciprocal mappings between physical and mental representation
  1. Grounding Understanding in Body-based Schema

Their process of solving kinetic problems abstractly and concretely.

  • • Design Concepts: Leverage children’s understandings of bodily-based concepts to help them understand abstract concepts.
  1. Grounding Understanding in Spatial Schema

Children use rich spatial schemata as foundation for the development of other concrete and abstract schemata.

  • • Design Concepts: Leverage children’s understanding of concrete spatial schemata to help them understand abstract concepts.
  1. Space for Friends
    1. Collaboration

Most GUIs are constrained to a single user interaction. TUIs can allow collaboration from several users.

  • • Design Concepts: Interaction supports rather than requires collaboration. Multiple input units or modes. Protocol for changing mode or transfer of control.
  1. Imitation and Intentional Affordances

Imitating actions from a guide or an authoritative figure can allow children to understand how to use tools.

  • • Design Concepts: Clues to intentional affordances. Visual access to performative actions. Turn-taking of physical or spatial controls.

Magic Story Cube: an interactive tangible interface for storytelling

Zhou, Zhiying, Adrian David Cheok, JiunHorng Pan, and Yu Li. 2004. Magic Story Cube: an interactive tangible interface for storytelling. ACE 2004, 2004, at Singapore, Singapore.

This is a project where a multi-faced cube is utilised to narrate stories from the Bible. By combining pre-defined sequences on to each face of the cube, the storytelling process takes place. This may be handy when required to tell a story before some activity still keeping the game factor.

They argue that presenting the narrative this way appears to be more attractive and understandable to children.

Children’s Storytelling and Programming with Robotic Characters

Ryokai, K., M. J. Lee, J. M. Breitbart, and Acm. 2009. “Children’s Storytelling and Programming with Robotic Characters.” C & C 09: Proceedings of the 2009 Acm Sigchi Conference on Creativity and Cognition:19-28.

This project combines GUI with TUIs. By utilising a robot, in this case a dinosaur, the children can relate more to the activities provided. They manage to translate into the mind of the dinosaur which then starts some actions. Enriched drawings are programmed to control this robotic character.

This way events can be shared with others and they explore possible outcomes of different events and negotiate them with other users. Children manage to create abstract concepts and create meaning between these ideas.

By combining several sensory elements in the interface, there is no exclusion on a medium over another. This way every children can use different engagements to represent their ideas.

There are some robotic elements that allow the incursion to these type of interfaces, including: Lego Mindstorms, WowWee and Ugobe’s Pleo.  The problem with using interfaces like this is that it requires high level of technical competency.

In storytelling terms, there are some tools developed KidPad for example allows generation of storytelling based on children’s drawings. StoryMat records oral stories and movements of stuffed animals on an augmented play.

This project uses a transparent table, underneath a camera is placed to detect the markers. This may be handy when using tangible objects that require a marker.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s