Tag Archives: collaboration

Sandscape and Illuminating Clay in the Context of TUI Frameworks

24 Sep

Sandscape and Illuminating Clay, projects by the Tangible Media Group at the MIT Media Lab, are TUIs for landscaping and architectural modeling that allow users to mold organic materials to actively construct a model of their design.  A camera or laser detects the changes in the physical environment and models the changes on a computer display.  In addition, information such as wind flow is projected directly onto the physical model, allowing the user to dynamically see how their model will interact with the environment.

Like most TUIs, Sandscape and Illuminating Clay aspire to enhance a user’s experience behind the tired and true “WIMP” (window, icon, mouse, pointing device) interface.  While there is indeed a display screen involved in this project, these projects are more akin to playing in a sandbox than playing a computer game.  This TUI contains many of the elements of a “Reality-based Interaction” framework, which is described in “Reality-based Interaction: A Framework for Post-WIMP Interfaces”.

Reality-Based Interaction (RBI) Themes:

Naive Physics (NP):

These featured TUIS heavily take advantage of a user’s sense of naive physics- a user will be able to sense when a landscape structure is precarious or unstable rather than relying on a computer’s computation.  Much more intuitive than a CAD program, the user is able to mold, build up, and depress the material instinctively instead of searching through a library of complex extruding, sweeping, or filleting options.

Body Awareness and Skills (BAS):

The dexterity present in the human hand is much greater than the dexterity possessed by a mouse pointer.  When we mold sand and clay with our fingertips, we are able to delicately press and mold exact features.  We are much more in-tune with our fingers than features in a software program.

Environmental Awareness and Skills (EAP):

In both of these TUIs, the user is able to model environmental conditions, projecting additional information onto the organic form.  With both TUIs, users can view information about elevation, water flow, etc. designated by the color gradient projected onto it’s surface.  As the user manipulates the material, they can see how the changes they make directly influence how the landscape interacts with the environment.  In this way, users are not limited by being unable to physically model all factors they would want to consider while landscaping a space.

Social Awareness and Skills (SAS):

Rather than using one input (a mouse), multiple users are able to create a model together.  A large rectangular container allows at least 4 people to stand on each side and make their own respective changes to a model.  One user can literally “make suggestions” to a model by molding another person’s creation.  These TUIs would also allow persons unfamiliar with computer modeling software to design a landscape.  For instance, a sculpture artist that has no previous experience with landscaping but creates beautiful forms may be able to translate their art into a designer landscape.


1)  There is no “undo” button.  While a digital representation of a model may be preserved, it may be difficult to return to a previous version of the physical model once changes have been made.  Users have to be judicious when deciding to make major changes.

2)  The physical model is most likely not permanent.  Clay dries out.  Sand degrades.  Not to mention that once you’ve created a physical model, you cannot make a different model until you destroy the first one.  Both of these TUIs are likely intended for exploratory models.  I assume permanent digital models will have to be created in any case.

3) Textures and complex shapes may be difficult to mold.  Some shapes are more easily created with a computer.  When attempting to repeat similar shapes or structures, users interacting with these TUIs would have to hand-mold the same shape over and over if they do not already possess multiple tokens resembling their design.  In addition, when it comes to “perfect” shapes, computers are better than humans.  We leave fingerprints and may frequently mold uneven shapes.