Understanding the Composite and Spatial

The original three landscapes were never conceived with a narrative backbone. Instead I relied on the concept of each landscape to dictate its aesthetic; meaning, by deriving a heirarchical set from either cliff/forest/swamp images were found from around the world. After slicing, distorting and scaling, pasting three imaginary landscapes were born. In order to further analyse each as a prospective site, I referenced the composing images back onto their geographic origins. Instead of a site being one stationary location, site is now redefined as a collective, fixed through the lens of a camera and uploaded into cyberspace.

Applying AR and HT to space

Working with Augmented Reality (AR) and Head Tracking (HT) has rooted the project in its dependency upon the image. Rather than accept the image as being a single superficial representation. AR + HT allow a composite reading of space via multiple images. Readings of inaccessible spaces  are possible via HT; you control the depth of perceived space and the cone of vision, to peer around, up and above. Augmented Reality presents a different system by referencing the virtual back into the actual world. The same spatial applications can be applied, by seeing inaccessible spaces simply through an overlay.

Inside floors and walls are spaces that do not existe in everyday conciousness. These inacessable depths hold the building's organs and seem mysterious only when it comes time to repair damages. By measuring light with composite images a perception of depth and obstruction is formed. HT places the collective in an environment where we the viewer can look inside walls and under floors.

Under the Floorboards

 

Moving Up the Wall

The Swamp composite and spatial

The Forest composite and spatial

The Cliff composite and spatial

The what why how

This Masters project centers on the agent of change within a given environment. Presently, the project has evolved into an investigation that uses Augmented Reality [AR] as a transmitter between systems in a given setting. In adopting Augmented Reality as a tool, the Master’s project must confront the issues of AR’s spatial application and the ethics behind the projected image. Necessary to the goals of the projects is this discussion of ethics. As the projection is inherently an application into space, an environment is not affected physically at all. Within an installation involving projections, the factors are the people, sound or movement within a space; leaving no impact upon the environment. Rather than contend with a superficial result, the use of Augmented Reality desires a more invasive result. Two avenues of possible architectural application currently under study are: spatial camouflage and either inducing or alleviating a Synesthetic condition.

Camouflage is promising because of its tactical approach to space; the desire of individuals to connect with their surroundings could lead to a blending of the physical and virtual. The intention here is to venture beyond the end result being an installation piece that relies on projections. In order to do so, an understanding of the limitations of Augmented Reality is essential in order to explore it as a tool. To properly understand Augmented Reality a comparison with Virtual Reality [VR] is necessary. The user in Virtual Reality undergoes a complete immersion into a virtual environment; whereas Augmented Reality is the superimposition of information or virtual objects over the real world. AR operates by recognizing prescribed symbols in a real-time video feed. Over these symbols a virtual object is then projected. Now imagine an object in a room with its own symbol, the Augmented Reality reads the space behind the object and superimposes it over the object. No longer visible within the virtual world, the object is now camouflaged or transformed. It would seem plausible that if what was on the screen was fed back via a projector into the real room the object would be camouflaged in reality. Since AR works with a live video feed, an object in motion would be hidden as long as the camera could capture the space behind it. Parallel to the Augmented Reality concept is the Wii Remote Head Tracking1 hack. Developed by Johnny Lee while at MIT, the Wii Remote is reversed in orientation and instead searches for the user’s position in space. This extroverted system then allows the display to become three dimensional within its frame. Further investigation will occur in order to combine this tool with the intention of spatial camouflage using Augmented Reality.

The alternative application is to use the overlay between virtual and real-time time is to enduce or alleviate the condition of synesthesia. Synesthesia is the neurological condition that occurs when one sensory element crosses paths with another sensory element involuntarily. In the case of induced synesthesia, the Augmented Reality tool can be used to create a sensory environment for the user that cross pollinates sound with vision and possibly smell. Instead of the brain triggering automatic reactions in other senses, a computer is programmed to respond to sound as a series of colors or movements. Induced Synesthesia can be applied on an individual basis to either alleviate mental or physical strain. By employing Augmented Reality, the user creates his/her own environment without affecting others.

Bibliography

Hisel, Dan. Camouflage: Or the Miscommunication of Space, http://temptationbyspace.blogspot.com/2009/01/camouflage-or-miscommunication-of.html, 2002.

Lee, Johnny Chung. Johnny Chung Lee: Projects, Carnegie Mellon University http://www.cs.cmu.edu/~johnny/projects/wii/, Redmond Washington, 2008.

The Synesthetic Experience, MIT, http://web.mit.edu/synesthesia/www/,  Oakbog Studios, 1997.

---

1 “As of June 2008, Nintendo has sold nearly 30 million Wii game consoles. This significantly exceeds the number of Tablet PCs in use today according to even the most generous estimates of Tablet PC sales. This makes the Wii Remote one of the most common computer input devices in the world. It also happens to be one of the most sophisticated. It contains a 1024x768 infrared camera with built-in hardware blob tracking of up to 4 points at 100Hz. This significantly out performs any PC "webcam" available today. It also contains a +/-3g 8-bit 3-axis accelerometer also operating at 100Hz and an expandsion port for even more capability. These projects are an effort to explore and demonstrate applications that the millions of Wii Remotes in world readily support.”  Johnny Lee

Possibility of...

I came across the work of Enzo Mari, an Italian designer whose work spanned 50 some odd years. I find the two pictures below fascinating and way beyond their time-aesthetic. The potential for playing with depth perception is fantastic and could really help structurally with my project. Have to work it in somehow.


http://www.designboom.com/weblog/cat/8/view/4127/enzo-mari-the-art-of-design-exhibition-at-the-gam-museum-turin.html


homage to fadat', lights, switches, plexiglass and steel, 1967

 structure, anodized aluminium, brass and steel, 1964

 For the Wiimote Augmented Reality to work, the viewer needs to be wearing InfraRed sensors. This allows one person at a time to see the projected image. This part of the project has been interesting because I am having to construct a circuit board for the LED's to work. Using the Instructables website has been helpful in order to learn a lot about wiring a light system. The next step is to setup the Wiimote relay between the remote, infrared lights, and the display.

Success!

Last week I began testing out the AR plug-in for Sketchup, and luckily the problem was in the way I was saving the jpg. Sketchup does not view jpgs with transparency so after a few modifications I finally had a transparent image in the AR viewfinder. I staggered the split landscape and can now project the cliff into Augmented Reality through the webcam and into my computer. So far the plug-in has a few limitations: as the pieces line up they need to be progressively enlarged to provide a faithful image, and it is limited in its 3dimensionality. As a feedback system this could be used with a more sophisticated system to have an environment altered by certain amounts of data. Say, placed in a room, the projected environment could morph positively or negatively depending on sound level, number of people, their movement through space. So the more people were in the room, the landscape would flourish, if it was a dead zone, the landscape would decay.

Setting the Table

After doing some further research into Augmented Reality I have been working with ARToolkit.

The ARToolKit is a collection of libraries, utilities applications, and documentation and sample code. The libraries provide the user with a means to capture images from video sources, process those images to optically track markers in the images, and to allow compositing of computer-generated content with the real-world images and display the result using OpenGL (Phillip Lamb, 2004). ARToolKit is designed to build on Windows, Linux, SGI Irix, andMacintosh OS X platforms.

It seems that the ARToolkit runs on a .net programming architecture instead of plugging into another programme - like the AR plugin for Sketchup, a much more direct system. So far the parts for the toolkit are downloading and then it is a matter of connecting everything together.
My plan is to separate the landscapes into components and then spatialize using either the ART or the plugin for Sketchup. The plugin requires fineagling, first in 3Dsm and then put into Rhino.

.........................................

The Wii Remote arrived and I am in the process of constructing a sensor bar. Maplin has the IR LED's thankfully!

The Tools

A few of the tools I am using to develop the landscapes in 3D are AR Toolkit, Augmented Reality and Johnny Lee's Wii Hacks. It seems like every Wii remote has been bought in the city of London, so I am waiting for one online.
The Description below is taken from Lee's website:

Head Tracking for Desktop VR Displays using the Wii Remote

Using the infrared camera in the Wii remote and a head mounted sensor bar (two IR LEDs), you can accurately track the location of your head and render view dependent images on the screen. This effectively transforms your display into a portal to a virtual environment. The display properly reacts to head and body movement as if it were a real window creating a realistic illusion of depth and space.

The program only needs to know your display size and the size of your sensor bar. The software is a custom C# DirectX program and is primarily provided as sample code for developers without support or additional documentation. You may need the most recent version of DirectX installed for this to work.

This type of technology would really allow to experiment with change over time within the landscape. I am curious as to how it could develop away from a flat screen. Perhaps how one can occupy the same space as the 3dimensional projection.

So far I have been testing out the spatialization of the cliffs with the Augmented Reality plugin for Google Sketchup. The setup being so simple I could immediately have a simple volume projected [see below]. However when I spliced the cliff into different staggered layers, like a loaf of bread on a flat plane,  the jpg images would not transfer with transparent backgrounds, prohibiting a compounded view of the cliff when projected in 3D. Since Sketchup is so basic I am planning on working with ARToolkit and seeing if the results will be better and at a more sophisticated level.

The Reaction

Post the Barbican exhibit [mentioned below] I began thinking about how a simple information relay could be applied to my project. The siting within the nuclear fallout idea [previous posts] was a tangent that began to take ideas into a limited direction. As the three renders are landscapes within themselves, the issue of siting is there already. Furthermore by the end of the project a smooth process needs to be apparent.
S suggested spatializing the landscapes in order to test out relationships within each [cliff, forest,swamp] and as a collective. The eventual change in each site is hazy still but I am learning that not every step needs to be pre-determined. So, going back to Hemmer's exhibit, the ideas of humans tuning the space/surrounding audio could easily be translated to the tuning of any surrounding landscape. Necessary elements would be: factors [people in the room, temperature,movement, shadow], a warping device [a system that would read factors and allocate a prescribed rule of change], this change then would trigger the landscape to distort.


The previous post mentioned an interface called Reactable, a demo below. The factors that make up the interface are incredibly simple. A good model to follow when producing my own system.

Frequency and Volume at the Barbican_January 2009

This Saturday was the ending of Rafael Lozano-Hemmer's project Frequency and Volume. Not knowing what exactly to expect it was pretty brilliant to have the installation explained via one's own exploration. Meaning that if you truly wanted to understand the project you would have to look for the clues indicating the mechanics of the installation. The premise was so simple. The human body moving through the long curve of the Barbican hall is a giant tuning device for several radio stations. The stations belonged to news, popular music, maritime, aeronautical and even pirate radio stations. Volume was controlled by one's distance from a projector on the ground. The projector casts your shadow and while you move a webcam records your position in space. Position is then relayed back to the main radio control room where equalizers, amplifiers and computer harddrives calculated the appropriate radio station and volume. This setup was repeated every ten steps; if a shadow moved even slightly in either direction the station would change.

In Kevin Kelley's book Out of Control, hive mentality was explained through a similar experiment. In the mid nineties a relay system was devised so that a camera could read a room full of blue and red placards. Each person held a placard and the objective was to collectively land a virtual plane. After several tries, the group began moving instinctively instead of individually. Similar to Hemmer's project one could easily imagine a roomfull of people tuning the DJ's set at a party.
As we were exploring the exibition at the Barbican the docent gave us a mini tutorial and mentioned a similar system that Bjork used in her recent tour called Reactable.

 

The above is video taken from the Curve space at the Barbican, judging from the previous exhibition spaces Curve couldn't be more appropriate. The space one occupies to tune the radios has to be linear. The people at the exhibition seem to be missing the point, interested more in the shadow projections than the radio stations. *Notice when the giant baby shadow is cast, the volume soars.

The Skeleton of the Skeleton: The site destroyed

The idea of the site being a Nuclear Fallout result came from the last three drawings [the final layer] and this mapplet from CarlosLabs. They devised a project called Ground Zero.

"Have you ever wondered what would happen if a nuclear bomb goes off in your city? With Google's Maps framework and a bit of Javascript, you can see the outcome.
And it does not look good."

Skeletal Locations

++++++++++++++

Shopping for Ingredients

Within the process of any project it is imperative to ground concepts with research. Looking at Theory, History or Contemporary concepts gives a new dimension to one's work. During the Christmas break it was lovely to just gather books together and collect my thoughts and those of others.
Of the phrases typed into the Library's search engine the most popular were: landscape, viral landscape, bio-morphism, biomechanics, evolutionary architecture, evolving, war, conflict, hybrid...


This waterfall of reading yielded some standout literature.


Frazer, John. An Evolutionary Architecture. Architectural Association Press. London. 1994.
John and Julia Frazer's experiments at the AA were published in An Evolutionary Architecture. AD magazine had some further articles on the Groningen Experiment, a mid 90's testing for the capabilities of computers to assist with "lazy" [Frazer's term, not mine!] architecture. Taking cues from natural selection, DNA & its creation, they applied several rules to their design. In the case of the Groningen Experiment, the logic of science was applied to the urban planning for the city of Groningen.
Basically they thought of allowing the fabric of the city to grow organically through what they termed data-structure. The Frazers conceived the data-structure as a set of points in space through which information is passed locally from neighbor to neighbor.
The Urban nature would evolve via this pattern: [the chart below was interpreted parallel to the readings].
              
This method proved an interesting possibility within the concept of viral landscape; in order to determine how the landscape would change, dissemination of said virus, etc. Using logic such as the Frazer’s data-structure and urban design pattern could lead to a mapping within the cliff, forest & swamp. Aside from a mapping, the ideas they ascribed to the growing urbanism can by similarly transferred to how the organism would operate. The Keeper is the evolver, the Instigator is the enabler; both governed by methods of control in the case of the Groningen Experiment they are a genetic algorithm [which solves replication] and a neural network [providing checks and balances within an environment]. Hence, an understanding of the technology-machination behind the Keeper/Instigator is necessary.

The Party has Begun

Post the term review, and piggybacking on the three final renders came the reality that the project had too many unanswered questions. How exactly does the organism work? Where lies the Architecture, within the organism or the affected landscape? etc, etc. So to posit any response, a scientific methodology had to be adopted. I could imagine the project unfolding in a five step process:
I. Initial Concepts (ideas that I have been grappling with from the very beginning of the project).
II. Within a site defining desired outcomes of the project.
III. Adding variables and responses.
IV. Define the method by which outcomes are achieved.
V. Testing through drawing / fabrication (models).

The Tea Party in the Swamp

The effects of the Instigator and Keeper upon one environment. Here the Swamp; in the last stages color is heightened, miltiples are exaggerated to mutate the landscape.

The Tea Party in the Cliff

The effects of the Instigator and Keeper upon one environment. Here the Cliff; not only limited to the nominal environment of rock cropping but to also Ice floes and structures within like stalactites.

The Tea Party in the Forest

The effects of the Instigator and Keeper upon one environment. Here the Forest.

The Pre-Party Invitation

 

For those liberated from computers after reading this message, simply point and shoot with any mobile device at the code above and this blog will be in your pocket always. 

To download the reader software, go here to download on your computer or mobile phone.

*I can just imagine integrating a shotcode, like the one above, into a landscape insidiously. Growing one from grass like crop circles, or stones, clusters of trees, etc.  Information could be projected into space and snapped up from airplanes or viewed from the moon.

*Image taken from Your Competent Esoteric Guide Online.

The Image Made

One of the Keeper+Instigator shells. The problem lies in surfacing the point clouds of the tentacles, massive cleaning.

Scratch that and Bake it Again: a Revision

 

This structure inherent everywhere becomes the all-consuming drive of the organism; to collect, compile an addendum of surroundings, a DNA database. This organism would be the lab assistant of Carl Linnaeus or Ernst Haeckel. Without any database the organism is limited to its seed volume. Only through the acquisition of DNA from others does it have a method of defense, feeding or survival. Within its own system, the hierarchy of information would determine what form it would take in any given situation. The organism is now a Keeper. It becomes the guardian of life itself, storing elemental data in preparation for nature's eventual obsolescence. Its brain is a hybrid of a left/right brain capability and that of an octopus. An octopus actually has two brains, one that controls its motor skills, directing the eight extremeties. While its second brain directs its survival.
  

Transmittance would occur with touch. Through a two-way osmosis, the Keeper releases nano particles into the organism as a tracking device, this link serving to transmit new skills the organism would be constantly acquiring. In return the Keeper collects both a DNA sample and a steady stream of data referencing the organism's fight for survival. Information is stored closest to the seed within. Layers of DNA are fused in thin films of mother of pearl [twice as strong as any high tech ceramic]. A product of self-assembly, the shells around the seed are formed in a sac of sea water. Surrounding the sac of water is the malleable body of the Keeper, shedding and appropriating the various forms found within the surrounding environment.  Its tentacles allow for efficient travel on both land and water. A hierarchy of exterior shells is shed or appropriated in definition to each enviromnet. A wet swamp will preclude a fin extrusion, etc.

As part of any environment, the Keeper cannot simply exist without a consequence upon its context. Like Cerebrus, the two headed guardian of Hades, the Keeper communicates with the Instigator. The Instigator operates on a nano-scale by accelerating evolution. It does so by injecting a mutation within an organism's DNA, thus constantly altering the environment. The same method of transmittance is used by the Keeper. By brushing up against or touch, mutations are disseminated into the expecting organism altering its genetic code. The Instigator reads the same stored data as the Keeper, the end result to modify the data and modify the organism. The environment is altered positively or negatively. Trees grow knobbly brances, Lily pads turn flourescent, Banyans grapple with each other to extremes.

 The organism is in constant flux to suit its environment, it is a hybrid of a hybrid of a hybrid...  Hence the shape has been a sticking point. I always imagined it form-less. The everything and nothing of Universality. However for the purpose now, the above form is how the Keeper and the Instigator are conjoined; sharing a unit of data, reading the seed differently. 

 

Image from RampantBycicle

The Image Maker

New Technologies/Methods [and learning them] is perhaps my favorite aspect of Architectural Education. Last week I got in touch with the individual that runs the Arius 3D scanner at UCL. Mainly used for Archaeological and Art History conservation purposes, he sometimes gets the random Bartlett student in for a scan. The last being scans of chicken bones. He informed me that the DAS clay model was perfect because white or off white was the ideal surface for working with. I have a few pictures below of the scanning process, sans flash and on my camera.















It is a rather large machine essentially directing light into a mirror triangulated with light passing through a lens. If both streams to not connect then the scanner reads nothing, as in spots with shadow or pitch black. The end result is a point cloud model. I was hoping to have a 3d model ready in a couple of days however the files are so incredibly heavy due to the data per sq cm that Rhino kept crashing.

Connecting the points to form a surface is proving to be difficult as well. However i have found that with a point reconstruction set, a Delauny Triangulation will work to form a mesh that needs further cleaning.

Will post renders in the future.

The Bouncer as a prototype

Hercules capturing Cerberus. 

For the purpose of this project the two headed Cerberus is ideal as a model for an organism that survives based on an input and output [Keeper and Instigator]. Both halves share the same source of energy and information found within the core of the seed.

Information is stored closest to the seed within. Layers of DNA are fused in thin films of mother of pearl [twice as strong as any high tech ceramic]. A product of self-assembly, the shells around the seed are formed in a sac of sea water.

Ingredients

These are just a few images that have been floating around in my studio, sparking ideas for the Keeper & Instigator. Thinking about why the Jellyfish or Octopus as a form are so popular when mechanics marries design brings to a conclusion efficiency.  Their shape is pared down to body and extensions which is a form perfectly suited for any environment.

Images from FFFFound

Image from Screendrafts of Matrix movies for the Sentinel robot

Image from FFFFound

Image from SpaceCollective.
Project taking Gregg Lyn's form and supersizing it into lifescaled architecture.

The Food Critics

Within an operational sense how the organism works was a big question: thinking thoroughly how it disperses within an environment, how it collects surfaces [shedding, suction cups, bio mimicry]. So far it is an "it". Where does it come from, a thistle as a generator. The energy comes from the seed. Will the seed bring forth an animal or a hybrid?
[looking into robot prosthetics to model the mixture of mechanics with a living tissue as part of an animal; if a starfish could regenerate a bionic arm instead of skin and bones]


Setting the form aside [the form seen in photographs below], the naked idea is an organism that can live in three basic environments, born out of the concept of hierarchy. This structure inherent everywhere becomes the all-consuming drive of the organism; to collect, compile an addendum of surroundings, a DNA database [remember Peter Postlethwaite in Aeon Flux, a guardian of a library in the sky that held the living memory and of every human being]. This organism would be the lab assistant of Carl Linnaeus or Ernst Haeckel. Without any database the organism is limited to its seed volume. Only through the acquisition of DNA from others does it have a method of defense, feeding or survival. Within its own system, the hierarchy of information would determine what form it would take in any given situation. The survival skills of a shark would take precedence over that of a seal. The organism is in constant flux to suit its environment, it is a hybrid of a hybrid of a hybrid... The organism is now a Keeper. It becomes the guardian of life itself, storing elemental data in preparation for nature's eventual obsolescence. 

Transmittance would occur with touch. Through a two-way osmosis, the Keeper releases nano particles into the organism as a tracking device, this link serving to transmit new skills the organism would be constantly acquiring. In return the Keeper collects both a DNA sample and a steady stream of data referencing the organism's fight for survival. Information is stored closest to the seed within. Layers of DNA are fused in thin films of mother of pearl [twice as strong as any high tech ceramic]. A product of self-assembly, the shells around the seed are formed in a sac of sea water. Surrounding the sac of water is the malleable body of the Keeper, shedding and appropriating the various forms found within the surrounding environment. 


As part of any environment, the Keeper cannot simply exist without a consequence upon its context. Like Cerebrus, the two headed guardian of Hades, the Keeper communicates with the Instigator. The Instigator operates on a nano-scale by accelerating evolution. It does so by injecting a mutation within an organism's DNA, thus constantly altering the environment.

++++++++++++++++++++++++++++++++++++++++++++++++++++

Rupert Sheldrake deals with memory ["the basis of memory in nature....the idea of mysterious telepathy-type interconnections between organisms and of collective memories within species."] and Morphic Resonance. MR  is defined as a pattern within organisms determined not by their prescribed natures but by repitition.
..."it is not at all necessary for us to assume that the physical characteristics of organisms are contained inside the genes, which may in fact be analogous to transistors tuned in to the proper frequencies for translating invisible information into visible form. Thus, morphogenesis fields are located invisibly in and around organisms, and may account for such hitherto unexplainable phenomena as the regeneration of severed limbs by worms and salamanders, phantom limbs, the holographic properties of memory, telepathy, and the increasing ease with which new skills are learned as greater quantities of a population acquire them."

The Interacting Multifarious Guest

 

 

 

Lost Under the Table

This image was a first pass at rendering in 3D studio max. Needless to say the elementary nature of the render is useless and have begun working on a different render approach entirely.

The Multifarious Guest