Four Corners (Tentative project title)

Overview:
Four Corners is a guided audio visual experience that travels with the user through space and time. Using an Android mobile device and augmented reality(AR), the user will be enabled and encouraged to navigate a space. They will be engaged by location specific audio-visual feedback. An onscreen character will drive the narrative. The character will behave uniquely based upon where in the room the user is situated.

This piece is an audio-visual software composition. It will encompass aspects of new media production that I am experienced in, as well as areas that I am interested in exploring further. The more familiar part of the work is the audio-visual composition, as well as the narrative as a tool. These more familiar aspects will be enhanced with an area of new media that I more recently began to explore. That area being coding. I am excited to explore how these areas of my practice will evolve and intersect.

My Motivation:
I am making this work in order to continue my research into the narrative and how it is impacted by emerging technologies. This work is an evolution from my previous “Choose Your Own Dementia” work, where software enables a more fluid audio-visual experience based upon choice. Four Corners is also a step towards my forth year project intention. It is a natural progression given that it is solidly based upon new media practices that are at the core of much of my art. This work however moves away from the static and one dimensional narrative by allowing the user to drive the narrative based not just on isolated choices, but on their “state”, where they are situated. It opens the doors to an even more fluid and profound encounter with the narrative.

The onscreen character helps to ground and humanize a technologically mediated interaction. I am intrigued by the possibilities that may be uncovered through further exploration. In the case of this work I will move further than just to allow for user choice. I will allow for a character to respond uniquely based upon where the user points the device. AR by nature, when used in an artistic context, broadens the discussion as to the relationship between the user and the piece. The lines between the narrator and the audience is blurred.

How it Will be Done:
I will capture still image, video, and audio materials. Materials will be associated with locations in a space using either augmented reality markers, or physical coordinates. The captured materials will then be arranged to speak uniquely to the given location. This project is a combination of relatively entrenched new media practice and emerging technologies, namely augmented reality and mobile technologies. The most challenging aspect of this endeavor will be to program the device to respond to the environment and delivering and narrative that demonstrates the implications of the technology.

Conclusion:
I will create an AR driven narrative that will speak to emerging possibilities. By enabling a narrative with user specific information, such as location, a user can have a more personalized, and potentially a more personal, experience. They are enabled to drive the experience. An engaging audio-visual design will further augment the experience and advance a well needed discussion into the future of the narrative as it is impacted by emerging technologies. It is my aim to build upon m my experience and knowledge regarding today’s emerging technologies. I am also interested in how humans respond to these technologies.

 

The 4thDimension – No Longer Magic – A Response to “Why Things Matter”

I see my my fourth year New Media final project as an exploration into a sort of fourth dimension, a sixth sense. That sixthsense is being enabled, and proven to exist, through technological developments of recent times. What once may be considered magic, is now a norm, enabling us to see into realms that where recently unfathomable to most. This new sixth sense is the suject of my interest. It is comonly called “augmented reality”, a form of data representation interwoven with the “real”. This however is a very general look at my true area of attraction. I am interested in gaining knoweledge as to how software may enable, inform, and help us to learn. This year it is my intent move towads this end goal by exploring tools that exist, and learning more about the possiblilities and the impact of an impending explosion of augmented content.

There were a number of interesting ideas in “Why Things Matter” that caught my attention. Aside from the practical possibilities that are revealed through the paper, the work is also quite prophetic for its time. Today we see the evolution of “Things” connected, interwoven, and sharing data, and hstories, with other “Things”. We also see technology driving conversations and defining new perspectives for how we observe and interact with data, and the world at large. More and more data is becoming inseperable from that larger world and making it smaller. On one side, I find it intreging that if data is not apparent or detectable without technology, then that technology becomes one with a given experience. In a way this is comparable to “The Pigeon That Blogs”. The pigeon is democratized through her ability to go where humans cannot, to places that are invisible to humans and make visible the otherwise unseen. Though it is not the pigeon’s ability to fly alone that has earned her freedom, it is also technology.

 

The Pigeon That Blogs questions and draws attention to how we value members of society, human and otherwise. Contribution increases the value of an object, as does presence. If either of the two are not apparent, an member of society is inevitably undervalued. If societal member is seen as irrelevent, if it does not contribute in a recognized way, or if an “object” is not “present”, it is not valued. When important decisions are made, whether or not the deccision impacts the object, the object has no voice.

 

The Pigeon That Blogs represents a compelling look at the evolution of data, and the progression of the use of data. Looking at this concept informed a particular project idea I call “Time Travel”. In Time Travel, an observer can look at a particular thing in various states of being. For example we could see a sort of stop frame view of how a metal may erode over time, as opposed to excusivily viewing the shiny static imagery that we are used to in AR. Augmented Reality may unlock interesting possibilities for this project. Using augmented reality, one could look at an engine, for example, and see what erosion may take place in the future. Conversly, one may look at an older engine and see what it looked like upon its creation. In other words Time Travel will allow for the user to step into different points in time visually and audibly. When this technology is coupled with “smart parts”, meaning parts that are able to track their history, as is mentioned in Why Things Matter, it will enable parts to be swapped out and create a modular scene that would inform the observer of the greater Blogject’s parts. Time Travel will ideally store its own histories, both self diagnostic, maintainance, and otherwise. Why Things Matter outlines the transformative effects of objects taking this leap. I have seen very few AR objects that have anywhere near this level of sufistication. It is no trivial task, however this is the future that Bleecker predicted.

 

When Bleeker speaks to the Blogject, he drives home one particular point that I feel is very important. He outlines how individually present the Blogject is, and how its path, position, and history makes each blogging object valuable. As he puts it, “Blogjects have self-contained (embedded) histories of their encounters and experiences”. What was once invisible and valueless now is recognized. Bleeker goes on with the parallel, “Whereas once the pigeon was an urban varmint…the pigeon that blogs now attains first class citizen status”. Again in the realm of AR, we give presence to the invisible. “Choose Your Own 3Dementia” is the evolution of a previous project that I created called Choose Your Own Dementia. The latter is built on an AR framework whereas the original was 2D. The latter will also be more intelligent. It will not just know that you have selected left or right. It will know in real time the where abouts of objects in relation to the user, and will give the user approppriate feedback through an onscreen personality who delivers a personal experience. The user points to AR items throughout the space. Music/poetry is generated based on what the user points at. Choose Your Own 3Dimentia will focus on Augmented reality, but importantly is a character driven experience. This work also makes a statement upon another point that is detailed in The Pigeon That Blogs. That point is presence. Ironically the augmented personality is an apparition to the unaided eye. Technology gives form to the character as value is given to our blogging pigeon. I feel this is an important conceptual aspect of Choose Your Own 3Dementia.

 

Another idea that I have conceptualized is See through You. It uses AR to look into layers of an object, and to see varying aspects of a things’ composition. One can look at the composition of a given item, peeling away various layers of an object. The question naturally arises, is this object real, as much as it appears to be present. The user will however be looking at a real tangible object being peeled away at will. On another level, if See Through You or Time Travel could be used to make a health and awareness statement by showing adverse effects of human behaviour within the human, or other systems that are not readily visible, it may in this case have an effect somewhat similar to the iSee surviellance project. The exposing of the otherwise invisible may effect people’s actions.

 

In summary, I intend to delve into the world of augmented reality. I will choose between three projects, all of which will be revealing and informative in their own way. Choose Your Own 3Dimentia allows for user choice and character interaction. I am interested to explore how these factors will work together in an attempt to make a lasting impact on a user. See Through You, uses AR to question presence, or maybe reenforce a an unseen presence, while Time Travel allows for the user to experience a greater aspect of the life of what we consider static. It allows for the observer to see a larger narrative that is invisible to us without technology. See Through You is actually a relative of Time Travel. I would truly wish to one day see both present in one work, however each does maintain its own meaning when looked at seperatly. These projects push questions of how cutting edge technologies may be used as learning tools, and how characters and personnalities may assist in an instructional capacity. Why Things Matter braodened my perspective of data representation and sharing. The paper also introduced me to the “Blogject”, and their “peculiarities”. I do appreciate the implications that these concepts have had on my own ideas. The Pigeon That Blogs speaks to the usefulness and impact that a particulare object, or species, may have upon humans. It seems to me that Blogjects are empowered by technology. The idea speaks less to the traditional aspects of utility and more so to a contemperary new media perspective. The Pigeon That Blogs illustrates this distinction, transforming, as does luguage that blogs, into an upstanding member of society by becoming useful, and present, to the human eye.

 

 

Net Worked – A response to my networking exercise

It all seemed too easy at first. Our goal was to connect two computers, with Processsing, and Arduino over the internet. We needed to convey a real world action from one side of town, to a real world reaction on the other side of town. Our choice physical input device over at my classmate’s place where the server was setup, was a potentiometer, as well as a button to send an on off signal to another pin.

Calvin, my classmate, set up his side while I got the client hardware setup on my side. We were on Skype, so we could not only speak through how the process was going. We could literally show each other. The setup didn’t take long at all. After all we’d set this network up a few times in class. The process had gotten drilled into both our brains and was now being further reenforced. I connected my Arduino via USB, and loaded the appropriate code. Now it was time for me to upload to the board. Error! What the….? Yes, was getting used to sometimes finicky nature of Arduino, com ports, breadboards, windows, the net, and of course Windows.

 

One thing I realize from experience is that even when something seems to be straight forward it may take a methodological approach to resolve some of the finer issues. I tried to plug in my second Arduino to see if Windows would detect it. Nothing. I then switched the USB port that the Arduino was plugged into. NOTHING! All my comm. Ports were still greyed out, meaning that it is not being properly recognized. OK, one more thing. I then unplugged the Arduino and turned off the software and restarted…. Bingo! I knew it was now working because I could see the com3 port as active in my Arduino software settings. I was then able to upload to the Arduino from my computer.

 

 

We were ready to go! Calvin started the network up on his side, so it was my time to connect. I started up Processing and pressed run. Not only did a graphical box appear displaying numbers between 0-255 that reflected the position of the potentiometer on Calvin’s end, I could see one of the lights brightly illuminated. Then all of a sudden I saw it jump from one value to another. Could it be working with such ease? Could it be this simple?

 

It got unsimple again real quick, when we started to debug, or to notice little details. Like what about the other light? Or, why was the network behaving so slowly? Upon inspection we realized that the connection seemed lightning fast for the first few seconds. As Calvin rotated the potentiometer the values changed immediately, but that only lasted temporarily. Moments later the system began to lag. We waited almost a minute at times to a change that had been made on the server side.

 

We ended up fiddling with the circuits and observing, and dissecting, various facets of the code and hardware. Calvin even switched computers. I went from wireless to plugged in directly to my router. I even ripped apart my breadboard in an attempt to ensure that the pins were working as planned without some loose wire confusing the issue. After almost two hours of tinkering, we declared… a sort of victory. We got the connection open and functional, and were able to transmit our intended data. We did have questions as to why there would be such latency after a prolonged connection. By prolonged I am talking 15 seconds before the latency became unbearable. We wondered how Skype could transmit immense quantities of video and audio data, while our simple code got choked up. We theorized as to why. Maybe it was a caching issue. Maybe Skype was hogging resources, or was it our hardware, or network conditions. This is the question that plagues us to this day.

 

OK, maybe I am being over dramatic, but the exercise was a fun and interesting one. We were able to connect our computers across town and manipulate physical things. We troubleshooted some of the problems, though others remained. In doing this exercise we strengthened out understanding of how networks are created and some of the issues that may be present. It also reenforced to me the amazing nature of the internet. Calvin could have been on the other side of the planet and our system would still have worked, albeit with even more latency I expect. This simple connection to me is the opening a door into numerous possibilities.

 

Choose Your Own Dementia Documentation

Choose Your Own Dementia
A Play on Truth & Perception

History
“Choose Your Own Dementia” is the culminating point in a journey. It is not the end of that journey but certainly a key point. I had thought about code previously as a mysterious realm, which I could interface with only through others who were more adept to speaking the language of the machine.

http://www.youtube.com/watch?v=0hNv2mA0Z28

In 2010 I enrolled in course that explored Data Visualization. This course would prove to broaden both my over all perspective, as well as advance my ability to communicate with the very machines that enable my art. I initially found it challenging to merge coding with my creative approach, however through the study of data visualization, I was exposed to a number of creative art projects that represented data computationally. These works opened my eyes to a new way to approach art. The process of exploring art that used data to enable, beautify, etc became truly empowering to me when I realized that my poetic and musical background can itself be the data that I am representing as an artist. Words, text, image, code, and concept were my tools. Coding became the process of learning to craft art through computer language based upon my original concept.

Concept
Using words, being of particular interest to me as a poet, I intend to create a poetic representation of word, sound, and rhythm, that allows for an observer to direct the path and outcome of the musical, and visual composition.

User input at key intervals, I call Decision Points, will determine the sequence, and composition of the art work. The software starts off playing the loop and visuals for the first Decision Point. A host fades into the scene. The hosts’ purpose is to guide and prompt the user to action. The user is prompted to action if they do not select either left or right. If the user hovers over the left side, the character on the right taunts them coolly to choose the right. Words and graphics reinforce this divide, and widen the gap between left and right.

The software will loop in this Decision Point until the listener prompts the path which the song will proceed with by clicking left or right within the canvas.

Choose Your Own Dementia represents a dichotomy that is a norm to us in North American society. Consider “Politics”, where we are seasoned to pick between political poles, welding alliances based upon oppositional ideologies.

Sample lyrics – Subject “Moon Landing”
Right side selection – negative perspective
Associated words underlined

“They say they landed on the moon in 1969
A tall tale a hoax turned national pride.
Wind flapping the flag in no atmosphere
They lying in my ear but no one cares
Damn man, a sham, fabrication that rallied a nation”

Sample lyrics – Subject “Politics”
Left side selection – positive perspective
Associated words underlined

“This political tapestry, the true actuality
Basis, without this
no stability
Law and democracy would be a casual casualty
Legitimately certified, factually verified
This political side is where order resides”

Inherently, this binary divide is reminiscent of true computer language, where each process is simplified into ones and zeros, true or false. In the case of Choose Your Own Dementia, as is the case in Western politics, the user is craftily catered two choices. The choices act as a way of dichotomizing the given issue. One character, on the left, is unwaveringly supporting and positive towards any given topic, while the other opposing character, on the right, is consistently pessimistic. Though this method of delivering music may seem to give more choice than is available in music, or video, it is far from perfect or complete. In the same way that our democracy could only be considered perfect in comparison to an extreme dictatorship, or our capitalist methods may only be considered optimum only in comparison to another defunct economic system. The limitations that are enforced are regulated through carefully delivered choices, inevitably to the artist, this draws attention to the range of unspoken selections that cannot be made. As an artist I see the potential for such an art work to allow the user infinite choice, yet in this incarnation “choice” is represented only by two opposing views.

Progression
At the Decision Point the rhyme, audibly, prompts the user to select another choice, left or right. The subject appears boldly at the top of the screen, with two opposing words appearing on either extreme of the screen. One will be suggestive of a “fact”, the other will speak to a “fiction”. I have selected a number of subject words that may be associated with both fact, as well as fiction, depending on ones perspective.

I used a thesaurus to generate a list of words that are synonymous with fact as well as fiction. “Fact”, for example is associated with the words “truth”, “real”, “brass tacks”, “official”, etc.., while “Fiction” is connected to “Fake”, “Fabrication”, “hoax”, etc… If a user selects the path on the left side, they will hear an almost apathetic representation of the generally accepted truth. If they select the path on the right the user will experience a pessimistic view that opposes the popular sentiment. This division of left and right, positive and negative, allows for a perceived choice, as controlled as it may be. A user can choose what they wish to experience through their input.

1) A song will play and loop. A subject word appears, as well as two dichotomizing words that represent the choice positive or negative. The user is prompted to select left or right

2) If the user scrolls over the center left or right the animation changes uniquely

3) If the user doe not select anything, or places the pointer outside of the canvas the animation will change prompting action.

4) If the user clicks on the right or left selection the appropriate vocals will play accompanied by custom music. Lyrics speak specifically to the user’s request positive or negative, based on the left or right selection.

5) Move to the next Decision Point. If this is the last Decision Point then the program returns to the first Decision Point

Future Iterations
In the future I would like to implement a more visually appealing and interactive user interface to the work. I will add more graphical variance to stylize the work as well as to cue the user. Implementation of video as well as programmatic stop frame animation style that was used in this iteration will improve that over all work. I would like to achieve the effect of a blend between music, video, and a video game combined. More topic variations, as well as increasing the pool of poetic, audio, and visual data pool will promote a deeper experience, and increase replay value, while breaking a way from a binary core structure.

Scopophelia 2010 Documentation

Scopophelia 2010
Concept and design: Kwame Newman-Bremang
Illustration: Alan Bao

“Scopophelia” is a play on space, perspective, and perception. It is a new media montage that reflects classic new media techniques fused with traditional artistic styles and methods. The work, inspired by Kara Walker’s shocking silhouette art, pulls the observer into a world all to itself.  Scopophelia is a gritty yet stylish blend between colidoscope, collage, illustration and sculpture.

Composition
The Project is encased in a jet black housing, the exterior stylized with Alan Bao’s.”Fighting Racism with Political Correctness” as its cornerstone. When the observer peers into the small window in the side of the box they seem to be looking into the lives of a variety of unsuspecting characters. The a range of situations that the observer is exposed to is reminiscent of the work that inspired Scopophelia. It speaks to the pains, and to the joys, of human life. The observer is in fact looking at the black and white illustrated scene through a series of reflections that reveal glimpses into the lives that are on display through the work. Masterfully illustrated graphics, collage, and sculpting techniques are merged into an intricate miniature world of reflection, stylistic tension, and movement.

Artistic Style & Inspiration
Kara Walker’s controversial reflections on power, and the complicated often complicit relationship between dominant and subservient lent direction to Scopophelia. However, where she uses stark silhouettes in her work, we chose a uniquely stylized approach. Our project maintains in common with Walker’s work a bold representation of the human condition. It is a commentary on our sometimes painful relationships and the turbulent situations which life offers.
Art 21′s interview with Kara Walker is a very good reference point:

http://www.pbs.org/art21/artists/walker/clip2.html

The illustrative style of Scopophelia is intentionally distinct, still it is informed by the bold style of Kara Walker. Instead of situating the subject in the foreground in an empty white canvas with carefully selected items balancing the portrait, Allan Bao, the illustrator, has chosen to blend black and white within each of the portraits while still delivering a bold and deliberate, though in some cases suggestive, message with the use of contrast and careful artistry.

Scopophelia also draws inspiration from the work of another notable artist. That artist Anish Kapoor who uses reflection as a way of binding the observer, the environment, and art. Scopophelia explores this technique of using art as a metaphoric, as well as a littoral, reflection of a world while at the same time it distorting perspective. “Scopophelia”, speaks to the human desire or compulsion to watch, rendering the viewer almost powerless, as do the images that Kara Walker casts through her silhouettes. Scopophelia allows the observer to unlock a window into a handful of human experiences, both painful and desirable, combining the act of watching with the act of being watched. The fact that the view of the world is guided by a hardly visible motor within the world that turns a mirror and slowly shows the observer more of the world further adds to the tension.

Alien Landing

Alien Landing
Data Visualization Fall 2010
Kwame NB

http://www.symbolikmusic.com/AlienLanding_KNB/

As a poet and artist, I have always been interested and inspired by language and audible communication. I am intrigued by rhythm, style, content, as well as the tools that have emerged to change, enabled, and sometimes hinder how we internalize the world around use. I do hold a deep value for the transmission of cultural data through music in particular. This interest sparked my will to explore what can be done with Processing’s sound libraries in order reflect a data set in a an compelling, yet meaningful, way. “Alien Landing”, is a work of digital art that reflects the aforementioned interests and curiosities. I find audible representation of data to be often overlooked. As such, it is an area of mystery to me, rather like the UFO data that drove this particular work. It was my goal to use the powerful tool that Processing is, to generate an appropriate a particular mood, using data, sound, and visual cues.

I chose to experiment with allowing user input to drive a musical reflection of different eras. When the user inputs any date between 1960 and 2010 the program references a data set of UFO sightings that were reported from that date range. If the date inputted by the user is between 1960 and 1979, music is played that incorporates an original musical production with popular music samples from that particular era. The samples are mixed with the music in a very moody (and “trippy”) manor. The same is done for the range of dates between 1980 and 1999, as well as dates between 2000 and 2010. They a mixed in subtly enough to make the sample hard to identify. The samples are also chopped up and used with a number of other samples from the same era. When one considers this, ass well as the fact that effects are applied to these era specific samples, they do seem ambiguous when incorporated into the final piece.

Visually, Alien Landing draws from the reported shapes of UFOs that are on record to have been sighted in the skys of the United States. The shapes of the UFOs are drawn on screen over a map of the US in the area that the data states the sighting took place. I chose to generalize my representation by drawing only rectangles, triangles, and circles. I further generalize my depiction of the data by classifying ovals, circles, and spheres as the same, while I also classify squares, rectangles, and boxes as the same. In essence, if the data says that an oval, circle, and a sphere where sighted between a given date range in the same area of the US, my program will draw three circles, and not three different shapes. They are drawn in semi transparents, and so the more sightings that were reported the darker the representation of that shape will be.

The core aspects of Alien Landing where, my data set, audio production software, and Processing’s Minim sound libraries. Once I brought my chosen data set into my program, I would need to allow user input to tell Processing what to play. I implemented this part using place holder music. I then selected appropriate music from my own library of instrumental music to replace the place holder tunes in the final version of Alien Landing. Once I got the sound working, and coded the music to play specific sound based on user input I then went to work on finding popular music samples from each era (1960-1979, 1980-1999, 2000-2010). I took those samples and incorporated them into my music. I created one base melody , and one melody for each era.

Through this project I was able to explore a less glorified method of data representation, that being the use of sound. This is a natural fit for me as a poet and musician. The project was revealing. However, as many questions that were answered, even more considerations arose to the surface through this process. For example, I wanted to learn more about Processing as an artistic tool, as well as about the Minim sound library. In general, the breadth of applications that one can use Processing for was firmly reenforced through the production of this work. As for the lessons drawn from my exposure to Minim, I learned both of its usefulness as well as its limitations, particularly in the Processing implementation. More significantly however, Alien Landing through its representation of particular popular cultural moments, and its use of data sets, that tie it more firmly the eras being depicted, inspires even more thoughts and ideas. The mood is appropriately eery, while it reflects this artists tastes and sentiments. Alien Landing is one small step, moving towards a more holistic approach to the representation of data. Not all data need be represented by sound alone, but when this vital sense is considered, and when the tools are better tuned, it will certainly lend to an improved and more immersive experience. This work is infused with the sounds of times. Alien Landing takes from a variety of pieces of data that were left behind as a last word on the given topic. My greatest lesson in producing this multifaceted work is that when we, the new media artist, learn to master crafting designs for varying senses and perspectives, while creatively utilizing appropriate portions of the sea of data that exist in our world, the sky is no limit.

Scopophelia Project and Its Inspirations

Scopophelia Project and Its Inspirations – Nov 6th 2010

“Scopophelia”, as I have named this piece, is inspired by Kara Walker’s shocking silhouette art, as well as Anish Kapoor’s reflective steal sculpture work.

The Project is encased in a cubic shell the size of a shoe box. When the observer peers into the small window in the side of the box they seem to be looking into the lives of a variety of unsuspecting characters. The a range of situations that the observer is exposed to is reminiscent of the work that inspired Scopophelia. It speaks to the pains, and to the joys, of human life. The observer is in fact looking at the black and white illustrated scene through a series of reflections that reveal glimpses into the lives that are on display through the work.

Kara Walker’s controversial reflections on power, and the complicated often complicit relationship between dominant and subservient lent direction to Scopophelia. However, where she uses stark  silhouettes in her work, we chose a more stylized approach.  Our project maintains in common with Walker’s work a bold representation of the human condition.  It is a commentary on our sometimes painful relationships and the turbulent situations which life offers.

Art 21′s interview with Kara Walker is a very good reference point…

http://www.pbs.org/art21/artists/walker/clip2.html

Anish Kapoor uses reflection as a way of binding the observer and the environment to the art. The art is a true reflection of the world while at the same time it distorts perspective.

“Scopophelia”, speaks to the human desire or compulsion to watch, rendering the viewer almost powerless, as do the images that Kara Walker casts through her silhouettes. Scopophelia allows the observer to unlock a window into a handful of human experiences, both painful and desirable, combining the act of watching with the act of being watched.

The illustrative style of Scopophelia is intentionally distinct, still it is informed by the bold style of Kara Walker. Instead of situating the subject in the foreground in an empty white canvas with carefully selected items balancing the portrait, Allan Bao, the illustrator, has chosen to blend black and white within each of the portraits while still delivering a bold and deliberate, though in some cases suggestive, message with the use of contrast and careful artistry.