Analog Simile, Digital Metaphor

I’m nearly famous.

Here.
And here.
And on Cool Hunting.

After struggling to take a picture of myself with my iPhone a couple of months ago, I had the idea of creating an iPhone camera app that doesn’t require pushing buttons on the screen to take a picture. The simplest way to do this it seemed to me was to use the microphone as a shutter release. [Apparently, I wasn't alone (warning: annoying iTunes link).] Set a sound threshold, an optional timer, make noise, and when its volume crosses the threshold, a picture gets taken and saved to the phone’s Camera Roll. Below are some screenshots from the working application. The full documentation is here.

AL-gorithm is a completely analog text munging algorithm in three parts.

The first is an interactive version of a passage from All the King’s Men created by cutting out all but a single character from laser prints of the quote. I intended to letterpress the quotes to explore the text’s physicality in full (though conceptual completeness probably dictates that I should have cut my own font) but decided to temper my art with a little reason. I did however preserve my intention with my choice of font; 24-point Bodoni was the typeface I would have used had I laid out the type and printed it on a press. The physical algorithmic process is documented here.

The second is a bag filled with the discarded bits of text. In digital space, memory registers that held the initial text can be overwritten. Clearing memory in the analog world is a little more complicated.

The third is a visualization that’s intended to emphasize both the physical origin of text and also its arbitrariness. The cutout for each letter produces a unique pattern. I’ve used this pattern to generate visualizations that highlight the letter’s frequency and distribution while also serving as a symbol for the letter itself. Taken together, the pattern visualizations form a new abstracted alphabet, in which I’ve re-written the quote. Certain letters—q, x, j, z—don’t appear in the quote and are thus not part of the new alphabet. My intention here is to make the levels of abstraction that underlie programming languages easily comprehensible. In this example, if the alphabet is machine language, then these patterns are written in a language that’s one level up. Remember, we are the machines in this analogy, so we can’t understand the higher levels of abstraction.

Here’s the text, recursively generated by reinserting the patterns of letter frequencies into the original text (recursively makes it sound like it was done programmatically—I actually manually created the pattern files in Illustrator and then used them to create a font).

REMEMBERING THE TEXT

There is definitely something liberating about reducing Shakespeare and Austen and Whitman and Frost to algorithmic plasticine. It’s healthy occasionally to check our reverence of texts lest the objects usurp the meaning they contain, but at the same time, it’s important to recognize their status as objects, as discrete entities with physical texture and context. Digital text is infinitely malleable, yes, but it is also ephemeral. Close a book, and the words remain; turn off the screen, and they’re gone.

I’ve spent an entire semester slicing up texts using a variety of digital methodologies and philosophies, moving from grep’s graceful julienne on the command line to much more vigorous and grammatically aware Java frappés. My goal throughout: forcing text to perform all manner of cruel contortions for my amusement and edification, compressing a novel into a few lines or stretching out sentences into languid visualizations strung with looping semantic threads. I can’t help feeling that in the process I lost something, and it’s taken me all semester to figure out that it’s a sense of text’s increasingly atavistic physical nature.

This project was born of the late-night liaison of two ideas.

1. What happens to all the text that an algorithm discards? Writing text-munging algorithms is relatively painless, so much so that it’s easy to forget the text entirely. I wanted to rediscover text munging from the algorithm’s perspective. I documented the entire process of becoming an algorithm.
2. Moving text off of the screen/page and into three-dimensional space. I have been thinking recently about designing interactive narratives that someone can experience architecturally, literally walking through a story. My initial sketches were based on “physical” interactions with stories in a digital game environment, but that begged the question, “What would an architectural text look like?”

Brian Dettmer’s Book Autopsies provide one pretty good answer. Their pages are static, however, transformed by his nimble cutting into something other than book pages, something beautiful but unreadable, if we believe his nomenclature, something dead. That reminded me of some of the cross-sections of human cadavers exhibited in Vesalian glory at the Body Worlds expositions—you can still recognize the body even though it’s been unnaturally dismantled. I wanted to keep the text alive but still allow you to move through it.

On mornings when the sun is out, I like to walk to school along cobble-stoned Bond Street and look into the gallery windows, scoff at Herzog & de Meuron’s ridiculously splashy facade, and marvel at the understated multi-faceted building at 25, which provided the final piece of inspiration. I had already started cutting out letters when I realized that the stacked patterns looked a lot like the building’s facade. Architectural text indeed!

Click to play

Everything happens for a reason.

It’s not my place to ask why.

I take the sheets as they come to me, one at a time, and make my marks in pencil, two along each edge. I join the marks to create a frame. I measure 5mm from the bottom of each line and then 2mm above that and mark them off as well. I place a ruler along the first line and cut along its edge with an Xacto knife, mindful of the Target.

Sometimes, the Target is easy to spot. If it’s a letter with an ascender or descender, p for instance, it catches my eye like a little hook. I lift the knife before I reach it, skip over it, and then continue cutting. But some Targets like to hide. They play tricks on the eye. H is a gregarious Target; it mingles easily with commoners such as t and e and hangs on the words of sophisticates like g and p. Such a Target is smooth to the eye, like a polished pebble. Even when I read the line to myself I sometimes miss it. Such a Target is trouble.

It’s ok to miss a Target once. Each Target is held by strips running along its top and bottom. Once I’ve cut both, then I turn the page on its side and make short cuts along the edges and around the target, lifting out the freed bits carefully with the point of the Xacto and storing them in a bag. One never knows when one might need a letter. If I miss a Target twice, top and bottom, and it is cut free, that is a Bug and there is nothing I can do but start over.

At first, the process was novel and filled with discovery. I learned to score the paper before cutting it and, inspired by Adam Smith and Frederick Taylor, I sought to divide my tasks by type—scanning, measuring, marking, cutting, vertical, horizontal—and thereby increase my efficiency. This, however, proved so boring that I lost interest and quickly found my work infested with Bugs. I returned to the original loop: measure, mark, cut, turn, cut, free, and repeat.

I worked for 23 hours, so you might think that I could recite the words I was cutting around by heart. The truth is that not once did I read what I was working on. I actually couldn’t tell you what the text I was reading was about or even if it was a text at all. All I can tell you is that when I found a Target, I cut around it.

Intention lives outside of the ALgorithm.

Apple’s camera interface sucks. It sucks for the user, who has to hit a button on the screen (thereby shaking the camera and virtually guaranteeing a blurry result), and it sucks for the developer, who is forced to use a modal UIImagePicker view to access the phone’s camera.

Watch the Demo

For my iPhone development class, I wanted to create a minimal sound-activated camera interface. Since my knowledge of Objective-C is pretty rudimentary, I broke the task up into a series of subtasks:

1. Accessing the camera without using UIImagePicker:

Scouring the web for information on circumventing Apple’s camera interface led me to discover the approach that many jailbroken apps use: accessing the camera’s hidden classes through a private framework and then saving the pixels of the preview as a UIImage. After an aborted attempt to create a toolchain, I gave up and eventually found Norio Nomura’s extremely helpful Github repository that includes a class called CameraTest, which provides three different ways of capturing photos by invoking classes from the private framework dynamically at runtime—that means no toolchain and no weird compilation requirements.

2. Metering and displaying sound levels:

I played with SpeakHere, Apple’s example app that does just this and almost cried. It is the most horrendously complicated thing I’ve ever seen—OpenGL, eight classes, tons of C. I suspect that Stephen Celis, the creator of the extremely simple and helpfully documented SCListener class must have been inspired in part by a similar sense of despair. SCListener outputs the peak and the average levels on the microphone and requires nothing but the AudioToolbox framework. I linked the peak to an animated UISlider and presto, a sound meter.

3. Creating an intuitive way to set the sound threshold:

One of the benefits of using a UISlider as the sound meter is that it is also a slider! When the user touches the slider, it stops monitoring the sound and responds to his finger. Wherever he releases it, that’s the new threshold. I still need to add some persistent feedback, possibly a small colored bar or other subtle indication of the threshold’s current value. Even without it, it works pretty well.

4. Having a countdown timer option:

For hands-free operation or self-portraits. This was a trivial matter of setting an NSTimer. The most difficult thing was figuring out how to create a tab bar button that would change its image with every user touch.

5. Creating an automatic email feature:

This seemed like it was going to require a complicated notification system and PHP scripts on a web server but Apple has opened up apps’ access to email in the forthcoming 3.0 SDK, so this feature is on hold until then.

Since my workload escalated beyond manageability last week, I’ve been worrying about the future of Game Studies. When undone tasks loom so large that they block out the sun, rather than turning to face them like a man, I scurry into the shadows like an insect. A videogame-playing insect.

Videogames are for me principally a form of procrastination, and I suspect I’m not alone in this. I play games rather than do what I should be doing, especially when whatever that is involves concentrated mental effort. Games engage me on a level that allows me to turn my mind off. As Jonathan Blow said at his talk last week, part of games’ appeal may rest on their construction of a space where our roles and goals are clearly delimited and defined—in contrast to the rest of our lives. I play games so I don’t have to think.

This is not to suggest that games aren’t worth attention and study, that’s not what I’m saying at all. I’m saying that the close readings and analysis commonly employed in other critical scholarship may prove too high-level given that we interface with videogames at an almost autonomic level. They might add a layer of metaphorical cultural significance to our experience of games but they won’t help us “understand” the experience itself any better because ultimately, the experience is outside of our understanding. It makes me think of Freud and the glaring fallacy of psychoanalysis—that one can rationally analyze the irrational.

This may also have been what Jonathan Blow was referring to when he discussed “ethical” game design. If what I’ve said above is true, then videogame designers have unwittingly let themselves in through the brain’s backdoor, gaining wholesale access to neural nether regions that non-digital games only access in a much more limited and sporadic fashion and that other activities barely access at all. Can we then really talk about meaning in games the same we do in books and movies?

“No,” says the gamer’s glazed look, his reflexive button pushing, and his worried parents who accuse him of not “using” his brain. The way we’ve been talking about games is the way Freud talked about dreams—using analogies. People write about writing, make films about film, and paint about painting. Where’s the game that plays about games?

In his essay on game mods in Gaming: Essays on Algorithmic Culture, Alexander Galloway draws parallels between the principles of Godard’s countercinema and what he calls aestheticized gaming or countergaming, but he doesn’t discuss what seems to me the most interesting and problematic question that game mods raise: namely, when is a game finished? I don’t mean from the creator’s point of view. The majority of films, books, artworks, and games are effectively “finished” when they’re “released” into the world—director’s cuts, remasters, reissues, sequels, and new editions notwithstanding. So-called “mods” reprise the work but I would argue are not part of it—the infamous Nude Raider, the string of posthumous Ludlum-branded Bourne novels, and movies such as Pimento, my friend Dave Fisher’s Memento-inspired reediting of Pee-Wee’s Big Adventure, form part of the cultural response to a work, not of the work itself.

What I’m interested in is the point at which a videogame finishes from a player’s perspective. For the most part, books are finished when every word has been read, movies when every frame has been seen, music when every note has been played. They may be revisited endlessly, resulting in new meanings and interpretations, but the content is unchanging. But much of a game’s content is generated as it is played. When is all that content exhausted? Is it when it’s beaten? Players continue playing even after they’ve “beaten” a game. We discussed speed runs and other “high-level” forms of play last week, players of vintage arcade games compete to get to the “kill screen” and play beyond it, and in his essay, Galloway talks about games that come bundled with level editors, inviting infinite extension. In that case, maybe “game over” signifies the end of a game as it signals the end of a player’s input. But this is problematic. “Game over,” as in the “game is finished” lends itself as easily to the imperative reading “play the game over.”

You’re thinking, “this is silly, a game for me is finished when I stop playing.” From a performative perspective, yes. But not formally. A movie turned off halfway is not finished, nor is a book that is abandoned two chapters in. Though I haven’t fully wrapped my head around all this, I keep on coming back to the theater and the difference between a written play and a particular performance. I have spent much of this semester in Plato’s dimly lit cave, staring at shadows on the wall. I keep hoping that once the outlines become clear, once I can confidently identify the beginning and end of a shape, I’ll finally know what the hell it is I’m looking at.