July 9th, 2010
Meet Eliza, the Flashiest Phone Bot Around!
Eliza sits at her desk in her office. She completes ordinary office tasks—she checks her email, she drinks her coffee, she gets up to go photocopy something or talk to a colleague, and once in a while she checks out the New York Times. Little does she know, she’s being livestreamed to the whole world over the web. If someone calls, she picks up. Sometimes she recognizes the caller, sometimes she does not, and sometimes the connection is so bad that she hangs up and calls back.
Eliza lives on a screen in an eddy of a high-trafficked area, say an out-of-the-way elevator lobby in a busy building. A user sees her and after a couple of minutes, his curiosity gets the best of him and he succumbs to the flashing invitation and calls. To his surprise, after a couple of rings Eliza picks up. Phone conversations are ritualized in the first place and the added awkwardness of voyeurism and conversing with a stranger create the ideal situation for Eliza’s black-belt phone jujitsu which with minimal effort wrests control of the conversation from her interlocutor. It’s a bit like a good dancer foxtrotting and waltzing an overwhelmed novice around the floor.
The prototype is rough, but it works, though because of Flash’s arcane and draconian cross-domain security measures, I can only run it locally through the Flash IDE or stream from my machine using a personal broadcasting service like ustream or livestream (in order for it to work properly on the web, I’d have to host all the components I enumerate below on a single box, something I have neither the hardware nor the inclination to do). The main problem is that I’m making XML socket connections from Flash; if I used a PHP intermediary, I could probably get it working, but again, the whole inclination thing is missing and the thing is already mindbogglingly complicated. Maybe at some point in the future. The following video demonstration will have to do in the meantime.
SO HOW DOES IT WORK?
Warning: this is not for the faint of heart.
Eliza has a ton of moving parts:
- The Asterisk script: A simple program that answers phone calls and hands them to a PHP script, which connects via socket to the main SWF.
- Various PHP scripts: One to handle connections from Asterisk, one to reset connections from Asterisk after a call ends, and one to initiate callbacks when required.
- A simple Java socket server: Adapted from Dan Shiffman’s example, this program runs in the background on the Asterisk server, waiting for connections (phone calls). When a call comes in, it connects it and broadcasts call events (new call, hangup, button press, etc) to the PHP scripts and the main SWF and allows them to talk to each other.
- The main SWF: This is the brains of the operation. It loads the movies of Eliza and controls the logic of their looping as well as the logic of the audio (via socket connection back to PHP and then to Asterisk via AGI).
- The looping movie files (not completely smooth in this prototype, notice the moving phone and the changing light conditions!): These live in the same directory as the main SWF, which streams them as needed (for a web deployment, they’d probably have to be pre-loaded and played back).
- The sound files: These live on the Asterisk box (completely separate from the movies) and are played back over the phone, not the web.
UPDATE: I’m presenting Eliza at Astricon in DC in October, so I should have some interesting observations to report soon. There are several things I’d really like to do. First, I’d like to actually get this working somewhere where I can observe lots of unsuspecting folks interacting with Eliza. I never really got to see someone who didn’t know the backstory calling in, partly because I was exhausted from thesis when I had the chance to show it and partly because there were lingering bugs I had not yet located that occasionally caused the whole thing to stop working—there are so many things on so many separate machines that can go wrong, it took quite a while to troubleshoot. A larger sample of reactions would allow me to rework the conversations so that they’re more disorientingly convincing—better pause timing, more realistically intoned, and taking into account repeat callers’ stratagem’s to see if Eliza is real. I could then reshoot the video so it is completely seamless. That would require monitors, good lighting, laser levels, an OCD continuity editor, and several days of shooting.
If you know of an easy way to overcome the cross-domain headaches, leave me a comment! If you want to fund such an undertaking, please do get in touch! Otherwise, enjoy the video above.