Interview with Sinan Ascioglu:
OpenProcessing Architect

By Tim Stutts on Friday, February 26th, 2010 at 1:00 pm.

“Driving through Iceland” sketch by dotlassie. Winner of Rhizome’s Tiny Sketch Competition.
OpenProcessing.org is a site that has built a community around sharing visual coding examples created in Processing. As user number 36, I had the unique privilege of watching the idea take shape, while in a thesis group with Sinan at NYU’s Interactive Telecommunications Program. During it’s first two years of activity, the site has grown to host thousands of user-generated sketches and subsequent conversations between artists / programmers, teachers, and students from around the world. Sinan and I escaped the snow recently at a café outside Washington Square Park to discuss OpenProcessing’s origins, Rhizome’s collaboration with OpenProcessing in the Tiny Sketch competition, and what we can expect for the future. – Tim Stutts

Tim: How did you first come up with the idea for OpenProcessing?

Sinan: I guess the first thing to talk about is OpenVisuals, which was my Master’s thesis project at ITP (Interactive Telecommunications Program, New York University). I was reading Edward Tufte’s books at the time, and I became very interested in data visualization. In the meantime …. read more on Rhizome

In this project, I took it as a challenge to capture the viewers’ eye movement patterns, which differ from movie to movie. I proposed that these patterns were directly the result of movie directors’ unique style of shooting scenes. In the first phase, I built eye tracking glasses and a software to capture the motion of one eye of a person watching a movie scene and the looking position data. In the second phase, I built the visualization which draws the eye movement curves on the wall, and the wooden controller lets users to explore these patterns by browsing different movies using an intuitive interface.

Our focus while watching movies shifts continuously from corner to side, actor to actress, dancer to assassin, making our eyes move continuously on the screen from left to right, top to corner, etc.. What we are not aware is that the smooth/complex patterns of these focus shifts of our eyes are different in each scene/movie and posses the signature and style of the director.

“Eyes on You, Tarantino!” tries to capture and visualize these patterns by projecting the patterns on a column, and gives users the opportunity to explore the movies in reference to these patterns using a physical controller surrounding the column.

Project includes two parts:

1. Tracking the eye movements of the audience with custom build eye capturing glasses, and calibration of the captured data.

At the first stage, I created the eye tracking glasses with a camera attached to one lens and recording one eye. Using Processing, I wrote a code to track the position of where the eye is looking, Using this glasses and the code, I recorded the eye movements of 4 people watching the different scenes from Death Proof (Quentin Tarantino), Rear Window (Alfred Hitchcock) and Hyperballad (music video from Bjork, Michel Gondry).

Early sketches of the installation

2. Creating a physical installation to present the captured patterns, in a meaningful data visualization which allows users to browse the movies in reference to these patterns.

At the second stage, I worked with data to see if I can observe any patterns that would be unique to the scenes and directors. The patterns I ended up focusing on:

• Directors’ shooting style defines how jittery or smooth our eyes follows the screen
• In the scenes where actors stand still, Tarantino prefers providing “eye action” moving camera continuously around the actor. Hitchcock prefers positioning actors on viewport separately to provide “eye action”. In Gondry’s example, he uses a unique technique to oscillate the camera that makes the eyes oscillate left and right.

After observing these various patterns, I designed a physical installation in which these patterns can be projected in accordance to the scenes being displayed. After a set of sketches and cardboard prototypes, I made the project ready to be exhibited at ITP’s Winter Show 2007.

Having raised a lot of attention in the recent years, data visualization proved itself to be an important subject to provide better understanding of data and information which cover a wide spectrum from data-intense scientific researches to election results. Most of the times, and except pie charts and bar graphs, good data visualizations are designed and tailored for a specific data set, and are not necessarily available publicly to be used with different data sets. This limits the potential of visualizations to be used in different purposes.

A simple user scenerio is:

>> A person interested in visualizing a data uploads the data set to the website. She browses the website and picks a visualization to visualize his/her data. In the meantime, She allows her data set to be used by anyone under Creative Commons License.

>> A visualization artist finds this data very interesting. He copy/pastes the given piece of library code to Processing and starts building his own visualization using that data set. The library allows him to access the data set that is in the cloud. Once he is done, he uploads his visualization to the website to share with others. In the mean time, the library he used makes his visualization possible to work with other data sets on the website with this visualization through the website.

This framework includes two assets:

• The website, OpenVisuals.org, which supports the framework as the place for people to upload, browse data and visualizations, to explore by mapping the data sets to visualization, and to communicate with each other using commenting, messaging, etc…

OpenVisuals Thesis Presentation from
Sinan Ascioglu on Vimeo.

• OpenVisuals Java Library, which is an API for Processing to make building visualizations easier by providing common functions. Also, this library functions as the bridge to map any uploaded visualization to any data set on the website.

This project can be considered as the ‘visualization counterpart’ of many data websites (eg. Swivel, Many Eyes, Gapminder), making it possible for users to develop and collaborate on the visualizations end.

The Fail Part

While working on this website, I preferred to design and develop while it was live on the web: anybody could see the website being developed on the fly. This gave me a great opportunity to get early feedback on the features, design and bugs. The most significant feedback that I had was to make the website open to any Processing sketch, and be less specific rather then focusing on the visualization theme. Before any visualization-sharing website, Processing users needed a place to share any of their sketches.

Upon this observation, I copy/pasted the whole website under a new domain, OpenProcessing.org, and stripped out the visualization&dataset focus by redesigning couple of pages. Since then, OpenProcessing has welcomed by the community with great interest and appreciation, and this led me to discontinue my efforts on OpenVisuals.org and put all my energy on OpenProcessing. Well, the rest of the OpenProcessing story is here.

This post is password protected. To view it please enter your password below:

Password:

“Shall you create something for the world’s largest high resolution video wall!” (120 x 12 feet, to be precise)

and we added:

“Shall there be storytelling! Shall there be live performance! Shall there be physical computing!”

IAC building in Manhattan consists worlds widest high-resolution screen, of which Robert Moon and I were lucky enough to get an opportunity to create ‘something’ for this massive screen, as part of Big Screens class at ITP, NYU. Although there were no limitations as of concept, we took this as our challenge to make sure we have storytelling, live performance and physical computing in the project! Then we produced

JohnTrain: an interactive cinematic performance

A housebreaker’s story as an assassin is reflected on this 120 x 12 feet square video wall, accompanied with the live performance of the housebreaker interacting with the screen simultaneously.

In the performance, a housebraker breaking into a house notices the train toy which makes him remember his last day working as an assassin. His playing with the train toy set interacts with the screen, on which the story of his last day is reflected on the screen. The housebraker tells his story to the audience in sync with the unfolding story being visualized on the screen.

In the design process, Robert Moon and I questioned how such a wide screen would be used at its best. The screen’s wideness make it impossible to see the all big picture at once, and made the audience to look either to the left or to the right, hence allowing them to focus only one side of the screen. We wanted to use this constraint as a complication of our project; we wanted users to follow multiple things happening on different sides of the screen.

Our solution was to display a train car with 7 windows, covering the screen from side to side. Audience could follow a hide’n'seek action inside the train through the windows. We wrote a story wrapping this feature: In the setting of 2050s, an assasin hired by the communist government trying to find and kill an activist girl in the train compartments, while the girl changes compartments for various reasons.

We shot the scenes for 7 compartments separately in front of blue screen, then edited them to create this 120 x 12 feet video. Using Processing, we developed the interaction between the toy train set used in performance and the train on the screen.

Exhibited at:

IAC Gallery, December 7th, 2007.

NYU Tisch Interactive Telecommunications Program Winter Show, December 16-17th, 2007, New York, NY.

Gameplay

Game play is very similar to Dance Dance Revolution: you are supposed to press on the corresponding shapes that you see on the screen with a good timing. However, in this game corresponding shapes are attached to a bra and to a boxer through touch sensitive sensors. The level increases as you accomplish catching a given number of shapes. As levels increase, game gets harder, faster but more intimate as new shapes are introduced to the game, which are on more intimate sections of the undergarments.

Design

In the design process, Jenny was already working on building these new controllers as her thesis project. Since we took the challenge of bringing a new type of experience to the game, I focused on creating the most simple game for users so that they can easily map the controller functionality of the undergarments to the actions in the game.

We got inspired by Dance Dance Revolution, since it was a fairly known game with a simple interface, where dancers tap on the circles on the platform that is corresponding to the shape they see on the screen. We applied the same concept for this project, and used distinctive icons on the undergarment sensor areas that could be easily mapped to the icons they see on the screen.

You can have more information on the project at JennyLC.com.

exhibited at:

SIGGRAPH Unraveled07,
Aug 2007, San Diego, CA

Around the Coyote, Jul 2007, Chicago, IL

NYU Tisch Interactive Telecommunications Program Spring Show,
May 2007, New York, NY

NYU Tisch Interactive Telecommunications Program Thesis Event,
May 2007, New York, NY