Soft Cinema:
Concepts |uncompressel version
Soft Cinema is based on three ideas.
The first is algorithmic editing of media materials. Each video clips
used in Soft Cinema is assigned keywords which describe both the "content"
of a clip (geographical location, presence of people in the scene, etc.) and
its "formal" properties (dominant color, dominant line orientation,
contrast, camera movement, etc.). Some of the keywords are generated automatically
using image processing software while others are input by hand. The program
(written in LINGO) assembles the video track by selecting clips one after another
using a system rules (i.e. an algorithm). Diffirent systems of rules are possible:
for instance, selecting a clip which is closest in color or type of motion to
the previous one; selecting a clip which matches the previous one party in content
and party in color, replacing only every other clip to create a kind of parallel
montage sequence, and on on.
The current version of Soft Cinema software lets the author define such systems
of rules which it then uses to put together a sequences of video clips which
best satisfy these rules. However, it is also possible to create other versions
of the software which would give the author a tighet control over the sequence.
For instance, one version may involve a video track completly edited by the
human author beforehand. Some shots are designated as “replaceable”
while others remain without modification (to keep narrative continuity.) Another
version may contain a variable which is set by the author and which tells the
program the probability that any shot will be replaced. In sumary, instead of
opposing complete randomness to the complete control of a human author, Soft
Cinema investigates a diffirent paradigm: using a computer as an "association
machine" that complements / reacts to images selected by the user with
other images.
(Interestingly, CD and MP3 players and well as software for music playback,
such as iTunes, all include an option to play songs in random order. Can this
be another example of electronic music culture being ahead of other parts of
culture in using new computer logic?)
While a dominant recent tendency in audiovisual computer culture (VJs, Flash
and Shockwave audiovisual pieces) is to synchronize image and sound (using video
output to control/generate the sound, or, conversely, using audio to control
video), Soft Cinema adopts another model, influenced by Eisenstein's theory
of audio-visual montage based on musical contrapunct. In Soft Cinema movies,
visuals create their own flow which mostly runs independently in parallel to
the flow of narrative, but "syncs up" with it in the key moments.
That is, periodically a particular video clip is selected to “anchor”
the narrative events.
The second idea
is what I call macro-cinema. While filmmakers such as Peter Greenaway
and Mike Figgis have already used a multi-screen format for fiction films, thinking
about the visual conventions of Graphical User Interface as used in computer
culture gives us a diffirent way to do macro-cinema. If a computer user employs
windows of diffirent proportions and sizes, why not adopt the similar aesthetics
for cinema? In Soft Cinema, the generation of each movie begins with the computer
program semi-ranomly breaking the screen into a number of square regions of
diffirent dimensions. During the playback diffirent clips are assigned to diffirent
regions. In this way, software determines both temporal and spatial organisation
of a work, i.e. both sequencing of clips and their composition.
Another inspiration for macro-cinema comes from contemporary cultural sites
which already adopted a multi-frame format. One example is news and financial
broadcasts which combine a video of an annoncer, a looping text, charts of stocks,
etc. Another example is the use of multiple frames in many computer games where
each frame may present the environment as scene by a diffirent character. Importantly,
in both examples, the information presented in diffirent frame is related to
each other but it also has a semantic autonomy (in contast to the traditional
cinema montage): for instance, an annoncer would still make complete sense if
all additional graphics are taken away. This gives us some directions of how
to use multiple frames in macro-cinema.
Finally, yet another inspiration for macro-cinema comes from the evolution of
video production and distribution technologies. While NTSC/PAL resolution of
analog video and television was hardly sufficient to present even a single scene,
HDTV standards (1920 x 1080 and the like) makes possible to devide the screen
into a few frames. In fact HDTV television specefications allow broadcasters
to break the total bandwidth of a signal (approximately 19 GB/sec in the US)
in diffirent ways: for instance, transmitting one high-res image and a few medium
images, or a larger number of low resolution images, etc. In short, "fixed
resolution - single image" convention of both 20th cntury cinema and television
has already become technologicaly and conceptually obsolete.
While at present (2002) HDTV equipment is quite expensive for artistic use,
it is possible to use Quicktime at DV resolution (480 x 720) to experiment with
how multi-frame high-res cinema and television may look like in the future.
This is the strategy used in Soft Cinema 2002 version. The original DV material
is scaled down to 320x240, 240x180 and similar resolutions and encoded in QuickTime
using Sorenson codec. This allows the Director program to play up to 6 clips
next to each other within one DV NTSC resolution frame (720 x 480).
In the installation and the PC versions a Director program assembles movies
in real-time. It is also possible to have linear versions of the project available
on DVD and video tape. To create this linear version, we (1) choose the movies
we want; (2) connect a video camera to a computer; (3) run Soft Cinema software.
The camera records what appears on a computer monitor. (The linear version is
available on DVD and all standard digital and analog video formats.)
The third idea is to create a true multi-media cinema. In Soft Cinema video is used as only one type of representation among others: 2D animation, motion graphics (i.e. animated text), stills, 3D scenes (as in computer games), diagrams, etc. In addition, Soft Cinema supplements a "normal" video image with other types of lens-based imagiory commonly used today by industry, science, medecine and military: the low res web cam image, an infrared image, edge-detected image as employed in computer vision, etc. While some music videos and artist videos already mix some of these diffirent types of imagery in one work, Soft Cinema assigns each type of imagery to a separate window in order to dramatize the new status of “normal” video, photographic and film image today – no longer the dominant but just one source of visual information about reality among many others. The additional inspiration for using diffirent representation of the same scene next to each other comes from the display setups used in medecine, aviation and other contemporary workplaces. Finally, rather than simply using these diffirent types of representation for a purely visual effect, Soft Cinema investigates the possibilities of using them together for fictional narration.
Lev Manovich
Berlin
July-September 2002