The Lytro Light Field Camera
Let's think about photography as people live it. A posed family picture might be taken once, then again, and again until the right combination of open eyes, smiles, and light and shadow produce an acceptable portrait. An action, performance, or sports shot that could speed by too fast for human judgement partakes of a surrogate: the high-speed motor drive capable of taking many shots per second, from which the photographer later chooses the most retrospectively opportune option. You don't have to be a professional to recognize a "missed shot" taken near to but not right at the opportune moment. Photography freezes moments in time, so capturing the right moment in the right way is important.
Getting an exposure precisely right is part of the craft of photography, but also part of its frustration. Someone's eyes always seem closed, or the lighting changes, or an accidental jitter of the photographer's hand shifts the framing or focus inadvertently. Corrections can sometimes be made after the fact in the (digital) darkroom, but even an amateur photographer knows that some exposures are always doomed — a blurry image can never be recovered with chemical or software exposure correction. Lytro, a Silicon Valley startup co-founded by a Stanford computer science PhD named Ren Ng, has been working for the last six years to prove otherwise — and to change the very idea of what a photograph is in the process.
* * *
Photography didn't used to be so easy. Before the 1920s, most high-quality photographs were taken with view cameras: large, bulky devices with bellows necessitating tripods, lights, and long exposures to expose large film plates. These were the tools of Brassaï and Walker Evans and Ansel Adams, devices that offered superb resolution and detail thanks to their larger film plates, which were typically reproduced with low- or no magnification during printmaking.
Like every important artistic material, the view camera remains a viable and appealing medium for photographers. But using one in the late 20th century (as did Stephen Shore and Sally Mann, to name but two famous examples) requires making a deliberate aesthetic choice. By contrast, in the early 1900s, "still camera" just meant "view camera." It's the type of shift everyone should be familiar with today. Just think of the different connotations of "computer" in 1962, 1982, and 2012.
Ansel Adams with his camera (Wikimedia Commons)
Still, entirely new photographic apparatuses are relatively rare. The transition away from the view camera era began between the two world wars, when manufacturers devised apparatuses that adapted cinema film for use in still photography. These were the first 35mm cameras, the most famous and influential of which was the Leica.
Almost all of today's cameras, analog or digital, owe something to the 35mm film format and the devices that exposed that film, so it's easy to overlook the formal properties that characterized the Leica when it was new. Most of all: size and portability. The Leica could be held and pocketed, and its film could be exposed quickly through short lenses that let through enough light to produce appealing images wide-open or stopped-down. It wasn't the first camera to open photography to snapshooting (the Kodak Brownie deserves that laurel), but it was the first serious, popular, professional example of photographic process that we know today.
The photographer Henri Cartier-Bresson is probably the most famous 35mm photographer. His work helped established reportage, street, and candid photography as a viable artistic and practical affair. His small Leica rangefinder could be easily carried throughout the streets of Paris, New York, Madrid, or Budapest, looking for fleeting moments that could be captured with precise timing, framing, and focus before they vanished forever. The camera was as much a partner in this affair as was the photographer: Cartier-Bresson could zone-focus wide-angle lenses stopped down to medium- to small-apertures, such that he didn't have to look, frame, and focus a scene before capturing it. Shooting from the hip became as common as shooting from the eye for street photographers looking to remove themselves from a scene so as not to disturb it.
Henri Cartier-Bresson
Cartier-Bresson called this window the "decisive moment." This search for the fundamental instant in a particular time and place has become synonymous with candid and street photography. In a 1957 interview with the Washington Post, Cartier-Bresson offered a compact definition of the concept:
Photography is not like painting. There is a creative fraction of a second when you are taking a picture. Your eye must see a composition or an expression that life itself offers you, and you must know with intuition when to click the camera.
That is the moment the photographer is creative. Oop! The Moment! Once you miss it, it is gone forever.
* * *
The early days of digital photography intensified our attention to the decisive moment, as cheaper digital cameras exhibited shutter lag — a delay between the time the photographer actuates the exposure and the recording of an image. But the decisive moment's prominence has also hidden other important aspects of photographic craft. A photograph can be made at the right time, but with the wrong exposure. Or it can be correctly exposed, but with a shutter speed inappropriate for capturing its action, or an aperture that produces too much or not enough depth of field. A picture can be mis-framed so that a key subject is in an undesirable location. But even so, darkroom and digital tools have offered solutions to most of these problems. An oddly framed negative could be cropped before or after enlargement, or with a single click in Photoshop. An over- or under-exposed image might still be restored in the darkroom or in software postprocessing. With digital photography, white balance and color balance can be easily corrected. A blurry photo can never be made sharp, but it might offer a desirable aesthetic outcome anyway — a decisive moment found unexpectedly.
All of these photographic methods still make the tacit assumption that the image originally produced is complete and atomic, light having been bent through a lens to expose a chemical film or photosensitive digital sensor all at once, forever.
An image taken with a Lytro camera is not really an image, but a machine capable of producing many possible renditions of a similar image
It's this aesthetic assumption that the light field camera challenges, embodied by the Lytro, which debuted roughly a year ago. It's a camera that hopes to rethink the legacy of the Cartier-Bresson decisive image. The Lytro takes photographs in a different way, a manner that's almost incomprehensible to anyone who experienced photography in the 20th century — which is to say, nearly everyone alive.
Lytro cameras create what the company calls "living pictures." It's a bit of a misnomer, especially since the term "living picture" (or tableau vivant) already refers to a live theatrical scene in which costumed actors pose in a live scene without movement or speech. A more accurate layman's term would be "living negative," even if that phrase is guilty of anachronism. An image taken with a Lytro camera is not really an image, but a machine capable of producing many possible renditions of a similar image, any one configuration of which can be chosen in the unique digital darkroom that is Lytro's desktop software.
Ng suggests thinking about it this way: a Lytro photograph is not a traditional still image, but an array of software simulations of many possible virtual cameras, coupled with an abundance of light data describing a particular scene. Imagine if you could freeze time at the moment you snap the shutter, creating not just one exposure, but many variations of that exposure, each with different points of focus and depths of field. The Lytro does something analogous, but with novel optics and computation instead of magic.
When a Lytro image is downloaded from the device for development, any one of these virtual cameras can be selected, each of which sees the captured scene in a different way. The process by which a virtual cameras is configured and selected is based on ray-tracing techniques — the same methods used to render a three-dimensional scene in computer graphics. Instead of creating a simulated scene from the interactions of virtual objects as pixels, Lytro's darkroom configures a real scene given a particular light vector to a particular focal plane.
These days, a new gadget is hardly a novelty. Now that the Lytro camera has been out in the wild for nearly a year, we can stop thinking about it as a gadget and start considering the aesthetics that the gadget affords. We can finally stop answering, "What is the Lytro?" and move on to "What does the Lytro mean for photography?" Just as understanding Cartier-Bresson's Leica helps us appreciate the concept of street photography, so we need understand more about how the Lytro works to grasp its unique visual aesthetics.
* * *
In a traditional camera, the film or sensor is exposed to light focused by a lens on a single focus plane. In other words, the light that enters the camera through the lens strikes the exposed surface having been bent in just one way — the manner that corresponds with the lens's current focus and aperture.
Imagine you are operating a single-lens reflex (SLR) camera. As photographer, you can zoom, focus, and stop down the lens at whim before depressing the shutter. Any of these combinations of optical circumstances represent "possible photographs" of a particular subject at a particular time — each with their own unique properties: a point of focus, a depth of field, an exposure, and so forth. But once the shutter is pressed, all those possible photographs are collapsed into one single photograph, just like a life decision collapses all the possible alternate timelines that radiated from the moment just beforehand.
The Lytro camera implements a different idea: What if that single shutter exposure could capture more than just one of the possible photographs of a scene at a single moment in time? Mechanical examples of light field photography (or plenoptic photography) date back to the early 20th century, and researchers have been implementing workable (if commercially inviable) versions of it since the 1980s. A "light field" describes the amount of light emanating in every direction from a single point. It is therefore "plenoptic" (or "full of sight") — that is, it is a camera that can see all possible paths light could take from the lens to the sensor, rather than just the single path recorded by an ordinary camera.
To accomplish this feat, light field cameras have two lenses. One is the traditional lens assembly that focuses light entering the camera's dark chamber — the object you would normally call a "camera lens." The second is really an array of small microlenses, which are positioned at the rear of the dark chamber in front of the sensor. This array is something like an insect's eye, but flat instead of rounded, and it's the key to light field photography. Each of the microlenses focuses on a different part of the dark chamber itself — taking a tiny picture of one part of the light field. The resulting digital negative is not a single photograph, but an array of tiny photographs, each representing a unique view of the scene, from a slightly different perspective and with a slightly different point of focus.
Ren Ng
Insects with compound eyes have a different perceptual apparatus than bifocal vision creatures like humans do, and their manner of perceiving the world is very different from our own. Looking at a raw light field camera "negative" is a bit like looking at a representation of a fly's vision: a mess of tiny, seemingly indistinct spherical renditions of a common scene. To assemble this raw data into a result deserving of the name "photograph" requires a unique process. Think of it as the plenoptic equivalent of developing film.
Lytro's software is based on Ng's doctoral research in viable methods for plenoptic image capture. The device combines each of the microlens perspectives on the scene into a single result. Because each microlens only records visual data on a small segment of the digital sensor, a Lytro image is much lower in resolution than a typical digital photograph. But in exchange, the Lytro produces not one image, but a set of virtual cameras capable of rendering a whole field of possible images, which can be configured and combined via software.
So what, though? What can you do with a light field photograph that you can't do with an ordinary one?
In the computer science research community, plenoptic photography is often suggested as a darkroom technique meant to assist the photographer in the production of a final, traditional, static image. And such a use case is very much possible with Lytro's software, even if the resulting static image is mostly good for screen display or small prints due to its low resolution (1080×1080 pixels) compared to today's consumer digital cameras. But Lytro's intended use is surprising: instead of using the light field data to "develop" a final image, they offer a Flash-based web viewer which allows a user to actively interact with the image, manipulating it live.
An aesthetically unremarkable light field photograph by the author
But how does one interact with a light field photo? For one part, such images are refocusable in post-processing. Go ahead and try it with the image above, changing the focus to the different tealight candles on the table. In his dissertation, Ng discusses a common case in which such a scenario would be desirable. When taking a portrait, it's common to use a large aperture to produce a shallow depth of field to isolate the subject. But given such a narrow margin of error, it's easy to misfocus due to subject or photographer movement. In this case, the decisive moment might have been captured — a particular facial expression — but focused too far: at the ear instead of the eye, for example. A light field photographer could correct this fault and produce the desired focal plane through a virtual camera in the Lytro digital darkroom.
This sort of example has given Lytro's the reputation of a "focus-free" camera, but that's not really an accurate depiction. After all, Cartier-Bresson's Leica images were largely focus-free, since he zone-focused at smaller apertures to insure that most of the scene would be in focus anyway, allowing the photographer to concentrate on anticipating and capturing the decisive moment. Likewise, point-and-shoot cameras and many cell phone cameras have small focal ratios thanks to very wide lenses, making them shoot almost an entire scene in focus anyway. Focus-free is different from refocusable.
The result offers a kind of visual pun or reveal that forms the current Lytro aesthetic, for better or for worse.
Even given that caveat, Lytro light field images aren't fully refocusable — that is to say, all the possible virtual camera configurations in a living negative don't correspond with all the possible real focus configurations of all possible traditional cameras. This is because the light field the Lytro captures is not in the world, but in the camera's dark chamber.
That means that the light field itself is partly a result of the camera's primary lens focal point. Even though the lens can open up to f/2 up to 300mm at its longest zoom, closer focusing still produces a shallower depth of field. The result is that Lytro photos software-focused to objects in the foreground produce shallow depth of field all throughout the image, while those with objects further in the distance produce less depth of field. For this reason, many of the more effectively "refocusable" Lytro images deploy a kind of one-two punch delivered by an object in focus in the very near foreground, which occludes some other object in the background. Clicking on this far object brings it into focus as a kind of visual punch-line.
For example, this image of a Scrabble board is addressed to someone in the foreground. The tiles in the background bokeh are clear enough for the viewer to discern them, and clicking on them shifts focus to reveal a message. The result offers a kind of visual pun or reveal that forms the current Lytro aesthetic, for better or for worse.
Images focused at a medium distance tend to deplete focal distinction due to the increased natural depth of field at further focused distances. This image of a girl playing with a bubble toy illustrates the effect:
The image is set to load at a very close focal distance, that of the bubble gun (A Lytro "living image" can be configured to choose any virtual camera as a starting point of focus). But clicking on the bubbles in the middle of the frame displays microlens data focused on a farther distance from the "natural" point of focus, an image whose native depth of field is simply greater. The resulting action just isn't very interesting. Cinematic techniques like focus in, focus out transitions have set conventions for disclosing a subject over time. But in Lytro's compromise with this filmic technique, the promise of revelation implied by the primary, out of focus portion of the image falls flat. It turns out to be just a mediocre snapshot.
While the effect of natural focal distance can be reduced by zooming the camera's optical zoom and thus bringing far objects nearer together, the idea of a camera whose focal point can be moved arbitrarily doesn't quite correspond with what the Lytro is really capable of.
Many Lytro images turn out this way: looking at a given scene, the photographer assumes that any object the human eye can distinguish can be resolved as a point of focus for the camera. But that's not how the camera sees things. Many living pictures feel like dolled-up versions of ordinary snapshots wrapped in a fancy Flash viewer and a the rhetorical cape of a tech demo.
When the technique works, though, it can deliver a satisfying surprise.
Given the optical circumstances that make most Lytro shots more like two-focus images rather than refocusable images, and considering the square format and small size of the resulting interactive "living" picture, the result is something like a two-frame photographic cartoon, but with both frames set atop one another. At its worst, such a format descends into kitsch, as Abe's Scrabble greeting card above exemplifies. But at its best, this design forces a viewer to consider where to look in an image and why to look there, and thus produces a strange brew of charm and disappointment. If the street photograph seeks the sublime hidden in the everyday, the Lytro living picture seeks the everyday hidden in the everyday, and then reminds you that such obscurity is mundane as well. In an age of irony, the Lytro offers a welcome earnestness. Yet, lest we yearn for sincerity too much, its images step in to remind us that candor is actually kind of tawdry, and maybe we want to stick with irony.
All that said, the Lytro's capabilities aren't fixed, so we must take care not to pass judgement on its aesthetic virtues prematurely. The camera captures the entire light field, and interpolation and display of that data is handled after the fact in software. For this reason, Lytro can update not only its camera firmware but also its desktop and web software to reveal new features of light field postprocessing — even for images taken before such overhauls.
Recently Lytro rolled out the first such update, adding a perspective shift and what they call "living filters." Perspective shift takes advantage of the fact that the microlens array records images from slightly different perspectives across the focal plane. When reassembled in software, this makes it possible to shift the perspective slightly, which Lytro implements with a click-and-drag interface. Try it with the flowers below:
The result is reminiscent of lenticular printing, a technique sometimes used on collectible cups, Cracker Jack prizes, and theater teaser poster to give a static image the appearance of motion. Unfortunately, like lenticular designs Lytro's perspective shift wears out its novelty quickly. Lenticular images are primarily used as visual piques; an apparently static image seeming to move as a viewer passes by a poster or turns a cup or a card in her hands. They are often used in advertisement or cover images because they help set an image in visual relief against the background noise of a world full of images.
But as a general photographic application, perspective shift feels precious, like multiple-exposure and high dynamic range (HDR) imaging. Lytro's living filters follow suit. they aren't much more than the most rudimentary Photoshop filters applied to the light field data, although the ability to blend different filters at different focal points offers some novelty (try the shot below for an example). The company may have felt a need to compete with Instagram and its ilk in the filterwars, but the addition feels like a mismatch; a serious technological and critical intervention in the history of photography covered in me-too hipster sprinkles.
While the Lytro software isn't currently capable of greater feats, Ng's dissertation discusses at least two other uses of light field photography proven in the research lab. The first is just an implication of the already-released perspective shift feature. Given that the microlens array can see the same scene from different perspectives, light field data can be used to create a 3D stereogram from a single exposure.
The second use, adjustable depth of field, is more complex. Given that the microlens array is capable of capturing all possible planes of focus within the dark chamber light field, computational blending can allow a virtual darkroom program to combine different depths of fields at different points in an image, for example, increasing the depth of field in the background while also retaining a high depth of field in the foreground for an image entirely in focus even if exposed at a large aperture. The computational effort and custom input required to accomplish this feat probably relegates it to a the farther future, perhaps as a kind of Photoshop depth of field "brush" capable of increasing and decreasing focus depth like the dodge and burn tools can do with exposure.
Ren Ng
Lytro's ability to update its software has pros and cons for the budding light field photographer. For one part, it's refreshing to find a Silicon Valley company devoted to a truly novel hardware and software solution that suggests actual improvements over time, rather than just another me-too social network app with photo filters. But for another part, Lytro's software puts a lot of control in the hands of Lytro, Inc., and a lot less control in the hands of the photographer. While the software that ships with the camera always allows users to export to a static JPEG image, the Flash-based "living" image is hosted on Lytro's servers and subject to changes in its software and policies. Given today's tolerance for cloud-based services, this quirk may not bother some photographers, but others may squint at the idea that a $400 camera doesn't produce files that can be arbitrarily saved, archived, uploaded, or shared.
One can imagine more lurid future capacities for Lytro, too. For example, its methods might exert a lasting impact on optical design if and when light field photography becomes a more widespread practice. Today, lens design is highly commodified but optimized for optical performance. Given that light field cameras exert most of their optical effort in software rather than on ground glass, it might be possible to change the physical design of optical lenses to focus less on correcting aberrance along a single focal plane, and more on capturing as much light as possible. Imagine, for example, a flatter, thinner lens with an effective aperture of f/1 or even larger at all focal lengths. This design would be impossibly large, heavy, and distorted when used with a conventional, single focal plane camera, but given software distortion corrections at the per-microlens level, such a lens might be feasible for some future light field camera. Such musings are purely speculative at this stage, but they do open one's eyes to a hypothetical future in which light field photography becomes the standard, relegating 35mm photography to historical time and aesthetic choice, as the latter did to the view camera.
If nothing else, the Lytro should force traditional photographers to ask what they think of such a future for their art. Some will see Lytro as a newfangled gimmick, one that promises to fix a "problem" that isn't a problem at all, but a deliberate aesthetic practice. They woudn't be wrong.
Others will see the promise in light field photography, but scratch their heads at Lytro's insistence on making the living picture an end result rather than a digital darkroom service in the interest of helping serious photographers make adjustments in focus, depth of field, and even perspective when producing static images. They wouldn't be wrong either.
Others still will embrace Lytro as a new, feral form of photography that is entirely unproven, its galleries full of ghastly specimens that invite new expeditions to find the beguiling ones. And they too wouldn't be wrong.
But perhaps the fairest and most accurate characterization of Lytro is that of a signal: light field photography is a thing now. At least it's a thing weirder and more beguiling than Instagram. For now, it's a thing that doesn't know quite what it is: part underdog consumer electronics gadget, part nascent social photo network, part zygotic imaging medium.
As a photographic format, the light field camera doesn't really upset or replace Cartier-Bresson's foundational observation about photographic practice: making still images is a process of finding the right moment to hold still forever. There are more and less decisive images, but every photograph must contend with the crisis of choosing such a moment, whether by intention or chance.
Lytro condenses other photographic circumstances, focus and depth of field, and creates a second axis of choice and accident that intersects with the decisive moment. Cartier-Bresson saw photography as a temporal activity — its spatial counterpart was created by the photographer's own movement, framing, and the accidents of his or her surroundings. Ray and Charles Eames-like, Ren Ng zooms in to a tinier, more obscure aspect of photography: the dark chamber, the space between the lens and the emulsion. For a century now, while photographers have scurried up and down streets, leaned against buildings, propped equipment atop benches, and mounted them to tripods, an infinity of light has gone unrecorded, unconsidered in that tiny box inside the camera. In addition to the decisive moment, photography might now also consider the decisive focal plane, the best vector to follow to find the image, not just the best sidestreet to walk to find its subject.