[FoRK] PiCam: An Ultra-Thin High Performance Monolithic Camera Array
Stephen D. Williams
sdw at lig.net
Tue Oct 8 22:53:20 PDT 2013
> We present /PiCam/ (Pelican Imaging Camera-Array), an ultra-thin high performance monolithic camera array, that captures light
> fields and synthesizes high resolution images along with a range image (scene depth) through integrated parallax detection and
> superresolution. The camera is passive, supporting both stills and video, low light capable, and small enough to be included in
> the next generation of mobile devices including smartphones. Prior works [Rander et al. 1997; Yang et al. 2002; Zhang and Chen
> 2004; Tanida et al. 2001; Tanida et al. 2003; Duparre ? et al. 2004] in camera arrays have explored multiple facets of light field
> capture - from viewpoint synthesis, synthetic refocus, computing range images, high speed video, and micro-optical aspects of
> system miniaturization. However, none of these have addressed the modifications needed to achieve the strict form factor and image
> quality required to make array cameras practical for mobile devices. In our approach, we customize many aspects of the camera
> array including lenses, pixels, sensors, and software algorithms to achieve imaging performance and form factor comparable to
> existing mobile phone cameras.
> Our contributions to the post-processing of images from camera arrays include a cost function for parallax detection that
> integrates across multiple color channels, and a regularized image restoration (superresolution) process that takes into account
> all the system degradations and adapts to a range of practical imaging conditions. The registration uncertainty from the parallax
> detection process is integrated into a Maximum-a-Posteriori formulation that synthesizes an estimate of the high resolution image
> and scene depth. We conclude with some examples of our array capabilities such as post-capture (still) refocus, video refocus,
> view synthesis to demonstrate motion parallax, 3D range images, and briefly address future work.
On 5/3/13 4:01 PM, Stephen D. Williams wrote:
> Apologies, too enthusiastic not to post.
> 5/03/2013 @ 4:07PM |850 views
> The Camera In Your Next Smartphone Could Have 16 Lenses, Almost Magical Powers
> Pelican Imaging's array camera inside a Qualcomm reference tablet
> Pelican Imaging's 16-lens array camera offers low-noise, focus-free imaging in a small and affordable package. (Image courtesy of
> Under normal circumstances, the product pitch you're about to read would have to be dismissed as too good to be true. Today,
> however, now that a little startup called Pelican Imaging has banked $20 million of investment from Nokia and Qualcomm QCOM +1.4%,
> the outlandish claims it makes about its 16-lens "array" camera deserve to be taken seriously.
> Pelican's funding success has been covered pretty heavily over the past few days, especially by Bloomberg and this writer's
> articles at Engadget, but what hasn't been fully explained is why the company is such a smart investment for those who have moved
> quickly enough to grab a slice.
> Nokia's Bet On Windows Phone Looks Like A Good One So Far - Trefis Team Trefis Team Contributor
> Let's start with value of the technology, which lies in Pelican's software at least as much as in its hardware designs-- in fact,
> the company intends to specify hardware requirements for others to manufacture, rather than getting its own hands dirty.
> Pelican's CEO, Chris Pickett, describes his company's invention as "fundamentally different" to anything else. Not only does it
> have 16 lenses, it effectively contains 16 separate cameras providing 16 streams of data, which Pelican's software interprets into
> a single image. These sub-cameras share some key components, but the fact that they are isolated from each at the point of image
> capture allows for some amazing photographic abilities.
> First and foremost, the 4×4 grid of sub-cameras creates a 'plenoptic' system that captures an 'image' (albeit not a viewable one,
> at this stage) representing many different focal planes simultaneously. Once the software has worked its magic, the user can
> choose which part of an image they would like to focus on --- i.e., they can focus the image even after it has been taken.
> Pelican Imaging sample image
> Pelican Imaging's software will provide sharp focus on any subject you wish, or all at the same time, thanks to its plenoptic
> Replacing the traditional need for focusing could make photography much easier and offer a level of creative freedom that was
> previously limited to much larger contraptions, such as the Lytro camera (another plenoptic product). But this approach also has a
> major benefit for manufacturers: the absence of a focus system means that Pelican's camera has no moving parts, making it "half as
> thick as state-of-the-art competitors", according to Pickett, as well as bringing it into the normal realm of costs, estimated at
> between $18 and $20 per module.
> A second advantage to having 16 eyes, rather than one, is that you can build a highly accurate depth map of a scene, allowing the
> user to edit different people or objects separately without damaging their surroundings-- or what Pickett describes as
> "Photoshop-level editing, non-destructively, with your finger on a cellphone or tablet." When shooting 1080p video, the camera
> could even correct for camera wobble separately at each plane of movement, offering powerful digital stabilization.
> Finally, each sub-camera captures just one color --- red, green, or blue --- instead of trying to cope with all three. This
> reduces "color cross-talk" interference and hence, Pickett claims, delivers better image quality in low light than any existing
> smartphone camera.
> Now, this is where the business angle comes into play, because all these photographic capabilities have one thing in common: they
> require vast amounts of computing power. That fact alone is enough to explain why Qualcomm is a keen backer: above all, a chip
> maker of that size depends on consumers seeing a genuine reason to upgrade to smartphones or tablets containing the latest and
> most expensive processors. Pelican's software is designed to exploit every part of a cutting-edge Snapdragon 800 mobile chip, from
> the CPU to the GPU and even the DSP, and indeed Qualcomm is already flaunting a reference tablet (shown at the top of this
> article) that has a functioning Pelican camera built into it.
> By now, it should also be clear why Nokia wanted to secure itself a stake in the Pelican Imaging adventure. Having manufactured
> the PureView 808 with a ground-breaking 41MP camera sensor, and the Lumia 920 with sophisticated "floating lens" stabilization,
> the Finnish manufacturer is banking on camera hardware to help sell its coming generations of phones. Pelican has already
> confirmed that its system could work in tandem with PureView designs, yielding some "pretty exciting possibilities."
> In fact, the real question is why Nokia used its investment arm --- Nokia Growth Partners (NGP)-- to claim a stake, rather than
> just snapping up the entire company as it recently did with Scalado, another imaging-related startup.
> Predictably, Pickett wouldn't answer this question or name any device manufacturer he's working with. He admitted that his
> company's latest reference design has some weaknesses that may make OEMs cautious: namely, a low resolution, low quality lenses
> and out-of-date pixels. But at the same time he dropped some strong hints that a deal has already been struck with at least one
> OEM, and that a Pelican-equipped smartphone will reach the market ("on shelves and ready to buy") as early as next year.
> Director and head of smart devices, Nokia Indi...
> The PureView 808 proved that Nokia has a knack for doing crazy things with smartphone cameras. (Image credit: AFP/Getty Images via
> For its part, Nokia provided the following statement, which at least suggests that the people in charge of building Lumia phones
> for 2014 have been paying close attention to NGP's new relationship with Pelican Imaging:
> "NGP is fully part of the Nokia family so there is a regular discussion between our NGP team and our engineers and product
> developers at any given time in a range of different areas. Together they look at some of the more interesting opportunities out
> there. However, we can't comment on whether or indeed when, technologies from invested companies may feature in our products.
> You'll know that the area of imaging and photography is strategically very important to Nokia so clearly keeping tabs on the new
> emerging technologies out there is simply good business sense."
> Whatever happens next, the "good business sense" in the Pelican Imaging investment speaks for itself --- so long as the technology
> lives up to what has been claimed of it. In terms of lingering uncertainty, the fact that Nokia didn't buy Pelican outright could
> simply be due to the smaller company's business model, which is more about making software for a plenoptic camera rather than
> building or licensing hardware. Regardless, this newly-enriched startup has emerged as one of the most exciting things to happen
> in the smartphone industry so far this year, and it is definitely one to watch.
> On 5/2/13 5:09 PM, Stephen Williams wrote:
>> A little redundant, but good coverage.
>> *Pelican Imaging's 16-lens array camera coming to smartphones next year*
>> *Pelican Imaging demos Lytro-like refocusing in an inexpensive, slim smartphone-friendly module*
>> Nokia's VC arm poised to invest in camera array firm Pelican Imaging*
>> *Nokia to invest in 'array' mobile cameras that use small lenses to capture big images***
>> *Nokia investing in camera tech that could change phones forever*
>> *Nokia to invest in array camera start-up Pelican Imaging*
>> *Nokia Invests in High-Resolution Cameras to Woo Apple Buyers (1)*
>> *Pelican Imaging Secures $20M In Funding From Qualcomm, Nokia Growth Partners, And Current Investors*
>> *Nokia Bets on Camera Tech to Boost Phone Market Share*
>> **Nokia Invests in Mobile Camera Firms in Bid To See Like a Bug*
Stephen D. Williams sdw at lig.net stephendwilliams at gmail.com LinkedIn: http://sdw.st/in
V:650-450-UNIX (8649) V:866.SDW.UNIX V:703.371.9362 F:703.995.0407
AIM:sdw Skype:StephenDWilliams Yahoo:sdwlignet Resume: http://sdw.st/gres
Personal: http://sdw.st facebook.com/sdwlig twitter.com/scienteer
More information about the FoRK