[FoRK] PiCam: An Ultra-Thin High Performance Monolithic Camera Array

Stephen D. Williams sdw at lig.net
Tue Oct 8 22:53:20 PDT 2013

> We present /PiCam/ (Pelican Imaging Camera-Array), an ultra-thin high performance monolithic camera array, that captures light 
> fields and synthesizes high resolution images along with a range image (scene depth) through integrated parallax detection and 
> superresolution. The camera is passive, supporting both stills and video, low light capable, and small enough to be included in 
> the next generation of mobile devices including smartphones. Prior works [Rander et al. 1997; Yang et al. 2002; Zhang and Chen 
> 2004; Tanida et al. 2001; Tanida et al. 2003; Duparre ? et al. 2004] in camera arrays have explored multiple facets of light field 
> capture - from viewpoint synthesis, synthetic refocus, computing range images, high speed video, and micro-optical aspects of 
> system miniaturization. However, none of these have addressed the modifications needed to achieve the strict form factor and image 
> quality required to make array cameras practical for mobile devices. In our approach, we customize many aspects of the camera 
> array including lenses, pixels, sensors, and software algorithms to achieve imaging performance and form factor comparable to 
> existing mobile phone cameras.
> Our contributions to the post-processing of images from camera arrays include a cost function for parallax detection that 
> integrates across multiple color channels, and a regularized image restoration (superresolution) process that takes into account 
> all the system degradations and adapts to a range of practical imaging conditions. The registration uncertainty from the parallax 
> detection process is integrated into a Maximum-a-Posteriori formulation that synthesizes an estimate of the high resolution image 
> and scene depth. We conclude with some examples of our array capabilities such as post-capture (still) refocus, video refocus, 
> view synthesis to demonstrate motion parallax, 3D range images, and briefly address future work.


On 5/3/13 4:01 PM, Stephen D. Williams wrote:
> Apologies, too enthusiastic not to post.
> http://www.forbes.com/sites/sharifsakr/2013/05/03/the-camera-in-your-next-smartphone-could-have-16-lenses-almost-magical-powers/
> 5/03/2013 @ 4:07PM |850 views
> The Camera In Your Next Smartphone Could Have 16 Lenses, Almost Magical Powers
> Pelican Imaging's array camera inside a Qualcomm reference tablet
> Pelican Imaging's 16-lens array camera offers low-noise, focus-free imaging in a small and affordable package. (Image courtesy of 
> Engadget.)
> Under normal circumstances, the product pitch you're about to read would have to be dismissed as too good to be true. Today, 
> however, now that a little startup called Pelican Imaging has banked $20 million of investment from Nokia and Qualcomm QCOM +1.4%, 
> the outlandish claims it makes about its 16-lens "array" camera deserve to be taken seriously.
> Pelican's funding success has been covered pretty heavily over the past few days, especially by Bloomberg and this writer's 
> articles at Engadget, but what hasn't been fully explained is why the company is such a smart investment for those who have moved 
> quickly enough to grab a slice.
> Nokia's Bet On Windows Phone Looks Like A Good One So Far - Trefis Team Trefis Team Contributor
> Let's start with value of the technology, which lies in Pelican's software at least as much as in its hardware designs-- in fact, 
> the company intends to specify hardware requirements for others to manufacture, rather than getting its own hands dirty.
> Pelican's CEO, Chris Pickett, describes his company's invention as "fundamentally different" to anything else. Not only does it 
> have 16 lenses, it effectively contains 16 separate cameras providing 16 streams of data, which Pelican's software interprets into 
> a single image. These sub-cameras share some key components, but the fact that they are isolated from each at the point of image 
> capture allows for some amazing photographic abilities.
> First and foremost, the 4×4 grid of sub-cameras creates a 'plenoptic' system that captures an 'image' (albeit not a viewable one, 
> at this stage) representing many different focal planes simultaneously. Once the software has worked its magic, the user can 
> choose which part of an image they would like to focus on --- i.e., they can focus the image even after it has been taken.
> Pelican Imaging sample image
> Pelican Imaging's software will provide sharp focus on any subject you wish, or all at the same time, thanks to its plenoptic 
> technology.
> Replacing the traditional need for focusing could make photography much easier and offer a level of creative freedom that was 
> previously limited to much larger contraptions, such as the Lytro camera (another plenoptic product). But this approach also has a 
> major benefit for manufacturers: the absence of a focus system means that Pelican's camera has no moving parts, making it "half as 
> thick as state-of-the-art competitors", according to Pickett, as well as bringing it into the normal realm of costs, estimated at 
> between $18 and $20 per module.
> A second advantage to having 16 eyes, rather than one, is that you can build a highly accurate depth map of a scene, allowing the 
> user to edit different people or objects separately without damaging their surroundings-- or what Pickett describes as 
> "Photoshop-level editing, non-destructively, with your finger on a cellphone or tablet." When shooting 1080p video, the camera 
> could even correct for camera wobble separately at each plane of movement, offering powerful digital stabilization.
> Finally, each sub-camera captures just one color --- red, green, or blue --- instead of trying to cope with all three. This 
> reduces "color cross-talk" interference and hence, Pickett claims, delivers better image quality in low light than any existing 
> smartphone camera.
> Now, this is where the business angle comes into play, because all these photographic capabilities have one thing in common: they 
> require vast amounts of computing power. That fact alone is enough to explain why Qualcomm is a keen backer: above all, a chip 
> maker of that size depends on consumers seeing a genuine reason to upgrade to smartphones or tablets containing the latest and 
> most expensive processors. Pelican's software is designed to exploit every part of a cutting-edge Snapdragon 800 mobile chip, from 
> the CPU to the GPU and even the DSP, and indeed Qualcomm is already flaunting a reference tablet (shown at the top of this 
> article) that has a functioning Pelican camera built into it.
> By now, it should also be clear why Nokia wanted to secure itself a stake in the Pelican Imaging adventure. Having manufactured 
> the PureView 808 with a ground-breaking 41MP camera sensor, and the Lumia 920 with sophisticated "floating lens" stabilization, 
> the Finnish manufacturer is banking on camera hardware to help sell its coming generations of phones. Pelican has already 
> confirmed that its system could work in tandem with PureView designs, yielding some "pretty exciting possibilities."
> In fact, the real question is why Nokia used its investment arm --- Nokia Growth Partners (NGP)-- to claim a stake, rather than 
> just snapping up the entire company as it recently did with Scalado, another imaging-related startup.
> Predictably, Pickett wouldn't answer this question or name any device manufacturer he's working with. He admitted that his 
> company's latest reference design has some weaknesses that may make OEMs cautious: namely, a low resolution, low quality lenses 
> and out-of-date pixels. But at the same time he dropped some strong hints that a deal has already been struck with at least one 
> OEM, and that a Pelican-equipped smartphone will reach the market ("on shelves and ready to buy") as early as next year.
> Director and head of smart devices, Nokia Indi...
> The PureView 808 proved that Nokia has a knack for doing crazy things with smartphone cameras. (Image credit: AFP/Getty Images via 
> @daylife)
> For its part, Nokia provided the following statement, which at least suggests that the people in charge of building Lumia phones 
> for 2014 have been paying close attention to NGP's new relationship with Pelican Imaging:
> "NGP is fully part of the Nokia family so there is a regular discussion between our NGP team and our engineers and product 
> developers at any given time in a range of different areas. Together they look at some of the more interesting opportunities out 
> there.  However, we can't comment on whether or indeed when, technologies from invested companies may feature in our products. 
> You'll know that the area of imaging and photography is strategically very important to Nokia so clearly keeping tabs on the new 
> emerging technologies out there is simply good business sense."
> Whatever happens next, the "good business sense" in the Pelican Imaging investment speaks for itself --- so long as the technology 
> lives up to what has been claimed of it. In terms of lingering uncertainty, the fact that Nokia didn't buy Pelican outright could 
> simply be due to the smaller company's business model, which is more about making software for a plenoptic camera rather than 
> building or licensing hardware. Regardless, this newly-enriched startup has emerged as one of the most exciting things to happen 
> in the smartphone industry so far this year, and it is definitely one to watch.
> sdw
> On 5/2/13 5:09 PM, Stephen Williams wrote:
>> A little redundant, but good coverage.
>> *Pelican Imaging's 16-lens array camera coming to smartphones next year*
>> http://www.engadget.com/2013/05/02/pelican-imaging-array-camera-coming-2014/
>> *Pelican Imaging demos Lytro-like refocusing in an inexpensive, slim smartphone-friendly module*
>> http://www.imaging-resource.com/news/2013/04/30/pelican-imaging-demoes-lytro-like-refocusing-in-an-inexpensive-slim-smartph
>> *
>> Nokia's VC arm poised to invest in camera array firm Pelican Imaging*
>> http://www.zdnet.com/nokias-vc-arm-poised-to-invest-in-camera-array-firm-pelican-imaging-7000014699/
>> *Nokia to invest in 'array' mobile cameras that use small lenses to capture big images***
>> http://www.engadget.com/2013/04/30/nokia-to-invest-in-pelican-camera-tech/
>> *Nokia investing in camera tech that could change phones forever*
>> http://www.fastcompany.com/3009004/nokia-investing-in-camera-tech-that-could-change-phones-forever
>> *Nokia to invest in array camera start-up Pelican Imaging*
>> http://news.cnet.com/8301-32973_3-57582070-296/nokia-to-invest-in-array-camera-start-up-pelican-imaging/
>> *Nokia Invests in High-Resolution Cameras to Woo Apple Buyers (1)*
>> http://www.businessweek.com/news/2013-04-29/nokia-invests-in-high-resolution-cameras-to-woo-apple-customers
>> *Pelican Imaging Secures $20M In Funding From Qualcomm, Nokia Growth Partners, And Current Investors*
>> http://online.wsj.com/article/PR-CO-20130430-916062.html
>> *Nokia Bets on Camera Tech to Boost Phone Market Share*
>> http://www.cio.com/article/732619/Nokia_Bets_on_Camera_Tech_to_Boost_Phone_Market_Share?taxonomyId=3234
>> *
>> **Nokia Invests in Mobile Camera Firms in Bid To See Like a Bug*
>> http://www.dailytech.com/Nokia+Invests+in+Mobile+Camera+Firms+in+Bid+to+See+Like+a+Bug/article31454c.htm
>> http://www.bloomberg.com/news/2013-04-29/nokia-invests-in-high-resolution-cameras-to-woo-apple-customers.html
>> http://www.theverge.com/2013/4/30/4285044/nokia-plans-pelican-imaging-array-camera-investment-says-bloomberg
>> http://www.livemint.com/Consumer/Jph7vyJjqd8OQmouHDwIMI/Nokia-invests-in-highresolution-cameras-to-woo-Apple-custom.html
>> http://gigaom.com/2013/04/30/nokia-to-invest-in-array-camera-outfit-pelican-imaging-report-says/
>> http://gizmodo.com/is-nokia-investing-in-lytro-style-camera-tech-for-phone-485522692
>> http://www.engadget.com/2013/04/30/nokia-to-invest-in-pelican-camera-tech/
>> http://www.stuff.tv/news/phone/news-nugget/future-nokia-phones-will-let-you-focus-after-taking-photos
>> http://www.zdnet.com/nokias-vc-arm-poised-to-invest-in-camera-array-firm-pelican-imaging-7000014699/
>> sdw
Stephen D. Williams sdw at lig.net stephendwilliams at gmail.com LinkedIn: http://sdw.st/in
V:650-450-UNIX (8649) V:866.SDW.UNIX V:703.371.9362 F:703.995.0407
AIM:sdw Skype:StephenDWilliams Yahoo:sdwlignet Resume: http://sdw.st/gres
Personal: http://sdw.st facebook.com/sdwlig twitter.com/scienteer

More information about the FoRK mailing list