Florian Kainz starts a little story with one of his photos that he took on a full moon night last year. He took a shot of the Golden Gate Bridge with the whole city in the background by his professional rig – a Canon 1DX, Zeiss Otus 28mm f/1.4 ZE and a tripod of course.
Like anyone, Florian thought the photo was worth showing to others for some appreciations may be. He showed it to his then-teammates at Gcam – Google’s Research team that focuses on computational photography. They challenged him to re-take the same shot with a smartphone camera instead of a DSLR.
That’s actually when Florian got a job to do but a little about Gcam is that, it’s the team behind HDR+ mode in the camera app on Nexus and Pixel smartphones. If you didn’t know HDR+ “allows you to take photos at low-light levels by rapidly shooting a burst of up to ten short exposures and averaging them them into a single image, reducing blur due to camera shake while collecting enough total light to yield surprisingly good pictures.”
Florian tells about the HDR+ feature because it’s not something to take a nighttime shot somewhat near the one from a professional camera. He, being a software engineer already, coded a camera app for the smartphone with features like manual control over exposure time, ISO and focus distance. On pressing the shutter button, the app takes up to 64 frames and saves the final shots in DNG raw format which are due for post processing on a computer.
He then needed to test his app if it works as he desired. He again chose a full moon night to have some light through the smartphone camera sensor. Florian used Nexus 6P and shot a burst of 32 four-second frames at high ISO 1600. Every time, he shot an additional burst of 32 black frames by covering the camera lens with an opaque tape. He got back in the office and started working with those RAW files. The individual frames were very grainy as you would expect from a smartphone camera. but computing the mean of all those frames cleaned up most of the grain, and subtracting the mean of the additional black frames removed faint grid-like patterns caused by local variations in the sensor’s black level. The photo you might be thinking is down below.
After having satisfying results he wanted to bring the stars in the sky above. Sure in above shot a thin layer of clouds conspired with the bright moonlight to make the stars nearly invisible, but on a clear night a two or four second exposure can easily capture the brighter stars. That’s only a technical note from a few Florian wanted to tell while writes.
Interestingly, next he tried in the dark night, unlike under the full moon. He used the Pixel smartphone – currently the top camera smartphone. Expecting more noise in the individual shots, he took burst of 64 frames this time. The final image after processing is what you can see below.
Now, what’s unbelievable from a smartphone camera is the Milky Way – far from the city lights, no moon at all, no artificial light source but only the sparkling stars on the dark sky. Pixel smartphone with 64 two-sec exposures at ISO 12800 and corresponding 64 black frames actually can impress by producing a shot with recognizable milky way.
With no doubt, the experimental results are too good to be from a smartphone camera. At 9 to 10 megapixels the resolution of these pictures is smaller than most DSLR cameras but image quality is surprisingly good.
Sure in above shots, Florian had to do a lot of post processing but the Pixel smartphone or Nexus 6P camera cannot handle exposure times longer than two seconds. With longer exposures the camera could lower the ISO setting, which would significantly reduce noise in the individual frames right in the phone. In result the post processing will also lower down.
This is just one of the few technology and software advancements we are heading towards. Hope to see the smartphone camera standing at a point where you didn’t believe it would. You can read more about above photos along with their full resolution shots. Here’s a Google Photos album with more photos which were produced with the same technique.