The lower the f-number, the wider the aperture and thus more light is able to pass through. Differences in f-number aren’t immediately obvious though, as double the f-number doesn’t equate to half the light gathering area (one stop less). Instead, due to the circular nature of an aperture, double the f-number is a two stop difference, providing one quarter the light gathering area.
The 20.7-megapixel, 1/2.3" Sony Exmor RS sensor in the Sony Xperia Z2, paired with an f/2.0 lens. Smartphone cameras typically use apertures ranging from f/2.0 to f/2.4, which are both wide in the overall camera ecosystem, but there are big differences between the two. f/2.4 is a half stop less than f/2.0, therefore an f/2.0 lens transmits 50% more light to the sensor. This can have a significant effect on low-light performance, with f/2.0-lensed smartphones typically producing stronger results than their f/2.4 counterparts. The difference in f-number doesn’t just affect light gathering properties. A lens with a higher f-number has wider depth of field, typically sharper images, less prevalent chromatic aberration (colored fringes in areas of a photo with high contrast) and weaker bokeh (pleasant blur as a result of defocused areas outside the depth of field range).
An average amount of bokeh from a smartphone camera. Taken with a 16-megapixel, 1/2.6" Samsung Galaxy S5 at ISO 40, 1/200s, f/2.2 Here we find another trade-off. In some situations, shallower depth of field and strong bokeh is preferred - especially when shooting subjects up-close or in macro mode - as it places the focal point of the image squarely on the subject rather than the background. DSLRs are particularly good at producing pleasant bokeh with a good lens; on smartphones the effect is less noticeable, but still present comparing f/2.0 and f/2.4 lenses. While chromatic aberrations and sharpness are issues with wider apertures (at times the HTC One M8’s f/2.0 lens can produce images with noticeable chromatic aberrations), it always falls second place to low light sensitivity and depth-of-field. This is why, in nearly all circumstances, wider apertures are preferred over smaller apertures. Unfortunately wide-aperture lenses are more complex and more expensive to produce, which is why not all smartphones manufacturers use them.
Image Stabilization
Smartphones are almost always used handheld when taking images, which is why image stabilization is crucial. There are two main forms: optical image stabilization (OIS), and electronic image stabilization (EIS). Generally speaking, OIS is preferred as it gives better results, but is more expensive and more complex to integrate. OIS works by placing the camera sensor or a lens module inside a stabilized rig. Using information from a gyroscope placed near the camera, the entire rig is shifted with electromagnets to counteract the physical movement of the device. This keeps the optical elements in the same relative position while the camera gently shakes in a users’ hand.
Harnessing manual mode with Nokia Camera and OIS. Taken with a 20-megapixel, 1/2.5" Nokia Lumia 1520 at ISO 100, 2.0s, f/2.4 EIS works without the assistance of any additional hardware. For still images, typically an EIS system will boost the sensor’s sensitivity so a faster shutter speed can be used, reducing blur. Other EIS systems I’ve seen will take a burst shot at normal sensitivity, using or merging together the least blurry images. For video, extra pixels are used outside the normal video frame, providing a buffer as the camera is moved around to reduce the effects of motion. A good OIS system will give a camera an advantage of a stop or more. This means that a slower shutter speed can be used that, without introducing blur into the image, allows twice the light to hit the sensor than otherwise. Some phones that integrate OIS into their camera modules include the original HTC One, the LG G2 and G3, and several Nokia Lumia smartphones such as the 1020, 930, and 1520.
Focus, Metering, ISO, White Balance and Shutter
On the more software-side of the camera system, these five terms are very important to getting good looking, accurate photos. Unlike professional cameras, smartphone cameras typically focus using contrast detection, which is entirely done on the software side by shifting the lens until there’s maximum contrast between adjacent pixels. Professional cameras typically use the quicker and more accurate phase detection method, which the hardware of a smartphone camera doesn’t support.
The LG G3 has a 13-megapixel 1/3.06" Sony IMX135 sensor with OIS and a unique laser autofocus system. Not all smartphones rely on contrast detection: the LG G3, for example, has a laser-assisted autofocus system that measures the distance from the subject to the lens, before setting the focus to mach. This autofocus system is quicker than contrast detection, and more accurate. Metering is entirely done using the sensor, again unlike professional cameras which pair the sensor with a light meter and other hardware to get better results. The smartphone’s camera software examines the preview and sets ISO, white balance, exposure, shutter speed and vibrance accordingly. Typically a center-weighted metering mode is used, which adjusts settings with a preference of getting the image correctly metered in the center. Spot metering can also be used, where it meters based on where you tap on the display, as can matrix metering, which uses a complex algorithm to estimate the correct settings to use. ISO refers to the ‘speed’ of the camera’s ‘film’, in the case of digital cameras telling us how sensitive the photodetectors are set to be. High ISOs capture lighter images through increasing the signal amplification, but are more susceptible to grain, especially so on a smartphone sensor. With such a small sensor, ISOs of 400 and above typically produce photos with noticeable grain, which is why they’re reserved for low light. In most conditions you should attempt to photograph with the lowest possible ISO.
Comparing the big four Android smartphones in good lighting. Click for 100% crops. White balance is often the biggest point of failure when it comes to correctly metering images, especially indoors. Again, the preview is used to estimate what color should be white, which sets the tone for the entire image. Some camera apps give you full manual control over white balance, which can be key in getting an accurate photo from your smartphone’s camera. The shutter on most smartphone cameras is entirely electronic, rather than mechanical like on DSLRs. Having a mechanical shutter can simplify the sensor, improving its performance, but due to build quality and size reasons, they’re seldom seen on smartphones. Electronic sensors are perfectly adequate in practical use, and have the advantage of being very fast.
Video
Video is an integral part of the smartphone camera, captured by taking essentially a burst shot of images at the resolution and frame rate of the video. The maximum supported resolution and frame rate of a camera is always restricted by both the circuitry of the camera model itself, and the image signal processor on the SoC’s side. Recording video is extremely bandwidth intense, and all the processing is done off the camera hardware: converting the raw input to H.264 is done by the video encoding block on the SoC after it passes through the image signal processor. This is why you don’t see crazy frame rates from smartphone cameras, because the hardware simply can’t support it. What sort of bandwidth are we talking about? Sony’s high-end IMX214 sensor, used in the OnePlus One, supports maximum output speeds of 2.4 Gbps, enough to support transmitting raw 4K data at 30 FPS (which requires 1.9 Gbps). A high-end smartphone in 2014 typically supports recording 4K (3840 x 2160) videos at 30 frames per second, which requires a sensor size of at least eight megapixels: this explains why the feature is missing from the HTC One M8 with its four megapixel camera. Other supported recording modes typically include 1080p at 60 frames per second, and 720p at 120 frames per second.
Manufacturers
The smartphone sensors I most often run into in handsets are from manufacturers such as OmniVision (parts designated OVxxxx), Sony (IMXxxx) and Samsung (S5Kxxx), which are used in everything from the high-end iPhone to the entry-level Moto E. Nokia also makes camera modules, although these are used exclusively in their handsets. ST Microelectrics sensors (ST VDxxxx) also crop up occasionally. The sensor is sometimes paired with a lens by the camera manufacturer to provide an easy slot-in module. At other times, a third-party will provide the lens system and potentially an optical stabilization rig; or the smartphone OEM will develop their own to use.
A 16-megapixel 1/2.3" OmniVision OV16820 CMOS sensor Over the past few years having hands-on time with many smartphones, I tend to find Sony and Samsung’s sensors to be the highest quality, which is why they are more prevalent in high-end devices. The entry-level and mid-range is almost exclusively dominated by OmniVision and their low-cost cameras, although from time to time they crop up in flagships: the HTC One M8 is a great example of this. Unfortunately camera hardware isn’t something you can simply swap out of a smartphone. It’s up to the designers to choose a camera OEM to deliver the parts for the final product, and the only way to change the camera is change the smartphone as a whole.
