MIT: Cameras with depth sensors are coming to mobile devices


Content

Depth sensor
Monochrome camera

Wide angle camera

Telephoto camera

What are the disadvantages

What are the prospects?

This time everything worked out. After the release of the LG G5 and Huawei P9 in 2020, Apple decided to respond to its competitors - the iPhone 7 Plus became the company's first flagship with a dual camera. As a result, almost all well-known companies came to dual cameras - now they are used by Samsung, Apple, Huawei, LG, Nokia, Xiaomi, Meizu, Lenovo (Motorola), Essential, ZTE and even small Chinese vendors like UMIDIGI, Doogee, Vernee and others.

HTC Evo 3D with dual camera, 2011
They all use different dual camera systems: some focus on a monochrome sensor, some on a wide-angle, and some on a telephoto lens. Trashbox decided to figure out what this or that duo of photomodules offers, how it works, and what are the advantages and disadvantages of the solutions used by manufacturers.

Fit more objects into the frame.

This may be the least creative use of a wide-angle lens, but it's still the most practical. Agree, there are often situations when you want to capture several objects at once in one frame or show simultaneously an object and the background on which it is located. There is not always an opportunity to leave. Another situation is shooting objects located on the table from the top point. Sometimes you have to rack your brains about what to stand on in order to become taller and at the same time it’s a shame to fit everything you need into the frame. In this case, turning on the wide-angle camera comes to the rescue: you remain in your place, but much more fits into the frame!

See also: 5 cool smartphones to buy on Aliexpress and how to save money on it

Both shots were taken from the same point. The first is for a standard camera, the second is for a wide-angle

Depth sensor

We'll start with the depth sensor, as this is the simplest form of dual camera system possible. In this case, the main camera is accompanied by a second sensor, the sole function of which is to create a three-dimensional map of the area of ​​the object being photographed. As you may know, a person sees the three-dimensional world in a two-dimensional projection, but is able to form a complete picture of the distance, shape, size and depth of surrounding objects. Our eyes are responsible for this, having stereoscopic ability due to different perspectives.

The depth sensor in the dual camera system works similarly to the main camera. Using the secondary module, the dual camera can roughly tell you how far away objects are in front of it relative to each other. This information is then used to separate the foreground object from the background.

An example of how the depth sensor works on HTC One (M8)
The main feature of a dual camera with this sensor is to create a realistic depth of field effect. The technology in its form comes from DSLR cameras with their large sensors and lenses, but smartphones cannot recreate the effect of the same shallow depth of field. The depth sensor first detects the boundaries of the foreground object and then applies a blur effect to the surrounding area. I note that the result obtained is very close, but illusory.

In theory and in practice, the depth sensor has its pitfalls. If the subject you are shooting does not have depth (for example, you may be shooting something flat), the camera will not be able to detect this and, along with the background, will blur the edges of the object. Even if the depth sensor does its job perfectly, the result does not look natural in any case. In DSLR cameras, the blur intensity increases with distance from the focus point. This is not the case with smartphones.

Dual camera systems with a special depth sensor are one of the rarest duos of photo modules. For the first time such a system was installed in the flagship HTC One (M8) - this is the most popular example. These days, the depth sensor is used in smartphones such as Huawei Honor 6X or Lenovo K8 Plus, that is, not in flagship models.

Intel unveils compact depth camera with solid-state lidar

Intel

Intel has unveiled a compact depth camera that combines a color camera and solid-state lidar. Such a camera allows you to simultaneously obtain color frames and depth maps - this can be useful for orienting robots in space and other applications, according to Intel's website. Sales of the camera will begin in April 2020, and the price will be $349.

The work of many robots, as well as all self-driving cars and many other devices, is based on spatial orientation, for which they usually use depth maps, which can be compiled in different ways. Often, a depth camera is used for this, which uses two conventional cameras to determine volume, or a projector with a camera, which allows one to understand the volume by projection distortions. For example, this method is used in the Microsoft Kinect sensor: it was originally intended for game consoles, but in addition to games it has become widespread among engineers (for example, it was recently used to create an acoustically levitating screen based on a balloon).

You can also use lidars to create depth maps, which provide “honest” depth data - they send laser beams and measure the time it takes for them to return. Such data has great accuracy, but the most popular lidar models currently cost several thousand or even tens of thousands of dollars, making them unaffordable for most research projects. Many companies are therefore developing a cheaper alternative in the form of solid-state lidar, but this technology is just beginning to come to market.

Intel's new off-the-shelf depth sensor, the RealSense LiDAR Camera L515, consists of a camera, a solid-state lidar, and a processing chip. The device allows you to receive two streams of images with a frequency of 30 frames per second: color frames from the camera with a resolution of 1920 by 1080 pixels and a depth map from the lidar with a resolution of 1024 by 768 pixels. The lidar's field of view is 70 by 55 degrees, and it works with objects at a distance of 25 centimeters to 9 meters. The measurement accuracy is up to one and a half centimeters at the maximum distance.

The device is adapted to work with popular platforms in the field of robotics, such as ROS and OpenCV, and is also equipped with USB-C for data transfer. The sensor is available for pre-order at a price of $349, and first shipments will begin at the end of April 2020.

Intel also produces regular depth cameras: it introduced two of them in early 2020.

Grigory Kopiev

Monochrome camera

A slightly more popular implementation of a secondary photomodule is a monochrome camera. In such a system, the main camera is complemented by a similar sensor, the peculiarity of which is monochrome shooting. Both cameras usually have the same sensors, aperture ratios, lenses and focusing systems. The only difference between the two is that the second camera lacks an RGB color filter. This means that a monochrome sensor cannot capture color information, but it can capture more light than a regular camera.

A dual camera with a monochrome sensor takes two photos at the same time. The operating principle of such a system is to combine images from both cameras into one image. As a result, the two combined images have more detailed information and a reduced noise level. In addition, you can take only black and white photos with a monochrome camera and get the highest image quality. The color module cannot provide such a result even after applying the B&W effect.

B&W mode of a monochrome camera on Huawei P9
One of the first smartphones with an additional monochrome camera can be called the flagship Huawei P9. Since then, the Chinese company has continued to improve its dual camera system and today it has the best monochrome sensor system. It is worth noting that Leica, a popular German manufacturer that produces premium cameras and optics, is involved in the development of dual cameras and processing algorithms for Huawei.

Wide angle camera

For the first time, a dual camera with a wide-angle lens debuted in the LG G5 smartphone early last year. LG has installed a 16-megapixel primary sensor (29 mm) with f/1.8 aperture and an additional 8-megapixel sensor (12 mm) with f/2.4 aperture in its new flagship. The 12mm focal length gives the secondary camera a significantly wider field of view, which captures a much wider area of ​​the scene without the need to deliberately move away from the subject.

The wide-angle camera is mainly used in LG flagships. Motorola recently equipped its new Moto X4 smartphone with such a system. The wide-angle lens brings a unique perspective to mobile photography that no other smartphone camera can offer. In addition to its practical value (such as capturing a large group of people at close range), a wide-angle sensor also allows you to create beautiful landscapes by capturing the entirety or at least most of the scene.

Wide-angle and ultra-wide-angle modes on the LG G5
The first generations of wide-angle cameras had their drawbacks. Users of the G5 and V20 complained about poor image quality in ultra-wide viewing angles and severe scene distortion in the corners. However, LG continues to steadily improve its dual camera setup with a wide-angle lens. In the latest flagship V30, the secondary lens has lost its main drawbacks - the image quality has been noticeably improved, and the distortion in the pictures has become significantly less.

With good implementation, such a dual camera system can actually be very useful when it comes to certain shooting scenarios. LG is going its own way, and that's good.

Login to the site

The range of focal lengths of smartphone cameras is continuously expanding, in all directions: increasing zoom and wider viewing angles are appearing. In this article we will talk about the possibilities a wide-angle camera offers. Using the HUAWEI P30 Pro as an example. The world of mobile photography is changing rapidly. Every year, smartphone cameras demonstrate greater capabilities in terms of shooting quality, and now a smartphone can safely be called a full-fledged photographer’s tool. Many famous photography masters actively use smartphone cameras, and some have even made a name for themselves in mobile photography. But nevertheless, until recently, when taking photos with smartphones, we were faced with significant limitations, and one of the main ones was the lack of variability in focal lengths. Older generation smartphones were equipped with a single camera with a fixed viewing angle. And if it was possible to narrow it, albeit “artificially”, through digital zoom, then expand it - in no way. The emergence of smartphones with multiple built-in cameras was a real breakthrough. In particular, owners of smartphones with an ultra-wide-angle lens, such as the Huawei P30 Pro, have more opportunities to realize their creative ideas. Let's see how it can be useful to a photographer.

1. Fit more objects into the frame.

This is probably the least creative use of a wide-angle lens, but it is nonetheless the most practical. Agree, there are often situations when you want to capture several objects in one frame or show both an object and the background on which it is located. There is not always an opportunity to leave. Another current situation is shooting objects located on a table from the top point. Sometimes you have to rack your brain about what to stand on in order to stand taller and fit everything you need into the frame. In this case, turning on the wide-angle camera helps a lot: you remain in your place, but much more fits into the frame!

2. Shooting in tight spaces.

In fact, this is a continuation of the previous topic. Often the space itself limits our ability to take a full-length photograph or capture several people in the frame, while we want to show not a fragment of the scene, but the plot as a whole, so that the person looking at the photo can understand what is happening and where, who is participating in the events. A wide angle may distort the proportions of people in the frame, but everything you need will fit!

3. Create a feeling of spaciousness.

Even if you are photographing in a not very cramped room, a wide angle will not be superfluous. Short focal length lenses are often used in interior photography not only to ensure that “everything fits” into the frame, but also to create the illusion of a large, spacious room. These are the kinds of pictures you can see in advertisements for hotels, restaurants, and apartments for rent. Therefore, if you want to impress the viewer by showing what a chic place you are in, it makes sense to use a wide-angle camera.

4. Landscape photography. Wide-angle and ultra-wide-angle lenses are traditionally used when shooting landscapes - they allow you to capture a large amount of space, interesting details into the frame, and show the versatility of the composition.

5. Accentuate the subject in the frame

There are different ways to focus the viewer's attention on a specific object in a photograph. Often, long-focus optics are used for this and the background is blurred as much as possible. The reception is good, but not always suitable. What if the background is important too? If it is important to show an object in the context of its environment? To prevent him from getting lost in the frame, photograph him with a wide-angle lens, placing him in the foreground. Thanks to optical distortions, an object, even if it is very small, will appear large in the photo. In this case, the surroundings will also remain in the frame, adding information to the picture.

6. Playing with scale

So, we have already mentioned the property of wide-angle optics to distort the proportions of objects: what is close to the lens will be large in the photograph, and more distant objects, even if they are larger in reality, will turn out small. This technique can be used creatively, playing with scale in the frame and creating unusual compositions. Don’t forget to choose the right angle! Try to shoot the object from the level of the surface on which it is located or slightly below - this will emphasize its size.

7. Cartoon portraits

Of course, you can use the property of wide-angle optics to “stretch” perspective in portrait photography. As a rule, we strive to ensure that the proportions of the person or animal in the frame are correct, but sometimes you can create a portrait in a humorous way by deliberately distorting the proportions. Try taking a photo with your smartphone's wide-angle camera from as close as possible. If you hold the camera directly in front of the model’s face, the nose will be as large as possible; if you shoot from above, the eyes will be as large as possible. If your character additionally makes faces on camera, you will get a finished cartoon.

8. Unconventional selfies

Continuing the portrait theme - everyone's favorite selfies. Everyone’s favorite, but already somewhat boring: there are few options - the face is a little on top, the face is a little on the side... But what if you approach the process more creatively? Turn on the wide-angle camera and timer mode. This way you can free up your hands and include them in the composition of the shot. Remember that the object closest to the camera will be as large as possible, and you can play on this.

9. Add speakers

Photos in which the eye moves rather than freezes at one point look more interesting. Diagonals help the eye move - these can be natural lines of architecture, roads, wires, etc. Parallel lines shot with a wide-angle lens appear rapidly converging in the frame, and seem to lead the viewer from the corner into the depths of the picture. Therefore, even a plot that is static in content will look livelier and more dynamic when using wide-angle optics.

Telephoto camera

We come to the last and most common option of all dual camera systems - telephoto. In this design, the primary module is connected to a second sensor with a telephoto lens. Telephoto is the exact opposite of a wide-angle camera system. It actually allows you to zoom in only on the subject being photographed, and not on the entire scene.

Since the release of the iPhone 7 Plus (the first Apple smartphone with a dual camera), manufacturers have turned their attention to the telephoto lens as a second camera for branded smartphones. Telephoto currently offers 2x optical zoom, meaning the secondary sensor in such a system has twice the focal length. OPPO unveiled an innovative dual camera system with a record 5x optical zoom at MWC 2020 in Barcelona, ​​but this solution is still far from commercial release.

Wide-angle (top) and telephoto (bottom) on the iPhone 7 Plus
The telephoto lens has many advantages. Firstly, what is most obvious is getting 2x optical zoom without loss in image quality. Smartphone photography upscaling has largely been digital until now, but with a telephoto lens you can quickly get "closer" to your subject with minimal loss of quality. And because digital zoom is now applied on top of 2x optical zoom, the results are much more effective.

Portrait mode with studio lighting feature on iPhone 8 Plus

Shooting with a telephoto lens has other advantages. Telephoto cameras are great for portraits because they have minimal distortion and capture the subject smoother than a wide-angle lens. Most manufacturers today also implement a background blur effect. The combination of a telephoto lens and background blur brings classic portrait photography almost to the level of a DSLR - the result is so impressive that not everyone will be able to see the difference. Of course, we should not forget about modern processing algorithms.

Why do smartphones need more than three cameras?

Reward accrued

This material was written by a site visitor and was compensated for.

For nothing. This is a short answer if you don't have time to read the whole article. But despite everything, the topic became very relevant after Samsung introduced the Galaxy A9 (2018) smartphone, which was the first on the market to have a quadruple main camera. It recently began selling in Russia at a price of 39,990 rubles, and for this money it has many worthy competitors, but none of them yet even have a triple camera. And this one has four at once. Moreover, in the coming weeks/months, the announcement of Nokia 9 with five lenses on the rear panel should finally take place. But is there any sense in them? Now let's try to figure it out.

What's missing from dual cameras?

announcements and advertising

2080 Super Gigabyte Gaming OC for 60 rubles.

Compeo.ru - the right comp store without any tricks

RTX 2060 becomes cheaper before the arrival of 3xxx

Ryzen 4000

series included in computers already in Citylink

The price of MSI RTX 2070 has collapsed after the announcement of RTX 3xxx

Core i9 10 series is half the price of the same 9 series

The price of memory has been halved in Regard - it’s more expensive everywhere

Huawei presented the first full-fledged smartphone with two cameras. We are talking about the flagship model of the 2020 model P9. Yes, there were other devices, but they became rather unsuccessful experiments. Be that as it may, it is now difficult to find a phone with a single camera. Even budget models costing 10,000 rubles have an additional module. All dual cameras can be divided into five configurations:

  • RGB sensor + telephoto lens;
  • RGB sensor + wide-angle lens;
  • RGB sensor + b/w sensor;
  • RGB sensor + sensor for night photography;
  • RGB sensor + sensor for depth analysis.

The latter is usually found only in cheap smartphones, since any other image sensors can take over its role without any problems. Moreover, all three generations of Pixel phones have a single camera and can perfectly analyze the depth of a scene using software. Since no one will install a telephoto lens or a wide-angle camera in devices costing $100, they get an auxiliary sensor with a resolution of 2 MP or 5 MP.

The main disadvantage of a dual camera is that users have to give up either a wide or telephoto lens. There is no point in talking about a monochrome sensor and a sensor for night photography, because... They are not very popular, which cannot be said about the first two solutions. In any case, if you buy a smartphone with a dual camera, you have to give up something.

How about a third camera?

Huawei P20 Pro is the first smartphone to have a triple camera. It was presented by the Chinese manufacturer on March 27 in Paris and became the best camera phone according to DxOMark, scoring 109 points. Whether a benchmark can be trusted, everyone must decide for themselves. Huawei P20 Pro offers three modules:

  • standard color 40 MP;
  • 8 MP telephoto lens;
  • monochrome 20 MP.

Their combination allows you to get beautiful photos at night. The telephoto lens offers 3x optical zoom without loss of quality, and when paired with a b/w sensor, it provides 5x hybrid zoom compared to a standard camera.

A few months later, it became known that Huawei would continue to collaborate with Leica, but instead of a black-and-white sensor, the Mate 20 flagship would have a wide-angle camera. Ultimately, this happened, but the Chinese company was ahead of LG. The South Korean manufacturer presented its flagship LG V40 ThinQ a few days earlier.

It became the first smartphone with five cameras thanks to a dual front camera, as well as the first camera phone that could combine an RGB sensor, a telephoto for 2x optical zoom and a wide-angle lens. Their resolution was 12 megapixels, 12 megapixels and 16 megapixels. And this combination significantly expands the list of options for using the camera.

It is worth recalling that between the announcement of Huawei P20 Pro and LG V40 ThinQ, two more smartphones with triple cameras were presented. In August, the OPPO R17 Pro was announced. It has two modules with a resolution of 12 megapixels (f/1.5-2.4) + 20 megapixels, as well as a Time-of-Flight (ToF) 3D camera for recognizing gestures and creating three-dimensional objects. In September, Samsung released the Galaxy A7 (2018) model with a 24-megapixel main module, 8-megapixel wide-angle and 5-megapixel auxiliary for analyzing the depth of scenes.

What about the fourth camera?

Samsung’s three modules turned out to be not enough, and a few weeks after the release of the Galaxy A7 (2018), it introduced the Galaxy A9 (2018) model with a “quad camera.” A 10-megapixel telephoto lens was added to the trio of modules.

Naturally, in the case of the Galaxy A9 (2018), the presence of a depth sensor is a marketing story. The smartphone became the first in the industry with a quad camera, and with this Samsung wrote itself into history.

Free space problem

Smartphone developers don't have the luxury of free space in their devices. Now they are even forced to abandon the headphone jack so that they can increase the battery capacity. Between the processor, display and battery, there isn't much room for image sensors.

Sensors are already compact in size, resulting in developers having to use various software solutions to compensate for the lack of light. An increase in the number of image sensors may result in a decrease in battery capacity or may result in the failure of some element.

Probably, now manufacturers believe that the advantages of such a solution outweigh the disadvantages. The only question is when the situation will turn upside down. Below you can see a device called Light L16. It was released last year and offers 16 lenses with different focal lengths (from 28 mm to 150 mm). The miracle of “camera making” allows you to take 52-megapixel photos.

You need to understand that an increase in the number of lenses does not always mean an increase in the quality of photographs. If there are cameras, then even if there are 10 modules at once, the quality of the pictures will remain appropriate. Huawei and LG have shown that a triple camera can satisfy the requirements of most mobile photography enthusiasts. However, there is still a risk that Nokia 9 will not be the only smartphone with five cameras next year.

What are the disadvantages

The only drawback for most dual camera systems is that no manufacturer has yet been able to achieve full parity for both modules. Additional sensors have a lower aperture compared to the main camera (from f/2.4 to f/2.8, while conventional cameras have already reached the record f/1.6 mark) and are not equipped with optical image stabilization. This year, only two flagships received OIS for two cameras simultaneously - the Samsung Galaxy Note 8 and the Apple iPhone X.

Because of this difference in the characteristics of the dual cameras, there is a noticeable discrepancy in quality when it comes to shooting with a specific module (wide-angle only or telephoto only). However, the level of photo capabilities that mobile devices like the Galaxy Note 8 or iPhone X offer today is incredibly high.

What are the prospects?

The development of smartphone cameras over the past couple of years has truly surpassed all expectations. There is no doubt that dual camera technology is the future of mobile photography. Already now they are equipped not only with flagships, but also with middle-class devices, as well as even some budget models. New life for black and white photographs, portrait mode with a blurred background straight from DSLRs, high-quality lossless zoom, higher detail - all this can now be done by smartphones thanks to dual cameras.

Rating
( 1 rating, average 5 out of 5 )
Did you like the article? Share with friends:
For any suggestions regarding the site: [email protected]
Для любых предложений по сайту: [email protected]