Separating people or objects in a photo from the background is commonly known as keying. To achieve perfect keying, it is impossible to do so without a certain foundation in PS (Photoshop), but now there is a website called Remove.bg, which claims to be able to complete complex keying in 5 seconds, completely reducing the difficulty of keying to zero threshold! How does this tool achieve extremely fast keying? Let’s explore the technology behind it.
Keying was once a technical job
Usually we shoot pictures, the foreground people or objects are often closely integrated with the landscape as the background, to achieve accurate keying is not easy, it requires a series of fine operations (Figure 1).
Figure 1: Step by step illustration of keying
Let’s take the channel keying hair operation as an example. First of all, you need to find the object of the clearest channel, then copy the channel, and then adjust the color level of the channel, and then use the brush tool to paint the person and the required part of the hair, and finally reverse selection, layer duplication and other operations to complete a keying operation (Figure 2). In the process, because the hair is relatively small and confusing, it takes a lot of time just to paint the selection.
Figure 2 Channel keying operation
In addition to the above, if you key a picture with a complex background, you may need to use multiple tools together, which requires knowledge of masking, channels, quick selection, layer modes, etc. It is because of the strong professionalism of keying, so for ordinary people to master the skills of keying, it is necessary to repeatedly practice, which is definitely a technical work.
The most important thing is that you have to be able to do this.
Most people don’t have the time to spend on learning keying, so a variety of keying tools have emerged, the most notable of which is the newly emerged Remove.bg website, which claims to be able to make this complex technical task a zero-threshold one. As you can see in the website’s demo, even difficult keying operations like hair strands can be easily accomplished through the website’s processing (Figure 3).
Figure 3 Website keying demo
Remove.bg’s extremely fast keying is actually achieved with the help of a deep learning Python tool, a programming language often designed to write automated scripts (shells) that automate many operations through a series of scripting codes, and keying is a typical application in the image field (Figure 4).
Figure 4 Python language
In the Python keying script, the script code first selects the region of the object to be extracted, and identifies the image into two parts: inside the region (foreground) and outside the region (background) through a certain algorithm. Then, based on the selected area, the Grabcut algorithm will segment the background and foreground of the image, while cycling through the foreground content, extracting the foreground from it and removing the background content, thus realizing keying (Figure 5).
Of course, the script alone cannot fully implement the segmentation of arbitrary image background and foreground, so in order to make the script more efficient and accurate, Remove.bg combines Python automatic processing script and deep machine learning, and in order to make the script accurately identify the foreground objects, the developer preempts some machine learning libraries and Python frameworks to develop some algorithms, such as Scikit and Tensorflow machine learning algorithms. These algorithms can automatically detect patterns in the input, so that the developer first gives some images to the algorithm to learn, and inputs a lot of mixed front and back view photos of people for the machine to learn. Take the example of inputting a photo of a zebra in a grassland, the algorithm can identify the tiger and the grassland environment in the photo (Figure 6).
Figure 6 Machine algorithm
In order for the algorithm to have higher recognition ability, the developer also prepared many similar pictures of zebras, such as various pictures of zebras in zoos, cages, and bushes for the machine to learn. Using deep learning and neural networks, the machine algorithm eventually recognized the foreground and hindground of various photos accurately through learning and self-correction of a large number of images.
Finally, the developer deploys the model generated by this algorithm on the server side of Remove.bg. Users only need to upload images through the website, and the algorithm in the background of the website can realize accurate recognition of foreground and background of photos and realize keying, and users only need to wait for a moment and then download the processed keyed images (Figure 7).
Figure 7 Illustration of the Remove.bg keying process
Python does more than just keying
As mentioned above, with the Remove.bg website, ordinary people can quickly key images through the algorithm service deployed on the website, which greatly reduces the technical difficulty of keying images. For example, if you want to show your travel pictures around the world to your friends, all you have to do is take a selfie of a person, upload the keyed picture, and post the keyed picture on a foreign tourist spot. Of course, you can also do all kinds of spoof effects production.
In fact, Python’s automatic processing script can be applied to more areas than just keying. For example, you can use the script to recognize the face of a person in a photo, and combine it with face recognition technology to quickly extract photos of suspects from numerous surveillance photos. For example, it can be applied to website shopping, which can extract all similar products from major shopping websites through automatic recognition for our selection and reference, so our life will be more and more convenient.