January 24, 2017

Introduction to Image Saliency Detection

An algorithm for saliency detectionWhenever we look at a photo or watch a video, we always notice certain things over others. This could be a person’s face, or a sports car, or even a symbol that is located in the corner of a video.

SalNet is an algorithm that automates the process of finding the salient regions of an image using deep learning.

What is Saliency Detection

Example of saliency detection before and afterSaliency is what stands out to you and how you are able to quickly focus on the most relevant parts of what you see.

In neuroscience, saliency is described as an attention mechanism in organisms to narrow down to the important parts of what they see.

In UX design, saliency is a feedback loop for understanding what parts of a design are useful, and which are not. They use the information they gather from usability and eye tracking studies this to design better interfaces.

Advertisers are well aware that many people don’t have long attention spans, hence they try to catch the eye of a user with a single glance. Saliency detection methods are used to better design ads and posters.

Saliency detection, essentially, can be used in any area in which you’re trying to automate the process of understanding what stands out in an image.

Why You Need Saliency Detection

Using saliency to crop images perfectly

We use saliency detection to make our algorithms smarter. One example of this would be the Smart Thumbnail Algorithm. This microservice uses the Saliency Detector algorithm to get information about the important parts of an image. Smart Thumbnail then uses this information to perfectly crop and resize the image every time.

Using Saliency Detection will make your app/service smarter by detecting the relevant (salient) parts in your images automatically. You can use this information to improve your service, and make your app smarter!

How to Use Saliency Detection

First step is to create a free account on Algorithmia.

After creating your account, go to your profile page and navigate to the Credentials tab. There you will find your API key. Copy that to a safe location, so we can use it later.

Now, let’s find a test image. I’ve picked a photo of a guy from Unsplash, but you can pick another photo if you’d like.

Last and not least, install the Python Algorithmia client using the command pip install algorithmia

Now by running the sample code (in Python) below, we’ll be able to get our stylized image in a few seconds!

Sample Input

[code python]
import Algorithmia

client = Algorithmia.client("YOUR_API_KEY_GOES_HERE")

# Create testing directory if it doesn’t exist

if not client.dir("data://.my/SalNetTest").exists():
client.dir("data://.my/SalNetTest").create()

input = {
"image": "data://deeplearning/example_data/guy.jpg",
"location": "data://.my/SalNetTest/guy_saliency.png",
"saliencyLocation": "data://.my/SalNetTest/saliencyMatrix.json"
}

result = client.algo("deeplearning/SalNet/0.2.0").pipe(input).result

print result
[/code]

Sample Input Image:

Example image for saliency detection

Sample Output:
[code python]
{
"output": "data://.my/SalNetTest/guy_saliency.png",
"saliencyMatrix": "data://.my/SalNetTest/saliencyMatrix.json"
}
[/code]
Sample Output Image:

Saliency detection outputYou can now find the stylized image in your newly created collection in the data API portal.

For more information, go to the SalNet algorithm page here. You can find all relevant documentation about the algorithm there.

Conclusion

Saliency detection is a way to spend less time and energy determining what’s most relevant in an image.

SalNet is an algorithm that utilizes the power of deep learning to extract information about important, relevant parts of any given image by finding salient regions and points.

The heavy lifting is handled behind a serverless microservices API, so you can focus on developing your app, and not worry about servers or infrastructure.

Let us know if you’ve used SalNet in your app/service via @algorithmia on Twitter!

Here's 50,000 credits
on us.

Algorithmia AI Cloud is built to scale. You write the code and compose the workflow. We take care of the rest.

Sign Up