Twitter is using machine learning to crop photos to the most interesting part

Twitter is using machine learning to crop photos to the most interesting part
Twitter is using machine learning to crop photos to the most interesting part

The lure of device learning isn’t all the time about massive new features; continuously, what it does easiest are small tweaks that subtly give a boost to consumer experience. So it is with Twitter’s use of neural networks to automatically crop picture previews to their finest phase.

the company’s been engaged on this tool for a while, but described its strategies intimately in a blog publish the day before today. It’s a captivating little read, with ML engineer Lucas Theis and ML lead Zehan Wang explaining how they began just using facial popularity to crop pictures to faces, however discovered that this method didn’t work with photos of surroundings, gadgets, and, most significantly, cats.

Their answer was once “cropping the usage of saliency” (saliency here which means whatever’s finest in an image — faces or no). To define this they used information from instructional studies into eye-tracking, which document what spaces of images people look at first. “this data can also be used to coach neural networks and other algorithms to foretell what folks might want to look at,” write Theis and Wang.

Extra examples of ahead of and after using Twitter’s ML-powered auto-cropper. Image: Twitter

After They’d skilled a neural network to identify these areas, they needed to optimize it to work in real time on the site. Thankfully for them, the cropping needed for a photo preview is beautiful broad — you’re simplest narrowing down an image to possibly its finest third. You don’t wish to aim in on specifics. that implies Twitter may pare down and simplify the standards the neural community was once judging the use of a method called “wisdom distillation.”

the tip consequence used to be a neural community ten occasions faster than its original layout. “This we could us carry out saliency detection on all pictures as quickly as they are uploaded and crop them in actual-time,” write Theis and Wang.

This new function is recently being rolled out on pc, iOS, and Android apps to all users says the company. So next time you notice a photo preview on Twitter that invites you to click be sure to thank a neural network.

What's Your Reaction?

Cry Cry
Cute Cute
Damn Damn
Dislike Dislike
Like Like
Lol Lol
Love Love
Win Win

log in

Become a part of our community!

Don't have an account?
sign up

reset password

Back to
log in

sign up

Join Community

Back to
log in
Choose A Format
Personality quiz
Series of questions that intends to reveal something about the personality
Trivia quiz
Series of questions with right and wrong answers that intends to check knowledge
Voting to make decisions or determine opinions
Formatted Text with Embeds and Visuals
The Classic Internet Listicles
Open List
Submit your own item and vote up for the best submission
Ranked List
Upvote or downvote to decide the best list item
Upload your own images to make custom memes
Youtube, Vimeo or Vine Embeds
Soundcloud or Mixcloud Embeds
Photo or GIF
GIF format

Send this to a friend