How ‘video understanding’ could transform Facebook

How ‘video understanding’ could transform Facebook

View gallery

.Understanding video is a multi-year challenge Facebook argues could transform the social network experience for the better.

Facebook (FB) users spend over 100 million hours a day gobbling up video on the social network. But despite all that content flowing through — & the technology & ingenuity powering it – Facebook still hasn’t figured out how to wrap its algorithmic prowess around video the way it already does with photos, using facial recognition, for instance, to identify you & your friends.

The reason: sheer complexity. A photo is one static image, yet a video is essentially copious images sequenced in a particular order to show a narrative in motion: a Siamese kitten purring or a professor in the middle of a BBC interview interrupted by his two young kids.

Using artificial intelligence to scan & analyze a video on the fly — “video understanding,” as it’s called — is a multi-year challenge Facebook argues could transform the social network experience for the better.

“We think video understanding is going to be ridiculously impactful, because if you go back in time & you think approximately the News Feed — even before photos were that prevalent — it was mostly text, & so that was the content you needed to understand in order to rank [people’s feeds],” Joaquin Candela, Facebook’s Director of Applied Machine Learning, told Yahoo Finance.  

View gallery

.“We think video understanding is going to be ridiculously impactful, ” Joaquin Candela, Facebook’s Director of Applied Machine Learning, told Yahoo Finance. Source: Facebook

“We’re at a point now where we’re pretty satisfactory at understanding photos, yet now there’s video,” Candela added. “You even have live video, & the question becomes, well, how swift can you figure out what’s going on in this video?”

If anyone at the social network can tackle that challenge, it’s Candela, who leads Facebook’s Applied Machine Learning group (AML). The group’s mission? Take the heady ideas & theories generated by the neighboring Facebook Artificial Intelligence Research group (FAIR) & turn those ideas into reality.

Already, the FAIR & AML groups algorithms are capable of identifying certain elements in a video — objects like a house, a pizza box or pet — yet they remain light years away from fully deciphering & tracking the most significant aspect: people’s behavior. 

“The majority of the videos that come to Facebook are people-centric,” explained Manohar Paluri, computer vision lead at the AML group. “And if we don’t understand what people are doing, we will never understand what the video is about.”

View gallery

.“If we don’t understand what people are doing, we will never understand what the video is about,” explained Manohar Paluri, research lead of computer vision at the AML group. Source: Facebook

Indeed, the context of a video is every bit as significant as quickly figuring out who is in the video. Is this Facebook user attending a rally? Giving a speech? Playing squash?

Once they do that, Facebook contends there are numerous practical applications for Facebook users. Although Facebook does not disclose how much Live video users shoot on any given day, the social network says people are 10 times more likely to comment on Facebook Live videos than on regular videos.

But how much more likely are you to check out that video if you received a notification because a friend of yours is being filmed? Not only that, yet what if the notification told you exactly what your friend was doing in that moment, like say, running on Zuma Beach in Malibu, Calif., or chowing down on sashimi at Nobu in New York City?

That kind of hypothetical is what Facebook hopes to offer its users in the next three to five years. And while the level of granularity may sound disconcerting to some — an algorithm smart enough to understand exactly what you’re doing — remember that just five years ago, people were up in arms over Facebook using facial recognition to identify people in photos. Now, many people take facial recognition within Facebook for granted.

For Facebook, the payoff for nailing & rolling out a feature like video understanding is increased user engagement. Although the social network has evolved into the third most-trafficked website in the world, it is always developing new ways to keep users inside Facebook or its stable of products & services, whether they be subtle speed improvements or rolling out the Snapchat Stories-like feature Facebook Messenger Day.

As time marches on, people become accustomed to, even dependent upon, many of those technological improvements. Facebook is hoping the same for video understanding, too.

JP Mangalindan is a senior correspondent for Yahoo Finance covering the intersection of tech & business. Follow him on Twitter or Facebook.  

More from JP:

3 major problems with Trump’s protectionist attitude toward China 

Fry’s Electronics: How this tech retailer has survived the fall of brick-and-mortar

Why ‘experience can injure tech workers in Silicon Valley

Why AI could be Silicon Valley’s latest ‘micro bubble’

Surprise & disgust: What 6 Silicon Valley CEOs said approximately Trump’s ban

Amazon is now worth more than the 8 largest retailers combined

Arts & EntertainmentMediaFacebook

Source: “JP Mangalindan”

Share and Enjoy

  • Facebook
  • Twitter
  • Delicious
  • LinkedIn
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS