Cloud Video Intelligence Api

Cloud Video Intelligence

The best way to experience the video API is through a live demo and let’s starts by taking this video of a Super Bowl commercial for Google Home and I’m going to play the first few seconds. We can see that it starts with a mountain landscape and we see a house, a city street. Then it goes to a dog and a garage. So lots of scene changes happening in this video and if we were to manually categorizes what was happening throughout it, we’d need to watch the entire thing and write down what was happening in every scene. Luckily, the video API takes care of this for us with just a single REST API request. So it tells us two things–

  • One, at a high level, it tells us what is this video about and then
  • It also tells us at a more granular level what labels it finds in the video in each scene.

So if we scroll down here, we can see it identifies a dog, and it can tell us exactly where in the video that dog appears. It also identifies at the end of the video there’s a birthday cake and if we scroll down a bit more, we can see that not only does it know it’s a dog, it knows what type of dog it is. That it’s a dachshund and if we scroll down through the rest of the labels, we can see that it also successfully identifies that mountain pass scene from the beginning. So this is what the API can do with one video, but you likely have more than one video that you want to analyze.

Using Video Intelligence API

So let’s take a look at how a company might use the video intelligence API. A media publisher could have hundreds of petabytes of video data sitting in storage buckets and one common thing they might want to do is create a highlight reel, focus on a specific type of content, or search their large library for a specific entity. So let’s see how we would use the video intelligence API to search a large library of videos given all this metadata that we get back from it. So we’ve got a lot of videos here and let’s say this media publisher has hours of sports video, but they only want to find the content relevant to baseball. So let’s go ahead and search our library here for baseball videos and we can see that not only does it show us which videos have baseball, it tells us exactly when in those videos baseball appears. My favorite example is this one.

We have this video, which only has a tiny bit about baseball, but it’s able to identify that clip for us whereas if we were to manually do this, we’d have to watch the whole video looking for that specific scene. So if we click on this scene, we can see that this is from last year’s year-end search video when the Cubs won the World Series. So let’s do one more search. I live on the East Coast where it’s pretty cold right now. I’ve heard there’s been a ton of rain in SF this past year. I think we can all agree it’d be pretty nice to be on a beach right now and while machine learning can’t take us there, it can do the next best thing and find all the beach clips in our video library for us. So let’s search for our beach videos, and then we can click to all of our beach clips in the videos below. So as you saw through this demo, the video intelligence API makes it easy to quickly and easily understand a large library of video content, something that was almost impossible just a few months ago. Tasks that used to take hours, now take seconds with the video intelligence API and I’m excited to make it available to all of you today.


Please enter your comment!
Please enter your name here