Cut down that movie

Posted by Swaroop C H on May 29, 2008 Filed under: research-paper, review, tech, video

How would life be if you could tell your computer to cut down a 3-hour movie to one hour?

Sounds impossible?

From what I understand of this paper called "Feature fusion and redundancy pruning for rush video summarization" by the people at the Vision Research Laboratory at UCSB, it is very much possible!

The basic idea is to find 'distinctive' parts of the video, for example, someone talking at a high pitch or lots of moving scenes which, intuitively, would be more important than a slow scene or repeated shots.

They consider multiple facets of the video such as speech, camera motion, significant differences in color, suppression of repeated scenes and of course, identification of visually distinct segments.

The caveat is that their test data set are drama "rushes" video which are raw footage including the clapboards, the color tones, repeated takes, etc. This is very conducive to such an algorithm, which could probably explain why they had such good results (details are in the paper).

But if this is the state of things today, I can imagine that around five years down the lane they would really be applying it to commercial movies and television shows. It is amazing on what can be done with a combination of mathematics, statistics and computers.

Interestingly, the final summaries were around 4% of the total video length. If this was applied to the 8-year long Kyunki Saas Bhi Kabhi Bahu Thi show, I wonder how much it would be reduced to...

Update : Now Microsoft Research has done it for audio as well!


Sunil Jagadish says:

"If this was applied to the 8-year long Kyunki Saas Bhi Kabhi Bahu Thi show, I wonder how much it would be reduced to"

ROTF! I'd pity the algorithm if someone were to use the Indian TV soaps as test cases.

Abi says:

Thanks for pointing out this paper. Really interesting idea. I guess this only works for soaps, amateur videos and maybe news. It would probably be pointless to watch a movie like this.

Swaroop says:

@Sunil Heh. Why not? It will always give good results no matter what the algorithm ;-)

@Abi Agreed!


There's no comment box, but please do email me or tweet me your thoughts and criticisms, and I will publish the relevant ones here.