How would life be if you could tell your computer to cut down a 3-hour
movie to one hour?
Sounds impossible?
From what I understand of this paper called “Feature fusion and
by the people at the Vision Research Laboratory at UCSB, it is very
much possible!
The basic idea is to find ‘distinctive’ parts of the video, for
example, someone talking at a high pitch or lots of moving scenes
which, intuitively, would be more important than a slow scene or
repeated shots.
They consider multiple facets of the video such as speech, camera
motion, significant differences in color, suppression of repeated
scenes and of course, identification of visually distinct segments.
The caveat is that their test data set are drama “rushes” video which
are raw footage including the clapboards, the color tones, repeated
takes, etc. This is very conducive to such an algorithm, which could
probably explain why they had such good results (details are in the
paper).
But if this is the state of things today, I can imagine that around
five years down the lane they would really be applying it to
commercial movies and television shows. It is amazing on what can be
done with a combination of mathematics, statistics and computers.
Interestingly, the final summaries were around 4% of the total video
length. If this was applied to the 8-year long Kyunki Saas Bhi Kabhi
show, I wonder how much it would be reduced to…
Update : Now Microsoft Research has done it for audio as well!