Representing video thumbnails for frame navigation

[glenrobson]

Transfered from: ArchiveLabs/iiif.archivelab.org#101

The Internet Archive creates derivative thumbnails from uploaded video files, which look suspiciously like frame stills to be used by players to aid timeline navigation.

In our test fixture video, for example, there are a couple dozen files within a subdirectory, which are listed in the manifest like this:

      {
         "name":"youtube-7w8F2Xi3vFw.thumbs\/34C3_-_International_Image_Interoperability_Framework_IIIF_Kulturinstitutionen_schaffen_interop-7w8F2Xi3vFw_000834.jpg",
         "source":"derivative",
         "format":"Thumbnail",
         "original":"34C3_-_International_Image_Interoperability_Framework_IIIF_Kulturinstitutionen_schaffen_interop-7w8F2Xi3vFw.mp4",
         "mtime":"1546551758",
         "size":"4428",
         "md5":"1d2cad1f4f511b1d0ab0bdf7cc611796",
         "crc32":"756fa5c8",
         "sha1":"b763f896e5275c59b02563b8750cdd34587baed5"
      },
      {
         "name":"youtube-7w8F2Xi3vFw.thumbs\/34C3_-_International_Image_Interoperability_Framework_IIIF_Kulturinstitutionen_schaffen_interop-7w8F2Xi3vFw_000894.jpg",
         "source":"derivative",
         "format":"Thumbnail",
         "original":"34C3_-_International_Image_Interoperability_Framework_IIIF_Kulturinstitutionen_schaffen_interop-7w8F2Xi3vFw.mp4",
         "mtime":"1546551794",
         "size":"7112",
         "md5":"36ad797e266ef4c79316d96927d9f2ac",
         "crc32":"2364fa78",
         "sha1":"a312123c2686088fb5ee16ceb447fdf8e394bf30"
      },
      {
         "name":"youtube-7w8F2Xi3vFw.thumbs\/34C3_-_International_Image_Interoperability_Framework_IIIF_Kulturinstitutionen_schaffen_interop-7w8F2Xi3vFw_000954.jpg",
         "source":"derivative",
         "format":"Thumbnail",
         "original":"34C3_-_International_Image_Interoperability_Framework_IIIF_Kulturinstitutionen_schaffen_interop-7w8F2Xi3vFw.mp4",
         "mtime":"1546551831",
         "size":"5380",
         "md5":"34c00081bd7faa2dfd123892d3a5b1a3",
         "crc32":"d2721265",
         "sha1":"af2a2006a93dcf603fef8a19211e47c2c7bac263"
      },

For @mekarpeles and the IA folks, what are these? If they are frames taken from the video, how can we determine which times correspond to which frame? (Both the filename suffix and the mtime attributes look promising.)

For the IIIF A/V folks, is there currently any way of enabling frame-based navigation within existing players? How should this kind of time-specific thumbnail within a canvas be modeled?

[glenrobson]

Doing a bit more digging:

• I don't think mtime is directly helpful here, it's a unix timestamp, potentially the last modified time
• here's a comment from an IA repo, discussing how they select their thumbnails for display:

? /*  The 'poster' is intentionally a direct Http link as its intended only for search engines etc
                Preference is 2nd thumbnail (first is usually black-screen) in .thumbs/ directory (e.g. for 'commute');
                if only one (e.g. item 'stairs') use that. */