Skip to content

v0.13.0 Release Plan & Test Matrix

Rob Walch edited this page Dec 13, 2019 · 32 revisions

v0.13.0

New Features

Bugfixes

Misc. Improvements

Fix #2400 - Cues doesn't show up on browsers which addCue doesn't throw error with invalid structure (WebOS)

v0.13.0 Manual Test Matrix

OS Browser Basic Playback 1 Live 2 Captions 3 ID3 4 Alt Audio 5 Attach/Detach 6
Desktop (MacOS/Windows) Firefox
  • mcunningham
  • mcunningham
  • mcunningham
  • mcunningham
  • Assignee
  • Assignee
Desktop (MacOS/Windows) Chrome
  • robwalch
  • robwalch
  • robwalch
  • Assignee
  • Assignee
  • Assignee
Android Chrome
  • robwalch
  • robwalch
  • robwalch
  • Assignee
  • robwalch
  • Assignee
Windows IE11
  • mcunningham
  • Assignee
  • Assignee
  • Assignee
  • Assignee
  • Assignee
Windows Edge 18
  • mcunningham
  • Assignee
  • Assignee
  • Assignee
  • Assignee
  • Assignee
  • Tests for specific features/fixes
  • Basic playback (mostly automated)
  • Live (not automated)
  • ID3
  • TS changes (could be covered by other suites)
  • Safari smoke test
  • Bundle analysis
  • Test with existing TS definitions

General:

Things to look out for:

  • Console warnings
  • Console errors
  • HLS errors
    • Especially internal exceptions

1 Test 1 Steps (Basic Playback):

  1. Begin playback
  2. Assert that playback begins at the configured start time (default 0)
    • Unless otherwise noted, Hls.js should not seek on startup (e.g. to skip a gap)
  3. Allow Hls.js to play for at least three segments
  4. Assert that playback continues smoothly and without issue
  5. Switch renditions
    • Test switching using both the Current Level and Next Level API
  6. Assert that Hls.js switches renditions smoothly and without issue
  7. Seek to at most three segments from the end
    • Unless otherwise noted, Hls.js should begin playback immediately (e.g. it should not try to skip a gap)
  8. Allow the stream to end
  9. Assert that Hls.js has signaled end of stream
    • video.ended should be true
    • BUFFER_EOS should have been fired
    • There should be no more network requests from Hls.js (check the Network tab)

2 Test 2 Steps (Live):

  1. Begin playback
  2. Assert that playback begins at the default time (default 3 segment lengths)
  3. Allow playback to continue for at least one minute
  4. Assert that playback continues smoothly and without issue
  5. Switch renditions
  6. Assert that Hls.js switches renditions smoothly and without issue
  7. Allow playback to continue for at least one minute
  8. Assert that playback continues smoothly and without issue
  9. Stop playback
  10. Assert that Hls.js does not download media while paused

3 Test 3 Steps (Captions):

  1. Begin playback
  2. Enable any captions track
  3. Assert that captions are displayed
  4. Allow playback to continue
  5. Assert that the correct cue is displayed at the corresponding time
    • VTT files can be inspected for the cue time
  6. Seek forwards
  7. Assert that the correct cue is displayed, and that the previous cue is no longer showing

4 Test 4 Steps (ID3):

  1. Set up Hls.js
  2. Attach a FRAG_PARSING_USERDATA to the Hls.js instance
  3. Begin playback
  4. Assert that ID3 cues are emitted at the expected time

5 Test 5 Steps (Alternate Audio):

  1. Begin playback
  2. Assert that current audio track is playing
  3. Select a different audio track
  4. Assert that the audio immediately switches to the new track
  5. Allow playback to continue
  6. Assert that audio continues to play on the same track
  7. Seek around
  8. Assert that audio continues to play on the same track
  9. Switch back to the original track
  10. Assert that the audio immediately switches to the new track
  11. Continue playback (and seek, etc)
  12. Assert that the correct audio track continues to play

6 Test 6 Steps (Attach/Detach):

Issues impacted:

  1. Begin playback
  2. Detach the player via detachMedia
  3. Assert that there are no exceptions in the console
  4. Attach media with the existing media element
  5. Assert that playback resumes in the same state as before playback
    • Ensure that features like alt audio and captions continue to work