[FEATURE] Integrate media player from slub_web_sachsendigital #1320

fschoelzel · 2024-08-28T15:23:15Z

This PR integrates the video/audio player from slub_web_sachsendigital into a new MediaPlayer plugin.
The PR is based on #796 and #818 from @dvoracek-slub

For testing this within the DFG Viewer, I have prepared a DDEV system at https://github.com/dvoracek-slub/ddev-dfgviewer-dist/tree/mediaplayer. It relies on https://github.com/dvoracek-slub/dfg-viewer/tree/dev-mediaplayer and includes a demo METS video document.

There also is a work-in-progress section in the documentation that describes some features and technical aspects of the player. composer docs:serve can be used for local preview.

Update (2022-07-06). Most notable changes:
* Kitodo in general
  
  * Read production place and person names in `Mods` format class. I use this to test GND linking of some audio-related metadata.
  * Add option to show full TOC.

* Media player
  
  * It is now possible to set and export markers/segments in the media. The list is shown in audio mode (press F2).
  * When the METS links preprocessed waveform data, this can be shown (and be used to edit markers). See the DDEV system for an example.
  * First version of handling recordings that are split across multiple media files. Currently, this only works via page reloads.

github-advanced-security

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

sebastian-meyer · 2024-09-09T14:31:30Z

You placed a lot of JavaScript files in Resources/Private/JavaScript/. Are you sure those shouldn't be put in Resources/Public/JavaScript/ instead, because they need to be accessible by the client?

Also, please put all unchanged third-party libraries in their own sub-directories separate from your own JS files. There are several reasons for that:

It's easier to update dependencies if you know which files are part of a third-party library and don't contain any custom changes.
We can exclude third-party libraries from all static analysis checks, especially regarding their code styling. (This will most likely greatly reduce the number of Codacy issues currently reported!)

fschoelzel · 2024-09-09T17:21:01Z

You placed a lot of JavaScript files in Resources/Private/JavaScript/. Are you sure those shouldn't be put in Resources/Public/JavaScript/ instead, because they need to be accessible by the client?

This is correct and was done on purpose. All the files in the Resources/Private/JavaScript/ folder are written to a compressed file called DlfMediaPlayer.js during the build process using webpack. This and the associated .map and vendor file are the only files that the client needs when loading the player.
Additional information to code organization: MediaPlayer/Developers.rst

fschoelzel · 2024-09-09T17:48:42Z

Also, please put all unchanged third-party libraries in their own sub-directories separate from your own JS files. There are several reasons for that:

It's easier to update dependencies if you know which files are part of a third-party library and don't contain any custom changes.

We can exclude third-party libraries from all static analysis checks, especially regarding their code styling. (This will most likely greatly reduce the number of Codacy issues currently reported!)

As far as i know this two third-party files were copied from the original repository and rewritten by @dvoracek-slub to meet the ES6 typescript requirements:
Resources/Private/JavaScript/DlfMediaPlayer/3rd-party/EventTarget.js
Resources/Private/JavaScript/DlfMediaPlayer/3rd-party/VideoFrame.js

The only other file I found is:
Resources/Private/JavaScript/SlubMediaPlayer/components/equalizer/filters.js

Which in my opinion should not easily be updated, like the other two files, without deeply testing the whole function of the Player.

Apart from that, all other third-party dependencies are located in the Build/node_modules folder and get compressed into the DlfMediaVendor.js file

fschoelzel · 2024-09-09T17:51:39Z

I agree that the files created after the build process should be stored in the Resources/Private/JavaScript/DlfMediaPlayer folder instead of just the JavaScript folder.

sebastian-meyer

Some general remarks in addition to the detailed comments below (as discussed in our call):

Please remove the DevServer or make it a dev-only dependency if that's possible.
Please try to merge DlfMediaPlayer, SlubMediaPlayer and lib/ as much as possible, since those are dependencies of each other anyway which won't work as stand-alone implementations.
** At the moment it is really difficult to tell apart what those do, where they are used and how they depend on each other.
** The best way of handling this would be to have just two directories: One containing the third-party Shaka player (and possibly other unchanged third-party libraries) and one containing all the Kitodo-specific code.
Please add license and author information to every non-third-party file.

sebastian-meyer · 2024-09-23T15:36:20Z

.gitignore

 /Build/Test/.env
+composer.lock


Please don't ignore the composer.lock! We want to deliver a known working state of dependencies.

Suggested change

composer.lock

sebastian-meyer · 2024-09-23T16:14:34Z

Classes/Common/MetsDocument.php

@@ -435,6 +435,7 @@ private function getFiles(array &$details, ?SimpleXMLElement $filePointers): voi
            // Check if file has valid @USE attribute.
            if (!empty($fileUse[$fileId])) {
                $details['files'][$fileUse[$fileId]] = $fileId;
+                $details['all_files'][$fileUse[$fileId]][] = $fileId;


I think this is highly confusing. Also, $details['files'] not being an array holding multiple file identifiers may be a bug.

I'd prefer adapting $details['files'] to hold all file identifiers instead of introducing a confusing new variable with just a slightly different use case. What do you think?

sebastian-meyer · 2024-09-23T16:21:18Z

Classes/Common/MetsDocument.php

@@ -1468,6 +1530,7 @@ private function getFileRepresentation(string $id, SimpleXMLElement $physicalNod
            // Check if file has valid @USE attribute.
            if (!empty($fileUse[$fileId])) {
                $this->physicalStructureInfo[$id]['files'][$fileUse[$fileId]] = $fileId;
+                $this->physicalStructureInfo[$id]['all_files'][$fileUse[$fileId]][] = $fileId;


See above. I'd prefer adapting files instead of introducing all_files.

sebastian-meyer · 2024-09-23T16:40:09Z

Classes/ViewHelpers/MediaPlayerConfigViewHelper.php

+
+        $localizationFactory = GeneralUtility::makeInstance(LocalizationFactory::class);
+
+        // TODO: Wouldn't there be a TypoScript utility?


Please have a look at https://docs.typo3.org/other/typo3/view-helper-reference/main/en-us/Global/Translate.html
It's not recommended to implement your own translation function, try to instead use one provided by the TYPO3 framework.

sebastian-meyer · 2024-09-23T16:42:30Z

Configuration/FlexForms/TableOfContents.xml

+                            <label>LLL:EXT:dlf/Resources/Private/Language/locallang_be.xlf:plugins.tableofcontents.flexform.showFull</label>
+                            <config>
+                                <type>check</type>
+                                <default>0</default>


The default here should definitely be 1 in order to keep the current behavior! Please also make sure that this is the default in TableOfContentsController if the option was not set at all (i. e. after upgrading).

sebastian-meyer · 2024-10-21T11:55:57Z

Resources/Private/JavaScript/SlubMediaPlayer/components/equalizer/FrequencyResponse.js

+        // Manual IIR filter
+        // https://www.earlevel.com/main/2016/12/01/evaluating-filter-frequency-response/
+        for (let i = 0; i < this.frequencies_.length; i++) {
+          const f = /** @type {number} */(this.frequencies_[i])


Suggested change

const f = /** @type {number} */(this.frequencies_[i])

const f = /** @type {number} */(this.frequencies_[i]);

sebastian-meyer · 2024-10-21T11:59:15Z

Resources/Private/JavaScript/SlubMediaPlayer/components/equalizer/filtersets/RiaaEq.js

+    // midGain = 10 * Math.log10(1 + (baseBoost / freq) ** 2) - 10 * Math.log10(1 + (baseBoostRolloff / freq) ** 2)
+    // midGain / 10 = Math.log10(1 + (baseBoost / freq) ** 2) - Math.log10(1 + (baseBoostRolloff / freq) ** 2)
+    // midGain / 10 = Math.log10((1 + (baseBoost / freq) ** 2) / (1 + (baseBoostRolloff / freq) ** 2))
+    // 10 ^ (midGain / 10) = (1 + (baseBoost / freq) ** 2) / (1 + (baseBoostRolloff / freq) ** 2)
+
+    // 10 ^ (((10 * Math.log10(1 + baseBoost ** 2) - 10 * Math.log10(1 + baseBoostRolloff ** 2)) / 2) / 10) = (1 + (baseBoost / freq) ** 2) / (1 + (baseBoostRolloff / freq) ** 2)
+    // 10 ^ ((Math.log10((1 + baseBoost ** 2) / (1 + baseBoostRolloff ** 2))) / 2) = (1 + (baseBoost / freq) ** 2) / (1 + (baseBoostRolloff / freq) ** 2)
+    // sqrt((1 + baseBoost ** 2) / (1 + baseBoostRolloff ** 2)) = (1 + (baseBoost / freq) ** 2) / (1 + (baseBoostRolloff / freq) ** 2)
+
+    // For midFreq, ask Wolfram Alpha to solve


sebastian-meyer · 2024-10-21T13:35:43Z

Resources/Private/Less/DlfMediaPlayer.less

@@ -0,0 +1,8 @@
+@import "../../../../slub_digitalcollections/Resources/Private/Less/Helper/Variables.less";


Please don't add dependencies to slub_digitalcollections!

sebastian-meyer · 2024-10-21T13:46:36Z

Tests/Functional/FunctionalTestCase.php

+                'fileGrpVideo' => 'VIDEO,DEFAULT',
+                'fileGrpVideo' => 'VIDEO,DEFAULT,VIDEO_ALT1',


Duplicate use of the array key fileGrpVideo.

sebastian-meyer · 2024-10-21T13:47:57Z

import-mediaplayer.sh

Please remove this file.

Based upon commit slub/slub_web_sachsendigital#6e68a11

- Adopt build system from `slub_web_sachsendigital` - Use player in plugin - In Table of Contents, link to timecodes - Make element ID configurable

…le View When working with single Frames the Playbutton is always on top of the Video and disrupt the workflow - In Mobile View we assume there is no detailed work with the AV-Tools

add seekStep Value from Typoscript to navigate.rewind and navigate.seek

Whether or not showing the Poster Image, if given, until playback is first started

… - on mobile always

see: kitodo#1150

…or retrieving timecode and file IDs

… readability

… entryArray as in Master Branch

… function on MarkerTable

…ata to add timecode and fps

…ions in webpack.config Updated the regular expression in the webpack configuration to ensure that both .css and .less file extensions are correctly anchored at the end of the string. This prevents misleading operator precedence and ensures only the intended files are processed.

…oller

…s in MediaPlayerController

- Updated .babelrc to include module-resolver plugin for alias imports. - Modified webpack.config.js to resolve alias paths for cleaner imports. - Update ESLint configuration for improved import resolution and compatibility This change improves the readability and maintainability of import statements by allowing the use of path aliases.

Codacy warning about usage of chr()

fix warning in webpack build: Skipped data-uri embedding of images/xxx.svg because file not found

Replaced PageRenderer with AssetCollector in MediaPlayerController.php for adding CSS and JavaScript assets - Feature TYPO3 v10.3: https://docs.typo3.org/c/typo3/cms-core/main/en-us/Changelog/10.3/Feature-90522-IntroduceAssetCollector.html#changelog-Feature-90522-IntroduceAssetCollector

github-advanced-security bot found potential problems Aug 28, 2024

View reviewed changes

sebastian-meyer self-requested a review September 9, 2024 14:06

sebastian-meyer assigned fschoelzel Sep 9, 2024

sebastian-meyer added the ⚙ feature A new feature or enhancement. label Sep 9, 2024

sebastian-meyer added this to the Kitodo.Presentation 5.1.0 milestone Sep 9, 2024

fschoelzel force-pushed the dev-mediaplayer branch 3 times, most recently from f8e7a39 to b591463 Compare September 9, 2024 16:09

fschoelzel force-pushed the dev-mediaplayer branch from 78a41f1 to d51c11c Compare October 9, 2024 14:26

sebastian-meyer modified the milestones: Kitodo.Presentation 5.1.0, Kitodo.Presentation 6.0.0 Oct 16, 2024

sebastian-meyer reviewed Oct 21, 2024

View reviewed changes

This was referenced Oct 21, 2024

[FEATURE] Multimedia support #542

Closed

[FEATURE] Integrate media player from slub_web_sachsendigital #818

Closed

fschoelzel force-pushed the dev-mediaplayer branch from d51c11c to f782308 Compare December 9, 2024 18:13

Kajetan Dvoracek added 11 commits December 17, 2024 14:01

Media Player: Add plugin boilerplate

7471397

Media Player: Add file group, read METS chapters

8b584a1

fixup "Add file group" [fileGrpVideo]

9cc4bee

Media Player: Import code from slub_web_sachsendigital

e68663d

Based upon commit slub/slub_web_sachsendigital#6e68a11

Media Player: Integrate imported code

a1403bd

- Adopt build system from `slub_web_sachsendigital` - Use player in plugin - In Table of Contents, link to timecodes - Make element ID configurable

Media Player: Chapter and page integration

fcdffb9

(WIP) Start documenting media player

8ecf419

(WIP) Add flexforms for player configuration

106ec88

Media Player: Build JS

8bb7963

Configure jsconfig path

7c935f5

Media Player: Fix and add poster

6e5ed6f

fschoelzel added 29 commits December 17, 2024 14:01

Media Player: Remove Playbutton from Desktop View - show only in Mobi…

102c8c0

…le View When working with single Frames the Playbutton is always on top of the Video and disrupt the workflow - In Mobile View we assume there is no detailed work with the AV-Tools

Media Player: Use Constants from Typoscript to Seek and Rewind

ad4f936

add seekStep Value from Typoscript to navigate.rewind and navigate.seek

Media Player: Make Poster Configurable

5ecbb9e

Whether or not showing the Poster Image, if given, until playback is first started

Media Player: show BigPlayButton only once after playback has started…

1c17b8a

… - on mobile always

update npm packages and build

5628ca8

[MAINTENANCE] Use nested extension configuration

3a8531b

see: kitodo#1150

fix and refactor MetsDocument.php after rebase

33ba799

specify getTimecode function to use fileGrpVideo instead of fileGrp f…

911ce47

…or retrieving timecode and file IDs

Refactor Mods.php using php-mods-reader to improve code structure and…

89f8a2c

… readability

Fix: Refactor Mods.php using php-mods-reader to improve code structure

4616778

Refactor TableOfContentsController.php to include year information in…

0464026

… entryArray as in Master Branch

Refactor MetsDocument to match with Musical Sources Feature

fbdd418

Update npm dependencies and Build

0ce0a55

Media Player: [FEATURE] Select "Remove timestamp" removed in Bookmark…

0e3b73e

… function on MarkerTable

Refactor ScreenshotModal.js to fix image filename with extended Metad…

82b21a9

…ata to add timecode and fps

Rename class name to bookmark-markertable-hidden in the ScreenshotModal

f271c84

Media Player: Fix PHPStan Error - PHPDoc comments in MediaPlayerContr…

e14b374

…oller

Media Player: Fix PHPStan errors by adding type annotations and check…

7f28321

…s in MediaPlayerController

add "video_frame_rate" property in AudioVideoMDTest

36a8e37

Media Player: Fixup - Update return type annotation for findFiles method

21ff0bc

Refactor import paths in JavaScript files to path aliases

f13ee2d

[MAINTENANCE] Replace chr(31) with pack('C', 31) kitodo#1207

575039e

Codacy warning about usage of chr()

Refactor: Reduce complexity of getVideoInfo method

97e9adb

[FIX] Add path for shaka-player/ui to lessOptions in webpack.config

71a590c

fix warning in webpack build: Skipped data-uri embedding of images/xxx.svg because file not found

Refactor: Update file paths for DlfMediaPlayer JavaScript files

50418ee

[STYLE] Update Buttons in BookmarkModal and ScreenshotModal

202f3fc

fschoelzel force-pushed the dev-mediaplayer branch from f782308 to 202f3fc Compare December 17, 2024 13:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE] Integrate media player from slub_web_sachsendigital #1320

[FEATURE] Integrate media player from slub_web_sachsendigital #1320

fschoelzel commented Aug 28, 2024

github-advanced-security bot left a comment

sebastian-meyer commented Sep 9, 2024 •

edited

Loading

fschoelzel commented Sep 9, 2024 •

edited

Loading

fschoelzel commented Sep 9, 2024

fschoelzel commented Sep 9, 2024

sebastian-meyer left a comment

sebastian-meyer Sep 23, 2024

sebastian-meyer Sep 23, 2024

sebastian-meyer Sep 23, 2024

sebastian-meyer Sep 23, 2024

sebastian-meyer Sep 23, 2024

sebastian-meyer Oct 21, 2024

sebastian-meyer Oct 21, 2024

sebastian-meyer Oct 21, 2024

sebastian-meyer Oct 21, 2024

sebastian-meyer Oct 21, 2024


		$localizationFactory = GeneralUtility::makeInstance(LocalizationFactory::class);

		// TODO: Wouldn't there be a TypoScript utility?

	const f = /** @type {number} */(this.frequencies_[i])
	const f = /** @type {number} */(this.frequencies_[i]);

		@@ -0,0 +1,8 @@
		@import "../../../../slub_digitalcollections/Resources/Private/Less/Helper/Variables.less";

		'fileGrpVideo' => 'VIDEO,DEFAULT',
		'fileGrpVideo' => 'VIDEO,DEFAULT,VIDEO_ALT1',

[FEATURE] Integrate media player from slub_web_sachsendigital #1320

Are you sure you want to change the base?

[FEATURE] Integrate media player from slub_web_sachsendigital #1320

Conversation

fschoelzel commented Aug 28, 2024

github-advanced-security bot left a comment

Choose a reason for hiding this comment

sebastian-meyer commented Sep 9, 2024 • edited Loading

fschoelzel commented Sep 9, 2024 • edited Loading

fschoelzel commented Sep 9, 2024

fschoelzel commented Sep 9, 2024

sebastian-meyer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sebastian-meyer commented Sep 9, 2024 •

edited

Loading

fschoelzel commented Sep 9, 2024 •

edited

Loading