Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed-up video processing when including Youtube videos #97

Open
rgaudin opened this issue Sep 29, 2021 · 1 comment
Open

Speed-up video processing when including Youtube videos #97

rgaudin opened this issue Sep 29, 2021 · 1 comment
Labels
enhancement New feature or request stale

Comments

@rgaudin
Copy link
Member

rgaudin commented Sep 29, 2021

--without-videos only prevent the Youtube ones from being processed and included but there are many regular videos anyway. Those regular videos are short and small-sized and easy to process.

Youtube ones can be huge and are retrieved from Youtube where exceeding a certain number of resource usage (requests? bandwidth ?) results in being banned for a period of time.

We thus need to process Youtube videos one by one but others can be handled concurrently.

Current code specify the nb of videos worker based on --without-videos but ideally, we should treat both sources differently no matter the option as including Youtube videos means processing all the videos on a single worker.

@rgaudin rgaudin added the enhancement New feature or request label Sep 29, 2021
@stale
Copy link

stale bot commented Mar 2, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

@stale stale bot added the stale label Mar 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request stale
Projects
None yet
Development

No branches or pull requests

1 participant