Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Support Multiple YouTube API Keys #606

Merged
merged 2 commits into from
Feb 10, 2025

Conversation

rebelonion
Copy link
Contributor

@rebelonion rebelonion commented Feb 7, 2025

What's new?

Add support for multiple comma separated YouTube API keys.

What's changed?

N/A

What's fixed?

N/A

Any other comments?

I am pretty unfamiliar with Elixir, so forgive me if my code is atrocious.

This helps users who use the api with many sources and hit the daily limit.

Nothing changes for other users and there is effectively no downside.
Nothing changes in the database, we just parse the comma separated values when we need a key.

This uses a round-robin approach for cycling through the keys. I chose it because it evenly uses the keys and is pretty simple.
The other options I thought of were:

  • random number: no agent, but not as uniform for api usage
  • use one key until (nearly) full: only use other keys when you actually need them, but need to keep track of each key's state.

All tests pass for me

  • I am the original author of this code and I am giving it freely to the community and Pinchflat project maintainers

TODO

Copy link
Owner

@kieraneglin kieraneglin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for taking a swing at this! I've left a few small changes, but overall this is in good shape

I'm really impressed given this is your first go with Elixir! The language itself has some paradigms that are strange if you aren't familiar with them (like Agents)

edit: also, would you mind running mix check before the next push? If that's giving you trouble then just let me know and I can run it 🤙

lib/pinchflat/fast_indexing/youtube_api.ex Outdated Show resolved Hide resolved
lib/pinchflat/fast_indexing/youtube_api.ex Outdated Show resolved Hide resolved
lib/pinchflat/fast_indexing/youtube_api.ex Outdated Show resolved Hide resolved
lib/pinchflat/fast_indexing/youtube_api.ex Outdated Show resolved Hide resolved
lib/pinchflat/fast_indexing/youtube_api.ex Show resolved Hide resolved
@rebelonion
Copy link
Contributor Author

Thanks for your detailed explanations! I think I fixed everything you suggested.

Here's the output of your requested mix check:

mix check ouput
/app git:(feat-multiple-youtube-keys) ✗ mix check
=> running compiler

Compiling 122 files (.ex)
Generated pinchflat app

=> running unused_deps in fix mode

=> running formatter in fix mode

=> running credo

Checking 192 source files (this might take a while) ...

Please report incorrect results: https://github.com/rrrene/credo/issues

Analysis took 0.4 seconds (0.09s to load, 0.3s running 55 checks on 192 files)
885 mods/funs, found no issues.

Showing priority issues: ↑ ↗ →  (use `mix credo explain` to explain issues, `mix credo --help` for options).

=> running sobelow

##############################################
#                                            #
#          Running Sobelow - v0.13.0         #
#  Created by Griffin Byatt - @griffinbyatt  #
#     NCC Group - https://nccgroup.trust     #
#                                            #
##############################################

... SCAN COMPLETE ...

=> running ex_unit

Compiling 20 files (.ex)

09:13:24.140 [debug] Exqlite: loading extension `/app/_build/test/lib/pinchflat/priv/repo/extensions/sqlean-linux-x86/sqlean`

09:13:24.142 [debug] Exqlite: loading extension `/app/_build/test/lib/pinchflat/priv/repo/extensions/sqlean-linux-x86/sqlean`
No ERD generated
Running ExUnit with seed: 402049, max_cases: 48

.............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Finished in 4.5 seconds (0.00s async, 4.5s sync)
941 tests, 0 failures
[os_mon] memory supervisor port (memsup): Erlang has closed
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed

=> running prettier_formatting in fix mode

yarn run v1.22.22
$ prettier . --write --config=.prettierrc.js --ignore-path=.prettierignore --ignore-path=.gitignore
.devcontainer/devcontainer.json 38ms (unchanged)
.github/ISSUE_TEMPLATE/bug_report.md 52ms (unchanged)
.github/ISSUE_TEMPLATE/feature_request.md 7ms (unchanged)
.github/ISSUE_TEMPLATE/other.md 2ms (unchanged)
.github/pull_request_template.md 2ms (unchanged)
.github/workflows/docker_release.yml 11ms (unchanged)
.github/workflows/lint_and_test.yml 10ms (unchanged)
.prettierrc.js 6ms (unchanged)
assets/css/app.css 16ms (unchanged)
assets/css/satoshi.css 12ms (unchanged)
assets/js/alpine_helpers.js 22ms (unchanged)
assets/js/app.js 11ms (unchanged)
assets/js/tabs.js 6ms (unchanged)
assets/package.json 2ms (unchanged)
assets/tailwind.config.js 52ms (unchanged)
docker-compose.ci.yml 2ms (unchanged)
docker-compose.yml 2ms (unchanged)
package.json 2ms (unchanged)
priv/grafana/application.json 27ms (unchanged)
priv/grafana/beam.json 77ms (unchanged)
priv/grafana/ecto.json 39ms (unchanged)
priv/grafana/oban.json 68ms (unchanged)
priv/grafana/phoenix_live_view.json 35ms (unchanged)
priv/grafana/phoenix.json 34ms (unchanged)
README.md 43ms (unchanged)
test/support/files/channel_source_metadata.json 3ms (unchanged)
test/support/files/example.info.json 108ms (unchanged)
test/support/files/media_metadata.json 190ms (unchanged)
Done in 2.02s.

=> finished in 0:09

 ✓ compiler success in 0:02
 ✓ credo success in 0:01
 ✓ ex_unit success in 0:07
 ✓ formatter fix success in 0:01
 ✓ prettier_formatting fix success in 0:02
 ✓ sobelow success in 0:01
 ✓ unused_deps fix success in 0:01

@kieraneglin kieraneglin changed the title feat: Support Multiple YouTube API Keys [Enhancement] Support Multiple YouTube API Keys Feb 10, 2025
@kieraneglin kieraneglin added the enhancement New feature or request label Feb 10, 2025
Copy link
Owner

@kieraneglin kieraneglin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! Thank you once again. I'll merge once CI passes

@kieraneglin kieraneglin merged commit 28f0d8c into kieraneglin:master Feb 10, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants