-
-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Async URL checks #375
base: main
Are you sure you want to change the base?
Async URL checks #375
Conversation
๐ Hello @glenn-jocher, thank you for submitting this
Regarding Your Changes:The asynchronous enhancements youโve introduced promise a massive leap in efficiency and resource utilization for URL validation workflows ๐. However, a couple of pointers for consideration:
If you encounter any blockers, refer to the Contributing Guide or leave a comment here. An Ultralytics engineer will review this PR soon to assist you further. Thanks again for contributing to Ultralytics โ weโre excited to see this in action! ๐โจ |
Signed-off-by: Glenn Jocher <[email protected]>
Signed-off-by: Glenn Jocher <[email protected]>
Signed-off-by: Glenn Jocher <[email protected]>
Signed-off-by: Glenn Jocher <[email protected]>
๐ ๏ธ PR Summary
Made with โค๏ธ by Ultralytics Actions
๐ Summary
This PR refactors URL and link-checking utilities to leverage asynchronous programming, significantly improving efficiency and performance.
๐ Key Changes
aiohttp
library for URL validation and link checking.is_url_async
andcheck_links_in_string_async
, withasyncio.run
used for their synchronous counterparts (is_url
andcheck_links_in_string
).ThreadPoolExecutor
and redundantrequests
code, streamlining logic.asyncio.sleep
for retry attempts when checking URLs.๐ฏ Purpose & Impact
ThreadPoolExecutor
, making the script lighter and more scalable.This refactor makes link validation faster and more dependable, benefiting developers by reducing delays when working with large datasets or documents containing many URLs. ๐