-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: replaced Beautiful Soup by selectolax to enhance performance #213
Conversation
Reviewer's Guide by SourceryThis PR replaces Beautiful Soup with Selectolax for HTML parsing to improve performance. The change involves updating all HTML parsing logic to use Selectolax's CSS selector syntax instead of Beautiful Soup's find/find_all methods. The core parsing functionality remains the same, but the implementation is more efficient. Class diagram for updated HTML parsingclassDiagram
class HTMLParser {
- create_bs_tag(html_content: str)
+ create_parser_tag(html_content: str)
+ store_response_data(response: httpx.Response)
}
class PlayerCareerParser {
- __get_title(profile_div: Tag) : str | None
+ __get_title(profile_div: LexborNode) : str | None
- __get_endorsement(progression_div: Tag) : dict | None
+ __get_endorsement(progression_div: LexborNode) : dict | None
- __get_competitive_ranks(progression_div: Tag) : dict | None
+ __get_competitive_ranks(progression_div: LexborNode) : dict | None
}
class HeroParser {
- __get_summary(overview_section: Tag) : dict
+ __get_summary(overview_section: LexborNode) : dict
- __get_abilities(abilities_section: Tag) : list[dict]
+ __get_abilities(abilities_section: LexborNode) : list[dict]
}
class RolesParser {
- parse_data() : list[dict]
+ parse_data() : list[dict]
}
HTMLParser <|-- PlayerCareerParser
HTMLParser <|-- HeroParser
HTMLParser <|-- RolesParser
note for HTMLParser "Replaced BeautifulSoup with Selectolax for parsing"
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Quality Gate passedIssues Measures |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @TeKrop - I've reviewed your changes and they look great!
Here's what I looked at during the review
- 🟢 General issues: all looks good
- 🟢 Security: all looks good
- 🟢 Testing: all looks good
- 🟢 Complexity: all looks good
- 🟢 Documentation: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Summary by Sourcery
Replace Beautiful Soup with Selectolax for improved performance in HTML parsing, refactor parsing logic accordingly, update dependencies, and adjust tests and documentation to reflect these changes.
New Features:
Enhancements:
Build:
Documentation:
Tests: