Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

For xpro, conditionally ingest topics for each resource depending on formatting #1611

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

mbertrand
Copy link
Member

@mbertrand mbertrand commented Sep 26, 2024

Blocked until the production xpro api has been fully updated with mit-learn topics.

What are the relevant tickets?

Closes https://github.com/mitodl/hq/issues/5551

Description (What does it do?)

Adjusts the xpro ETL pipeline to no longer transform topics. Any topic names matching mit-learn topics will be assigned, others will be ignored.

How can this be tested?

  • Set the following backend env variables:
    XPRO_CATALOG_API_URL=https://xpro.mit.edu/api/programs/
    XPRO_COURSES_API_URL=https://xpro.mit.edu/api/courses/
    ``
    
  • Run ./manage.py backpopulate_xpro_data, it should complete without errors.
  • Go to https://learn.mit.edu/search?offered_by=xpro, compare with https://xpro.mit.edu/api/courses/: only topics that match mit-learn topics should have been assigned (so, no topics with a ":" for example).

@mbertrand mbertrand force-pushed the mb/xpro_learn_topics branch from 3119fd3 to 5950b9b Compare October 1, 2024 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant