A monthly dump of the 8,000 most-downloaded packages from PyPI:
Unminified:
Note: It now takes too much quota to collect data for 365 days. Those files were last updated on 2021-04-01 and have been removed. Old versions can be found in releases.
From cron, it runs pypinfo to dump JSON and commit back to this repo.
For example on Ubuntu 22.04:
sudo apt-get install jq
Follow https://github.com/ofek/pypinfo to sign up for BigQuery, install and authenticate.
pip3 install "pypinfo>=13.0.0"
pypinfo --help
pypinfo --auth path/to/your_credentials.json
git clone [email protected]:hugovk/top-pypi-packages.git
cd top-pypi-packages
git config user.name "Deploy Bot"
git config user.email "[email protected]"
git config user.name
git config user.email
- Create SSH key on server: https://www.digitalocean.com/community/tutorials/how-to-use-ssh-keys-with-digitalocean-droplets
- Add your SSH key to the ssh-agent: https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/#adding-your-ssh-key-to-the-ssh-agent
- Add deploy key with write access at https://github.com/hugovk/top-pypi-packages/settings/keys/new
crontab -e
# First of the month
30 17 1 * * ( eval "$(ssh-agent -s)"; ssh-add ~/.ssh/id_rsa-top-pypi-packages; /home/botuser/github/top-pypi-packages/top-pypi-packages.sh ) > /tmp/top-pypi-packages.log 2>&1
Thanks to DigitalOcean for supporting open-source software.