Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DeviantArt scraper no longer pulls the image #229

Open
chaskayote opened this issue Apr 18, 2024 · 3 comments
Open

DeviantArt scraper no longer pulls the image #229

chaskayote opened this issue Apr 18, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@chaskayote
Copy link

Describe the bug
Current DA URLs don't seem to work anymore in the scraper. I've tried these two:
https://www.deviantart.com/rickgriffin/art/Housepets-Spa-Movie-Night-800928049
https://www.deviantart.com/rickgriffin/art/800928049
I checked the ex file and but seems to look for a url that ends with /art.

To Reproduce
Steps to reproduce the behavior:

  1. Go to Upload
  2. Paste a deviant art deviation URL.
  3. Watch as nothing happens.

Expected behavior
Thumbnail of submission appears in image preview and artist and description are pulled.

@chaskayote chaskayote added the bug Something isn't working label Apr 18, 2024
@mdashlw
Copy link
Contributor

mdashlw commented Apr 23, 2024

I had tried to fix the deviantart scraper but it fails to fetch images from images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com due to some ssl issues. Appears to be elixir's problem, I did not figure it out.

Try it at https://derpibooru.org/images/new
https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com/f/982c675c-20c2-4552-8800-7b6bf7b1b3e8/dh9v3q0-091931fd-d5b9-44af-94a8-df8ce59f0dc7.png?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsIm9iaiI6W1t7ImhlaWdodCI6Ijw9MTQ0MCIsInBhdGgiOiJcL2ZcLzk4MmM2NzVjLTIwYzItNDU1Mi04ODAwLTdiNmJmN2IxYjNlOFwvZGg5djNxMC0wOTE5MzFmZC1kNWI5LTQ0YWYtOTRhOC1kZjhjZTU5ZjBkYzcucG5nIiwid2lkdGgiOiI8PTI1NjAifV1dLCJhdWQiOlsidXJuOnNlcnZpY2U6aW1hZ2Uub3BlcmF0aW9ucyJdfQ.xkriM8CjntGHEt5p7RE11PECu-3SbRCWpTyyhYAsw1Y

@liamwhite
Copy link
Contributor

@mdashlw you might need to disable middlebox compat mode in Mint

@chaskayote
Copy link
Author

chaskayote commented Apr 26, 2024

To get a Pixiv scraper working, I had to put in a referrer. Do you think adding a referrer of DeviantArt.com would fix?

From viewlogs running in prod
Screenshot_2024-04-26-00-26-37-33_d365b52accad0f47adbc08c16219827d.jpg

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants