New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DocumentCleaner remove_nodes_re #125

Open

jeffnappi opened this issue Jul 23, 2014 · 1 comment

Contributor

jeffnappi commented Jul 23, 2014

I keep finding sites that fail to parse due to the remove_nodes_re. Do you think there's a better way this can be handled?

e.g. the following article fails due to the class of 'scrolling-wrapper' on the parent div of the article:

http://www.shutterstock.com/blog/ceo-jon-oringers-message-to-the-next-generation-embrace-failure

Owner

grangier commented Dec 30, 2014

I agree. The document cleaner is too strict. this commit should help a36b5a8 but it doesn't seems to be enough for this article.

It doesn't get the beginning of the article

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment