Not extracting UL LI text

[raistie]

Bulleted points in articles are not extracted and are totally missing from the extracted text

[grangier]

@raistie please provide URLs that fails

[grangier]

duplicate #42

[raistie]

http://psychcentral.com/blog/archives/2014/05/08/5-ways-to-stop-a-worry-filled-what-if-cycle/
would be one of the many i have tried

[grangier]

Problem is the ul and li tags are cleaned before the main node is detected (to remove menu and all other useless node). If we keep the ul li, we may add a lot of non wanted content to the extracted texte. I'm not sure how we can deal with it

[raistie]

I think the thing is these days articles have a lot of titles like "10 ways to...", "5 steps to..." That come in list form

As more and more articles that way, this will miss out a lot of the bullet points that is in the article - which will miss important information

[raistie]

Just tried boilerpipe - it extracts the bulleted points ok.