Default User-Agent is being rejected by some sites

[pcolazurdo]

Hi all,
I've tried using this tool recently and found that it wasn't downloading some info from sites. After some debugging I've found that, for example twitter, was rejecting the default user agent.

I've replaced it with: headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'} and it worked fine.

It would be nice if User-Agent would be an configurable option so the code doesn't need to be updated every time some site reject this.

Hope it helps,
Pablo