csv has moved, format changed

[datapharmer]

LoanStatsNew.csv is no longer at https://www.lendingclub.com/fileDownload.action?file=LoanStatsNew.csv&type=gen

Instead it is hosted as a zip file at https://resources.lendingclub.com/LoanStats3c.csv.zip

It also appears the issued date format has changed, and as such loans are able to be parsed. The new issue_d column contains mmm-YYYY. For example Sep-2014 or Aug-2013.

[gcflymoto]

Thanks for reporting

[Clete2]

Also the new one only takes this year's into account. For a full picture you'll have to get:
https://resources.lendingclub.com/LoanStats3b.csv.zip
and
https://resources.lendingclub.com/LoanStats3a.csv.zip

Found them here:
https://www.lendingclub.com/info/download-data.action

[gcflymoto]

The good news is I added support for multiple stats files and download links from the command line.. Bad news is that there have actually been many changes to the data format and values.. the new release won't be backward compatible with the old files. Also this release will be python only until the kinks are worked out.

[gcflymoto]

uploaded a new version. Please test it out and I'll close the issue

[Clete2]

Thanks! It seems to be working well, except I think it is trying to parse the header as a loan. No big deal, as it told me that it threw it out. I'll let it run overnight and see how it works.

One question, is there any advantage to using the "secured" version of the LoanStats? It seems to have a lot more data.

[Clete2]

Bug? It is always saying 0/mo for # of loans. Maybe it really is 0/month.

[iteration 95/4096 22.73 sec/iter] Matched 1050/466287 loans (0/mo.) test at 13.01% APY. 49 loans defaulted (4.00%, $17.87 avg loss) 9.7080% net APY

[gcflymoto]

It's kind of tricky with that header.. There is actually like a comment on the first row.. and the second being the header row
Notes offered by Prospectus (https://www.lendingclub.com/info/prospectus.action)
"id","member_id","loan_amnt","funded_amnt",...

[gcflymoto]

The 0/mo for #loans happens because of such few loans are found compared to the history line of all the loans. the number of loans is less the number of months since the older notes

[gcflymoto]

I'll look into the "secured" version as well

[gcflymoto]

BTW. I suggest Pypy if you are running with python.. It will be much faster.

[Clete2]

Using pypy on that line that I pasted. Not sure if it is configured properly though.

I really appreciate all the work you did on this app. I'm curious to see the results after running it for a while.

[gcflymoto]

Ok cool. I looked into the header issue and actually was already skipping the headers.. that bogus row is an actual row.. now printing the row number to give more information where it was found

[gcflymoto]

going to go buy some groceries and make dinner, when I come back will look into the "secured" version

[gcflymoto]

What you mind giving me some leads on what "secured" loanstats is and where I can find it? Not getting many useful hits of the old google.

[Clete2]

Sorry about that. First, login to your LC account. Then, go back to this page:
https://www.lendingclub.com/info/download-data.action

The loan history links are now *_secured.csv.zip and are much bigger.

[gcflymoto]

@Clete2 I tested with the secure CSVs and it works just fine but you will have to download them yourself.. and then run lcbt.py --stats LoanStats3a_securev1.csv.zip,LoanStats3b_securev1.csv.zip,LoanStats3c_securev1.csv.zip

[gcflymoto]

Unless there is anything else I'm closing the ticket.