index errors with large datasets?

[whoburg]

reported by tony tao:

"on the surface it looks like a data size issue ("MemoryError" and "Iterator too large" errors) but when I truncate the data set to something that already worked before, it returns indexing errors which makes me believe it's something in GPfit, but I can't figure out what it is. "

"Actually (as usual, problem is solved after calling mayday) I may have figured it out and now I have a Cd model as well.

The training input data set is around 40,000 data points over 7 dimensions (originally 80,000), so it takes around 16 GB of memory to build the model, which explains the memory error running in Python(x,y). Running it in Ubuntu and deleting about half the training data seems to have fixed it.

The index error is caused by line 73 of the max_affine_init.py script where if the while loop isn't fulfilled by the end of the dataset, it calls the next index location which is out of bounds. "