Rasters API #5
Comments
|
There's this folder in their github repo, but I don't quite understand what the files are here -- they all seem to be empty to me: https://github.com/nvkelso/natural-earth-vector/tree/v5.1.2/zips |
|
Yeah, I noticed that as well. I think they might have tried to include rasters there at some point, but because the file sizes were so large they gave up. They have some entries about rasters in the Makefile but they are pretty vague. I was planning on manually grabbing links from e.g. https://www.naturalearthdata.com/downloads/10m-raster-data/ and going from there, since they don't look like they're going to change. We should also post a note saying that that API is pretty breakable... |
|
Ah okay. Not as "automatic" as I thought... e.g. |
|
Here's a potential lead: the R-version of this package provides a function that discovers the relevant zip files from I don't have R installed on this computer, so I haven't had the chance to test their function yet. |
|
Okay. A little bit more info:
Note that we would need to add some utility functions to "translate" e.g. For GitHub CI or similar, we can periodically check that page if there exists any tags higher than the current tag (i.e. |
|
Do we need to store the zipfiles as Artifacts? Since they're lazy anyway (and most are usually on the order of hundreds of MB anyway), we could just download using |
|
Hmm. Though we're most definitely in the "write-once, read-many times" paradigm outlined by Scratch.jl, for which It's unfortunate we can't customize download/unpack-instructions with |
|
If we muck around with the code, it might actually be possible - Artifacts uses 7z to unpack stuff, which does support zip files. But for now it seems to make sense to use scratch spaces, and if possible transition to Artifacts after the fact. |
|
Good catch! I suspect we may need some support / buy-in from Should we proceed with Scratch then? It would be nice if we can hide the internals so that the user "API" is still only Do you have any design ideas for versioning? Feel free to take a stab at it if you have some time. |
|
I tried XML parsing for a bit but didn't make too much progress. Over the course of this I did find github.com/EcoJulia/RasterDataSources.jl, which sounds like something we could hook into. I would rather not have Rasters.jl as a dependency of this package :D so a user could, theoretically, just type |
|
Thanks for looking into this. Alternatively we could wait until julia 1.9 is released with the extensions feature. In that case the additional dependency might not be too bad. |
It doesn't look like there is an automated solution to get rasters in the way that we've obtained vectors, so we'd have to create a manual database of all links. Then, they are presented as zipfiles, meaning that we would have to use something like Scratch.jl and ZipFile.jl to unpack them manually and store them in a package-specific scratchspace. Assuming no name duplicates, this should also allow us to present a list of already-downloaded rasters.
The text was updated successfully, but these errors were encountered: