Add min- and max-created filters to lddb_json_shape.py

[niklasl]

Use like:

$ zcat SOME_DUMP.json.linez.gz |
    python3.11 lddb_json_shape.py /lddb_shapes/ --min-created "2023-08-01T00:00:00Z" --max-created "2023-09-01T00:00:00Z"

[andersju]

LGTM!

[klngwll]

I get a traceback (at different lines on repeated runs) but upon quick inspection doesn't seem to indicate a missing created property in the data but have not had time to investigate more properly.

  File "/libris/librisxl/librisxl-tools/scripts/lddb_json_shape.py", line 128, in <module>
    created = datetime.fromisoformat(data['@graph'][0]['created'])
                                     ~~~~~~~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'created'

[andersju]

Processing definitions/build/*.lines I see a few dozen things with no created property, e.g.,

{'@graph': [{'@type': 'SystemRecord', 'mainEntity': {'@id': 'https://libris.kb.se/'}, '@id': 'p76szt07r0kw1bjb', 'inDataset': [{'@id': 'https://libris.kb.se/dataset/syscore'}, {'@id': 'https://libris.kb.se/dataset/sys/apps'}]}, {'@id': 'https://libris.kb.se/', '@type': 'DataCatalog', 'title': 'libris.kb.se', 'article': {'@type': 'Article', 'articleBody': "<p xml:lang='sv'>Data på <b>LIBRIS.KB.SE</b>.</p>"}}]}
{'@graph': [{'@type': 'SystemRecord', 'mainEntity': {'@id': 'https://libris.kb.se/data'}, '@id': 'p76szt07r4pwm3dk', 'inDataset': [{'@id': 'https://libris.kb.se/dataset/syscore'}, {'@id': 'https://libris.kb.se/dataset/sys/apps'}]}, {'@id': 'https://libris.kb.se/data', '@type': 'DataService', 'titleByLang': {'en': 'LIBRIS-XL Linked Data Platform API'}, 'statistics': {'sliceList': [{'dimensionChain': ['rdf:type'], 'itemLimit': 400}]}}]}

[klngwll]

Hmm, the test sets I used only contain bib, auth or hold data (using method described here). I tried to fetch the line with sed when i got the At: 228,661Traceback (most recent call) but didn't immediately see anything suspicious unless it yields the wrong line.