Fix uploading Zarr within a BIDS dataset; typing-check guided fix for handling requests exception #1331
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #1331 +/- ##
==========================================
+ Coverage 88.81% 88.85% +0.03%
==========================================
Files 76 76
Lines 10200 10216 +16
==========================================
+ Hits 9059 9077 +18
+ Misses 1141 1139 -2
Flags with carried forward coverage won't be shown. Click here to find out more.
☔ View full report in Codecov by Sentry. |
…n BIDS
could not run tests locally unfortunately - likely due to my custom
configuration of networking for podman (?) or was it also done for docker? TODO to figure it out
ulling django ... done
Creating network "dandiarchive-docker_default" with the default driver
ERROR: Failed to Setup IP tables: Unable to enable SKIP DNAT rule: (iptables failed: iptables --wait -t nat -I DOCKER -i br-c26ebcecb035 -j RETURN: iptables: No chain/target/match by that name.
(exit status 1))
Removing network dandiarchive-docker_default
WARNING: Network dandiarchive-docker_default not found.
ERROR
============================================================ ERRORS =============================================================
____________________________________________ ERROR at setup of test_upload_bids_zarr ____________________________________________
dandi/tests/fixtures.py:403: in docker_compose_setup
run(
/usr/lib/python3.11/subprocess.py:571: in run
raise CalledProcessError(retcode, process.args,
E subprocess.CalledProcessError: Command '['docker-compose', 'run', '--rm', 'django', './manage.py', 'migrate']' returned non-zero exit status 1.
===================================================== slowest 10 durations ======================================================
119.18s setup dandi/tests/test_upload.py::test_upload_bids_zarr
(1 durations < 0.005s hidden. Use -vv to show these durations.)
==================================================== short test summary info ====================================================
ERROR dandi/tests/test_upload.py::test_upload_bids_zarr - subprocess.CalledProcessError: Command '['docker-compose', 'run', '--rm', 'django', './manage.py', 'migrate']' returned non-...
================================================= 1 error in 119.31s (0:01:59) ==================================================
python -m pytest -s -v dandi/tests/test_upload.py::test_upload_bids_zarr 3.87s user 3.43s system 6% cpu 2:01.11 total
Also flake8 was not happy either
flake8...................................................................Failed
- hook id: flake8
- exit code: 1
dandi/tests/test_upload.py:297:5: F821 undefined name 'new_dandiset'
dandi/tests/test_upload.py:298:16: F821 undefined name 'new_dandiset'
dandi/tests/test_upload.py:303:5: F821 undefined name 'new_dandiset'
so it seems it does not know how to deal with those pytest fixtures but odd
that just now and just for that one
issue reproduced!____________________________ test_upload_bids_zarr _____________________________
dandi/tests/test_upload.py:297: in test_upload_bids_zarr
bids_zarr_dandiset.upload()
dandi/tests/fixtures.py:516: in upload
upload(
dandi/upload.py:339: in upload
for v in process_path(dfile):
dandi/upload.py:264: in process_path
for r in dfile.iter_upload(
dandi/files/bases.py:339: in iter_upload
etagger = get_dandietag(self.filepath)
/opt/hostedtoolcache/Python/3.11.5/x64/lib/python3.11/site-packages/fscacher/cache.py:165: in fingerprinter
ret = fingerprinted(*args, **kwargs_)
/opt/hostedtoolcache/Python/3.11.5/x64/lib/python3.11/site-packages/joblib/memory.py:655: in __call__
return self._cached_call(args, kwargs)[0]
/opt/hostedtoolcache/Python/3.11.5/x64/lib/python3.11/site-packages/joblib/memory.py:598: in _cached_call
out, metadata = self.call(*args, **kwargs)
/opt/hostedtoolcache/Python/3.11.5/x64/lib/python3.11/site-packages/joblib/memory.py:856: in call
output = self.func(*args, **kwargs)
/opt/hostedtoolcache/Python/3.11.5/x64/lib/python3.11/site-packages/fscacher/cache.py:101: in fingerprinted
return f(path, *args, **kwargs)
dandi/support/digests.py:100: in get_dandietag
return DandiETag.from_file(filepath)
/opt/hostedtoolcache/Python/3.11.5/x64/lib/python3.11/site-packages/dandischema/digests/dandietag.py:147: in from_file
with open(path, "rb") as f:
E IsADirectoryError: [Errno 21] Is a directory: '/tmp/pytest-of-runner/pytest-0/dandiset236/sub-01/ses-01/micr/sub-01_ses-01_sample-A_SPIM.ome.zarr' |
…Zarr is used not of a file Also fixed up test
yarikoptic
changed the title
yarikoptic
changed the title
|
🚀 PR was released in |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
could not run tests locally unfortunately . See first commit for more details
Closes #1307
TODOs: