Skip to content

Commit

Permalink
Add a test checking/demonstrating the problem of uploading Zarr withi…
Browse files Browse the repository at this point in the history
…n BIDS

could not run tests locally unfortunately - likely due to my custom
configuration of networking for podman (?) or was it also done for docker? TODO to figure it out

    ulling django   ... done
    Creating network "dandiarchive-docker_default" with the default driver
    ERROR: Failed to Setup IP tables: Unable to enable SKIP DNAT rule:  (iptables failed: iptables --wait -t nat -I DOCKER -i br-c26ebcecb035 -j RETURN: iptables: No chain/target/match by that name.
     (exit status 1))
    Removing network dandiarchive-docker_default
    WARNING: Network dandiarchive-docker_default not found.
    ERROR

    ============================================================ ERRORS =============================================================
    ____________________________________________ ERROR at setup of test_upload_bids_zarr ____________________________________________
    dandi/tests/fixtures.py:403: in docker_compose_setup
        run(
    /usr/lib/python3.11/subprocess.py:571: in run
        raise CalledProcessError(retcode, process.args,
    E   subprocess.CalledProcessError: Command '['docker-compose', 'run', '--rm', 'django', './manage.py', 'migrate']' returned non-zero exit status 1.
    ===================================================== slowest 10 durations ======================================================
    119.18s setup    dandi/tests/test_upload.py::test_upload_bids_zarr

    (1 durations < 0.005s hidden.  Use -vv to show these durations.)
    ==================================================== short test summary info ====================================================
    ERROR dandi/tests/test_upload.py::test_upload_bids_zarr - subprocess.CalledProcessError: Command '['docker-compose', 'run', '--rm', 'django', './manage.py', 'migrate']' returned non-...
    ================================================= 1 error in 119.31s (0:01:59) ==================================================
    python -m pytest -s -v dandi/tests/test_upload.py::test_upload_bids_zarr  3.87s user 3.43s system 6% cpu 2:01.11 total

Also flake8 was not happy either

    flake8...................................................................Failed
    - hook id: flake8
    - exit code: 1

    dandi/tests/test_upload.py:297:5: F821 undefined name 'new_dandiset'
    dandi/tests/test_upload.py:298:16: F821 undefined name 'new_dandiset'
    dandi/tests/test_upload.py:303:5: F821 undefined name 'new_dandiset'

so it seems it does not know how to deal with those pytest fixtures but odd
that just now and just for that one
  • Loading branch information
yarikoptic committed Sep 28, 2023
1 parent 697af7c commit c19906b
Show file tree
Hide file tree
Showing 2 changed files with 27 additions and 0 deletions.
16 changes: 16 additions & 0 deletions dandi/tests/fixtures.py
Original file line number Diff line number Diff line change
Expand Up @@ -628,6 +628,22 @@ def bids_nwb_dandiset(
return new_dandiset


# TODO: refactor: avoid duplication and come up with some fixture helper which would
# just need specify bids example name
@pytest.fixture()
def bids_zarr_dandiset(
new_dandiset: SampleDandiset, bids_examples: Path
) -> SampleDandiset:
shutil.copytree(

Check warning on line 637 in dandi/tests/fixtures.py

View check run for this annotation

Codecov / codecov/patch

dandi/tests/fixtures.py#L637

Added line #L637 was not covered by tests
bids_examples / "micr_SEMzarr",
new_dandiset.dspath,
dirs_exist_ok=True,
ignore=shutil.ignore_patterns(dandiset_metadata_file),
)
(new_dandiset.dspath / "CHANGES").write_text("0.1.0 2014-11-03\n")
return new_dandiset

Check warning on line 644 in dandi/tests/fixtures.py

View check run for this annotation

Codecov / codecov/patch

dandi/tests/fixtures.py#L643-L644

Added lines #L643 - L644 were not covered by tests


@pytest.fixture()
def bids_dandiset_invalid(
new_dandiset: SampleDandiset, bids_error_examples: Path
Expand Down
11 changes: 11 additions & 0 deletions dandi/tests/test_upload.py
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,17 @@ def test_upload_zarr(new_dandiset: SampleDandiset) -> None:
new_dandiset.upload()


# identical to above, but different scenaior/fixture and path. TODO: avoid duplication
def test_upload_bids_zarr(bids_zarr_dandiset: SampleDandiset) -> None:
bids_zarr_dandiset.upload()
(asset,) = bids_zarr_dandiset.dandiset.get_assets()
assert isinstance(asset, RemoteZarrAsset)
assert asset.asset_type is AssetType.ZARR
assert asset.path == "sample.zarr"

Check warning on line 301 in dandi/tests/test_upload.py

View check run for this annotation

Codecov / codecov/patch

dandi/tests/test_upload.py#L297-L301

Added lines #L297 - L301 were not covered by tests
# Test that uploading again without any changes works:
bids_zarr_dandiset.upload()

Check warning on line 303 in dandi/tests/test_upload.py

View check run for this annotation

Codecov / codecov/patch

dandi/tests/test_upload.py#L303

Added line #L303 was not covered by tests


def test_upload_different_zarr(tmp_path: Path, zarr_dandiset: SampleDandiset) -> None:
asset = zarr_dandiset.dandiset.get_asset_by_path("sample.zarr")
assert isinstance(asset, RemoteZarrAsset)
Expand Down

0 comments on commit c19906b

Please sign in to comment.