Update for Pydantic v2

[jwodder]

Closes #1352.

Blocked by dandi/dandi-archive#1820.

Do not merge until after dandi/dandi-schema#203 is merged & released. Once that is done, the "[TEMP] Use Pydantic 2.0 branch of dandischema" commit in this PR should be replaced with a commit that sets the dandischema version requirement to ~= 0.9.0 instead. [Done]

[codecov]

Codecov Report

Attention: Patch coverage is 87.14286% with 9 lines in your changes are missing coverage. Please review.

Project coverage is 88.57%. Comparing base (7ca670b) to head (728882f).

Files	Patch %	Lines
dandi/utils.py	81.81%	4 Missing ⚠️
dandi/dandiarchive.py	33.33%	2 Missing ⚠️
dandi/cli/cmd_download.py	66.66%	1 Missing ⚠️
dandi/dandiapi.py	95.83%	1 Missing ⚠️
dandi/files/bases.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1381      +/-   ##
==========================================
+ Coverage   88.49%   88.57%   +0.07%     
==========================================
  Files          77       77              
  Lines       10492    10493       +1     
==========================================
+ Hits         9285     9294       +9     
+ Misses       1207     1199       -8     

Flag	Coverage Δ
unittests	88.57% <87.14%> (+0.07%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[yarikoptic]

Looks LGTM to me.

[candleindark]

LGTM

[CodyCBakerPhD]

@yarikoptic What was the timeline on getting this merged/released?

[yarikoptic]

Originally planned right after coming back from holidays, but that fell through.
I just followed up on dandi/dandi-schema#203 (comment) -- we should first ensure that there is no metadata migrations to be done, then we would merge that one and start going through the depends like here. But overall -- I hope that by the end of this week we get moving with that.

[jwodder]

@yarikoptic Various tests are currently failing because the Archive doesn't yet accept schema version 0.6.5.

[yarikoptic]

@jwodder - could we TEMP test against version in

• Boost dandischema to 0.9.* series so we get support for pydantic 2.0 and schema 0.6.5 dandi-archive#1823
  before it is merged/release so we know that all changes are ok with it? (also PR requires rebase)

[jwodder]

@yarikoptic Testing in dandi-cli requires a Docker image of the Archive PR. It would be easier to instead add a temporary commit in the Archive PR for running the CLI integration tests against this PR.

[yarikoptic]

Function name and description suggests being applicable to URL of any transport, not just https so better be smth like

Suggested change

	if path.lower().startswith(("http://", "https://")):
	if re.match('[a-z]+://', path.lower()):

to avoid surprises.

[yarikoptic]

but also how it is different from a standard urllib's join?

In [3]: urllib.parse.urljoin('http://a.c/', '/1/2/3')
Out[3]: 'http://a.c/1/2/3'

In [4]: urllib.parse.urljoin('http://a.c/', 's3://1/2/3')
Out[4]: 's3://1/2/3'

[jwodder]

• The function is only meant for HTTP(S) URLs.

• It differs from urljoin() for URLs that have a path that doesn't end in /:

  >>> urljoin("https://example.com/foo/bar", "quux")
  'https://example.com/foo/quux'
  

  joinurl() would instead give https://example.com/foo/bar/quux.

[yarikoptic]

• then better name it httpurljoin or at least describe it in its description
• also please add into description this difference in behavior as rationale

[jwodder]

Done: 728882f

[yarikoptic]

thank you @jwodder ! let's proceed but remember to not release until archive is released (attn @waxlamp - we need to sync release process etc )

[github-actions]

🚀 PR was released in 0.60.0 🚀