[Bug] 2.2.0 Breaks setting user agent for requests

[ivirshup]

Describe the bug

In 2.2.0, setting user agent for requests seems to have broken. This was observed on the cellxgene census test suite which has started to fail, see: chanzuckerberg/cellxgene-census#1451

The user agent is being set with the vfs.s3.custom_headers.User-Agent field.

To Reproduce

See: see: chanzuckerberg/cellxgene-census#1451

Versions (please complete the following information):

• TileDB-SOMA version: 2.2.0
• TileDB core version (libtiledbsoma):
• Language and language version (e.g. Python 3.9, R 4.3.2): Python
• OS (e.g. MacOS, Ubuntu Linux): *
• Note: you can use tiledbsoma.show_package_versions() (Python) or tiledbsoma::show_package_versions() (R)

Additional context
Add any other context about the problem here.

[jp-dark]

Hi @ivirshup - thank you for reporting this regression.

There was a big internal refactor in 2.2 that reduced legacy overhead from when the TileDB-SOMA Python API switched from using TileDB-Py to wrapping the internal libtiledbsoma C++ library. I suspect that might have impacted when the default context is initialized.

Do you know when/how the default context is set? A quick fix you could try is calling context.native_context on the context after it is initially instantiated (this should create the default context immediately). Let me know if that impacts your test failures - it will help me know what to check on the SOMA side.

I am also already working on a new API for the SOMA Context that will make setting the default more explicit. I will tag you on that PR when it is ready.

[ivirshup]

I'm going to see if I can find another person do a bit of the digging here 😅, I've actually just moved teams and found this regression in my last "dev-doc update/ offboarding PR". From what I can tell you without digging too much:

• We set the context when we connect to the store on s3 (via this _open_soma helper) e.g. we pass the context at tiledbsoma.open
• Other config options are taking effect. For instance, the failing test sends requests to s3 through a proxy (configured with vfs.s3.proxy_host and vfs.s3.proxy_port) so that we can monitor the headers. Since the proxy still sees the request, this level of config must be working.

BTW, I am not sure how to get to a human readable string representation of the SOMAContext from context.native_context.

[mlin]

Testing https://github.com/chanzuckerberg/cellxgene-census/pull/1453/changes

@jp-dark Yea please keep us appraised here about the new context API. We've marked the affected test cases xfail for now so this isn't blocking for us -- it's helpful to have the user agent set for internal analytics purposes, but fine to address this as a normal-priority bug. Thank you!

[mlin]

@jp-dark Looks like your suggestion to force .native_context worked! I'm going to keep that pocketed for now unless I hear some impulse to urgently fix our analytics, but hopefully it's a useful breadcrumb for tracking it down. Thank you!