Client does not use configured HTTP session for scraping provider list

[ml-evs]

The get_all_databases function does its own request handling. In cases where custom HTTP configurations are required (e.g., proxies etc.) this makes it impossible to use the client without manually setting the base URLs.

The get_all_databases (and related functions) could be refactored to use a custom HTTP client/session, but should default to doing their own thing (this function is called at initialisation of the reference server).

See #2274 for more info.