[ISSUE] download files not working from within databricks apps with version > 0.68.0

[ce-mwestdoerp]

Description
With the latest versions the w.files.download and w.files.download_to functionality is not working and ends in a timeout when using these methods within a deployed databricks app. Only downgrading to version 0.68.0 fixes this issue. Running the same code on a local machine (Windows) works as expected. The correct permissions to access the volume in the unity catalog are set in the databricks.yml.

Reproduction

import os

import streamlit as st
from databricks.sdk import WorkspaceClient

w = WorkspaceClient()

button = st.button("Run")
if button:
    download_file_path = "/Volumes/my_catalog/my_schema/my_volume/my_document.pdf"
    response = w.files.download(download_file_path)
    file_data = response.contents.read()
    file_name = os.path.basename(download_file_path)
    st.download_button(label="Download", data=file_data, file_name=file_name)

Expected behavior
The code should run as expected when deployed as a databricks app, but is stuck at the w.files.download line.

Is it a regression?
Yes, it works when using version <= 0.68.0

Debug Logs

Traceback (most recent call last):
  File "/app/python/source_code/.venv/lib/python3.11/site-packages/databricks/sdk/retries.py", line 38, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/app/python/source_code/.venv/lib/python3.11/site-packages/databricks/sdk/mixins/files.py", line 2280, in delegate
    raise _RetryableException.make_error(response)
databricks.sdk.mixins.files._RetryableException:  (HTTP Status: 500)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/python/source_code/.venv/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling
    result = func()
             ^^^^^^
  File "/app/python/source_code/.venv/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 590, in code_to_exec
    exec(code, module.__dict__)
  File "/app/python/source_code/my_app/test.py", line 12, in <module>
    response = w.files.download(download_file_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/python/source_code/.venv/lib/python3.11/site-packages/databricks/sdk/mixins/files.py", line 803, in download
    initial_response: DownloadResponse = self._open_download_stream(
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/python/source_code/.venv/lib/python3.11/site-packages/databricks/sdk/mixins/files.py", line 2332, in _open_download_stream
    result = self._init_download_response_mode_csp_with_fallback(file_path, headers, response_headers)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/python/source_code/.venv/lib/python3.11/site-packages/databricks/sdk/mixins/files.py", line 2455, in _init_download_response_mode_csp_with_fallback
    return self._init_download_response_presigned_api(file_path, headers)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/python/source_code/.venv/lib/python3.11/site-packages/databricks/sdk/mixins/files.py", line 2420, in _init_download_response_presigned_api
    csp_response: _RawResponse = self._retry_cloud_idempotent_operation(perform)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/python/source_code/.venv/lib/python3.11/site-packages/databricks/sdk/mixins/files.py", line 2297, in _retry_cloud_idempotent_operation
    return retried(
           ^^^^^^^^
  File "/app/python/source_code/.venv/lib/python3.11/site-packages/databricks/sdk/retries.py", line 67, in wrapper
    raise TimeoutError(f"Timed out after {timeout}") from last_err
TimeoutError: Timed out after 0:05:00

Other Information

• Works fine on Windows even on the latest versions
• When deployed as databricks app the functionality does not work.