How to skip query string percent encoding?

[trim21]

I have a url looks like "https://example.com/?TQC/KWC0qaYyEg=="

it's a base64 encoded string so it contains /.

requests just use the raw url input, and I find that httpx will try to encode / with %2F, so it's not the expected url, and I can't find how to disable this encoding.

import httpx


raw = "https://example.com/?TQC/KWC0qaYyEg=="
url = httpx.URL(raw)

print(url)  # https://example.com/?TQC%2FKWC0qaYyEg==
print(httpx.get(raw).request.url)  # https://example.com/?TQC%2FKWC0qaYyEg==

[tomchristie]

I can provide some pointers here if you want to dig into this.

First up... which existing issues/discussions do we currently have in the project around this?

[lloydzhou]

in urllib.parse.urlparse keep the existing escape string
but httpx._urlparse.urlparse encoding the existing escape string

>>> from urllib.parse import urlparse
>>> from httpx._urlparse import urlparse as hurlparse
>>> url = 'https://example.com?sig=iGyXM1mSehS4ONN5%2B2GillQ%2BYawyh5EryToDT/Bzf90%3D'
>>> urlparse(url)
ParseResult(scheme='https', netloc='example.com', path='', params='', query='sig=iGyXM1mSehS4ONN5%2B2GillQ%2BYawyh5EryToDT/Bzf90%3D', fragment='')
>>> hurlparse(url)
ParseResult(scheme='https', userinfo='', host='example.com', port=None, path='', query='sig=iGyXM1mSehS4ONN5%252B2GillQ%252BYawyh5EryToDT%2FBzf90%253D', fragment=None)
>>>

[dhensen]

I'm downgrading. This has been driving me crazy and losing my sanity for the last couple of days. Why? Because working on multiple projects with different versions of httpx and also other http libraries. At least now we know there is a discrepancy in httpx that needs to be solved. Let me know if you need anything

poetry add "httpx<0.25"

Updating dependencies
Resolving dependencies... (0.6s)

Package operations: 0 installs, 2 updates, 0 removals

  • Downgrading httpcore (1.0.1 -> 0.17.3)
  • Downgrading httpx (0.25.1 -> 0.24.1)

[alexmaurizio]

Upping the thread here too, because we had to downgrade our entire stack to 0.24.1 from 0.25.x due to this change.
This change does break a number of external and internal APIs and there is not a sane way to work around this "feature" without locking to previous version.

[tomchristie]

Resolved in #2990