httpx, unlike requests, sends null query parameters

[podj]

Hi, I've been using httpx as an async client and I noticed a difference in the behavior between httpx and the requests packages.

In the requests package, query params that are set to null are being ignored (excluded from URL):

response = requests.get("https://google.com", params={"q": None})
print(response.request.url)  # prints "https://www.google.com/"

while in httpx:

async with httpx.AsyncClient() as client:
    response = await client.get("https://google.com", params={"q": None})
    print(response.request.url)  # prints "https://google.com?q="

From what I see online, the standard is that if a query param is null then it should not be sent at all.
For example, FastAPI does not accept null as a query param value at all, even if the query param is said to be nullable (Optional) - only in cases in which the query param was never passed and the default value is null.

I now realize that passing the flag:
keep_blank_values=True
here: https://github.com/encode/httpx/blob/ee432c0d3098375df258b6d5844e7569080b2369/httpx/_urls.py#LL427C48-L427C48
is what causes this behavior.
I understand that it was introduced in this commit:
PR: #2354

as part of this issue: #2331
But, I believe it is actually incorrect :)

[karpetrosyan]

FastAPI can accept null as a query parameter value for example

server.py

from fastapi import FastAPI
app = FastAPI()

@app.get('/')
def root(a: str):
        print(f'a={a!r}')

curl

curl "http://127.0.0.1:8000/?a="

server stdout

INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [6589] using StatReload
INFO:     Started server process [6591]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
a=''
INFO:     127.0.0.1:43768 - "GET /?a= HTTP/1.1" 200 OK

If a value doesn't matter or is just blank, the server may use null values, which are valid.
Also see #2712

[podj]

FastAPI can accept null as a query parameter value for example

server.py

from fastapi import FastAPI
app = FastAPI()

@app.get('/')
def root(a: str):
        print(f'a={a!r}')

curl

curl "http://127.0.0.1:8000/?a="

server stdout

INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [6589] using StatReload
INFO:     Started server process [6591]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
a=''
INFO:     127.0.0.1:43768 - "GET /?a= HTTP/1.1" 200 OK

If a value doesn't matter or is just blank, the server may use null values, which are valid. Also see #2712

If you think about it: how can the server distinguish between a true null and just a string of "null"?
IMHO the only option is not passing the query param at all.

[zanieb]

In the example above ?a=null is distinct from ?a=.

Excluding the query parameter would be incorrect. See also https://stackoverflow.com/questions/4557387/is-a-url-query-parameter-valid-if-it-has-no-value

[tomchristie]

We treat null in the same way as "", which seems pretty intuitive to me.

>>> str(httpx.QueryParams(a="123", b="", c=None))
'a=123&b=&c='

The only adaptation that I might consider valid here would be differentiating between None and the empty string...

>>> str(httpx.QueryParams(a="123", b="", c=None))
'a=123&b=&c'

[podj]

Sorry for the delay.
I re-read our entire discussion and I must say that it still doesn't make sense.
In the example you sent here (curl "http://127.0.0.1:8000/?a=) it is clear that you send an empty string - not null.

Putting that aside, the main issue here is compatibility with the popular "requests" library.
httpx is said to be compatible with "requests" interface but actually, I opened this discussion because we encountered this error when migrating to httpx.

We solved it in our own codebase by emitting nulls from the params we send to other services anyway so it's not really urgent. Just thought it is worth mentioning for future companies who will migrate and encounter this error 🙏🏼

Thank you for your responses 🙌🏼