AsyncClient doesn't decompress based on Content-Encoding response headers #1814
-
|
When I use Error details import httpx
import asyncio
async def run():
async with httpx.AsyncClient(headers={"accept-encoding": "gzip, deflate, br"}) as client:
response = await client.get("https://www.indeed.com/")
response_code = response.status_code
if response_code == 200:
response_text = response.content.decode("utf-8")
else:
response_text = None
return response_text, response_codeError: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 4: invalid continuation byteManual Decompress Fix import brotli # use gzip if content-encoding is gzip
import httpx
import asyncio
async def run():
async with httpx.AsyncClient(headers={"accept-encoding": "gzip, deflate, br"}) as client:
response = await client.get("https://www.indeed.com/")
response_code = response.status_code
if response_code == 200:
response_text = brotli.decompress(response.content).decode("utf-8")
else:
response_text = None
return response_text, response_code
asyncio.run(run())Is this a bug or intended behaviour? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 3 replies
-
|
Hi - I'm not able to reproduce the behaviour you're seeing... Installation... $ pip install httpx[brotli]Usage... >>> import httpx
>>> r = httpx.get("https://www.indeed.com/")
>>> r.text
'<!DOCTYPE html>\n<html dir="ltr" lang="en">\n [...]'
>>> r.request.headers["accept-encoding"]
'gzip, deflate, br' |
Beta Was this translation helpful? Give feedback.
-
|
Oh! Have you tried the AsyncClient with the headers? |
Beta Was this translation helpful? Give feedback.
-
|
Well, this works fine?... import asyncio
import httpx
async def main():
async with httpx.AsyncClient() as client:
r = await client.get("https://www.indeed.com/")
print(r.text)
print(r.request.headers["accept-encoding"])
asyncio.run(main()) |
Beta Was this translation helpful? Give feedback.
-
|
Thanks. I reinstalled everything and it works fine with brotli support. |
Beta Was this translation helpful? Give feedback.
Hi - I'm not able to reproduce the behaviour you're seeing...
Installation...
Usage...