Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

send_progress_in_http_headers = 1 is not really needed #48

Open
x-yuri opened this issue Feb 8, 2024 · 0 comments
Open

send_progress_in_http_headers = 1 is not really needed #48

x-yuri opened this issue Feb 8, 2024 · 0 comments

Comments

@x-yuri
Copy link

x-yuri commented Feb 8, 2024

You only need X-ClickHouse-Summary. And that is returned with send_progress_in_http_headers = 0. I.e. as far as I can see send_progress_in_http_headers can be safely changed to 0. With send_progress_in_http_headers = 1, a relatively slow query (2 sec), and clickhouse being behind nginx, the latter says "upstream sent too big header" because clickhouse produces a lot of "progress" headers:

$ curl -sSv 'http://xx.yyy.xx.yyy:8123/?send_progress_in_http_headers=1' -d 'select ..., count(*) from ... group by ... order by ... limit 1' -X GET -u '...:...'
* Expire in 0 ms for 6 (transfer ...)
*   Trying xx.yyy.xx.yyy...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer ...)
* Connected to xx.yyy.xx.yyy (xx.yyy.xx.yyy) port 8123 (#0)
* Server auth using Basic with user '...'
> GET /?send_progress_in_http_headers=1 HTTP/1.1
> Host: xx.yyy.xx.yyy:8123
> Authorization: Basic ...
> User-Agent: curl/7.64.0
> Accept: */*
> Content-Length: 94
> Content-Type: application/x-www-form-urlencoded
> 
} [94 bytes data]
* upload completely sent off: 94 out of 94 bytes
< HTTP/1.1 200 OK
< Date: Thu, 08 Feb 2024 13:22:37 GMT
< Connection: Keep-Alive
< Content-Type: text/tab-separated-values; charset=UTF-8
< X-ClickHouse-Server-Display-Name: ...
< Transfer-Encoding: chunked
< X-ClickHouse-Query-Id: ...
< X-ClickHouse-Format: TabSeparated
< X-ClickHouse-Timezone: UTC
< Keep-Alive: timeout=3
< X-ClickHouse-Progress: {"read_rows":"1111953","read_bytes":"8895624","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"9434898","read_bytes":"75479184","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"18412695","read_bytes":"147301560","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"27260817","read_bytes":"218086536","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"35690512","read_bytes":"285524096","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"44667166","read_bytes":"357337328","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"53121310","read_bytes":"424970480","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"61969813","read_bytes":"495758504","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"70226587","read_bytes":"561812696","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"78697369","read_bytes":"629578952","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"86824087","read_bytes":"694592696","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"94819987","read_bytes":"758559896","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Summary: {"read_rows":"101288000","read_bytes":"810304000","written_rows":"0","written_bytes":"0","total_rows_to_read":"101288000","result_rows":"0","result_bytes":"0"}
< 
{ [11 bytes data]
* Connection #0 to host xx.yyy.xx.yyy left intact
165	1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant