Today when I was create a proxy in Python with Flask which accepts the inbound HTTP GET request, and does the actual downloading work with “requests” and returns the requested file content, I found the downloading always failed, when I tried adding the headers from the original response made by “requests” to the flask response. I used a browser(Chrome) to do the test, it just reports HTTP 200 but nothing was downloaded. However, if I removed the “headers” part, the downloading succeeded but the content type is “text” which makes the binary content was shown in the browser window. The code looks like below:

1
2
3
4

@app.route(CONTEXT_PATH_OFFLINE_FETCH_DATA, methods=['GET'])
def offline_fetch_data(**args):
    response = requester_offline_data.request(args)
    return response.content, response.status_code, response.headers.items()

If I remove the “response.headers.items()” part, it works although the content type is incorrect. First I think the original “response.headers” is not accepted by the Flask response instance. I print the headers content and it looks like:

1

{'Date': 'Wed, 21 Sep 2022 10:01:02 GMT', 'Content-Type': 'application/octet-stream', 'Content-Length': '1393351', 'Connection': 'keep-alive', 'vary': 'origin', 'content-encoding': 'gzip', 'x-envoy-upstream-service-time': '22', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains;'}

It took me a lot of time on it. The turning point came after I printed the length of “response.content” and found the value was “1,581,568”, while the length in header output was “1,393,351”. That made me realize that the length of “response.content” was not the original length of the response of “requests”. I guessed the reason was that the original response of “requests” was gzip content, but after “requests” got the response, it extracted the full content. So if I tried to set the Flask response header with “Content-Length = 1393351”, it doesn’t match the actual size of the response content which was “1581568”. When the browser got such value and found the problem, it refused to save the downloaded content. That’s why the server didn’t report any error, while the browser didn’t download the content successfully. This is confirmed by the official document here.

After that I got the solution “here“ by googling “flask wrapper response”. The post provides the code which fixes the issue exactly.

1
2
3
4

@app.route(CONTEXT_PATH_OFFLINE_FETCH_DATA, methods=['GET'])
def offline_fetch_data(**args):
    response = requester_offline_data.request(args)
    return response.content, response.status_code, response.headers.items()

Note that, the “requests.get()” method is invoked inside “requester_offline_data.request(args)” which sets “stream=True” as the post mentioned.