Requests get produces incomplete json in Heptapod CI
I can't use PDM in Heptapod CI because of a strange behavior of requests
that I cannot reproduce elsewhere. In particular, using the same Docker image locally on my PC do not lead to this bug.
Example of job for which I get that https://foss.heptapod.net/fluiddyn/fluidsim/-/jobs/2206610
I use a very simple Python script containing
from datetime import datetime
from requests import Session
session = Session()
print("before get:", datetime.now())
resp = session.get(
"https://pypi.org/simple/flit-core/",
headers={
"Accept": "application/vnd.pypi.simple.v1+json",
"Cache-Control": "no-cache",
},
timeout=120,
)
print("after get:", datetime.now())
print(resp)
print(resp.content[-400:])
print(resp.json()["versions"])
On Heptapod CI, I get:
before get: 2024-01-03 10:22:01.244619
after get: 2024-01-03 10:22:01.277003
<Response [200]>
b'g/packages/4c/8f/bed80c03f71cb3a2935882f391b53d2510c359191e5e0361650fa02d1365/flit_core-3.1.0.tar.gz","yanked":false},{"core-metadata":false,"data-dist-info-metadata":false,"filename":"flit_core-3.2.0-py3-none-any.whl","hashes":{"sha256":"6f25843e908dfc3e907b6b9ee71e3d185bcb5aebab8c3431e4e34c261e5ff1b5"},"requires-python":">=3.4","size":45693,"upload-time":"2021-03-21T21:20:19.175500Z","url":"http'
Traceback (most recent call last):
File "/home/appuser/.local/lib/python3.9/site-packages/requests/models.py", line 960, in json
return complexjson.loads(self.content.decode(encoding), **kwargs)
File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.9/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 10236 (char 10235)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/builds/fluiddyn/fluidsim/tmp_bug_unearth.py", line 20, in <module>
print(resp.json()["versions"])
File "/home/appuser/.local/lib/python3.9/site-packages/requests/models.py", line 968, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Unterminated string starting at: line 1 column 10236 (char 10235)
The json data is incomplete, which leads to the exception.
I'm looking for ideas to understand this annoying issue...