Python requests
Sessions, timeouts, retries, streaming downloads and the mistakes that hang production.
Key PointsNo timeout means an infinite hang — always pass `timeout=(connect, read)` and call `raise_for_status()`.
GET and POST
Python
import requests
resp = requests.get("https://api.example.com/users",
params={"page": 1, "size": 20},
headers={"Authorization": "Bearer TOKEN"},
timeout=(3, 10))
resp.raise_for_status()
data = resp.json()
r = requests.post("https://api.example.com/users",
json={"name": "alice"}, # 自动 JSON 序列化 + Content-Type
timeout=10)resp.json() raises if the body is not JSON. Check resp.headers['content-type'] first when scraping HTML pages.Sessions
A Session reuses the TCP connection and keeps cookies and default headers — 2-5x faster for repeated calls to the same host.
Python
s = requests.Session()
s.headers.update({"User-Agent": "myapp/1.0", "Authorization": "Bearer TOKEN"})
for page in range(1, 6):
r = s.get(f"https://api.example.com/items?page={page}", timeout=10)
r.raise_for_status()
print(len(r.json()["items"]))Retries
Python
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
retry = Retry(total=3, backoff_factor=0.5,
status_forcelist=[429, 500, 502, 503, 504],
allowed_methods=["GET", "POST"])
s = requests.Session()
s.mount("https://", HTTPAdapter(max_retries=retry))Streaming Downloads
Python
with requests.get(url, stream=True, timeout=30) as r:
r.raise_for_status()
with open("big.zip", "wb") as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)Status Codes
| Code | Meaning | Action |
|---|---|---|
| 2xx | Success | Parse body |
| 301 / 302 | Redirect | Followed automatically |
| 400 | Bad request | Fix payload, do not retry |
| 401 / 403 | Auth failure | Refresh token |
| 429 | Rate limited | Back off per Retry-After |
| 5xx | Server error | Retry with backoff |
requests is synchronous. In async code use httpx.AsyncClient — calling requests inside a coroutine blocks the whole event loop.GET 与 POST
Python
import requests
resp = requests.get("https://api.example.com/users",
params={"page": 1, "size": 20},
headers={"Authorization": "Bearer TOKEN"},
timeout=(3, 10))
resp.raise_for_status()
data = resp.json()
r = requests.post("https://api.example.com/users",
json={"name": "alice"}, # 自动 JSON 序列化 + Content-Type
timeout=10)resp.json() 在响应不是 JSON 时会抛异常。抓 HTML 页面时先看 resp.headers['content-type']。会话
Session 复用 TCP 连接，并保留 cookie 与默认请求头 —— 对同一主机的连续调用快 2~5 倍。
Python
s = requests.Session()
s.headers.update({"User-Agent": "myapp/1.0", "Authorization": "Bearer TOKEN"})
for page in range(1, 6):
r = s.get(f"https://api.example.com/items?page={page}", timeout=10)
r.raise_for_status()
print(len(r.json()["items"]))重试
Python
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
retry = Retry(total=3, backoff_factor=0.5,
status_forcelist=[429, 500, 502, 503, 504],
allowed_methods=["GET", "POST"])
s = requests.Session()
s.mount("https://", HTTPAdapter(max_retries=retry))流式下载
Python
with requests.get(url, stream=True, timeout=30) as r:
r.raise_for_status()
with open("big.zip", "wb") as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)状态码处理
| 状态码 | 含义 | 应对 |
|---|---|---|
| 2xx | 成功 | 解析响应 |
| 301 / 302 | 重定向 | 默认自动跟随 |
| 400 | 参数错误 | 改参数，别重试 |
| 401 / 403 | 认证失败 | 刷新令牌 |
| 429 | 限流 | 按 Retry-After 退避 |
| 5xx | 服务端错误 | 退避重试 |
requests 是同步库。异步代码里请用 httpx.AsyncClient —— 在协程里调 requests 会阻塞整个事件循环。