Python:urllib
Example
다음은 Request를 사용하여 PUT 요청을 수행하는 예입니다:
import urllib.request
DATA = b'some data'
req = urllib.request.Request(url='http://localhost:8080', data=DATA, method='PUT')
with urllib.request.urlopen(req) as f:
pass
print(f.status)
print(f.reason)
print(u.getcode())
print(u.headers)
print(u.read())
POST 로 josn 데이터 요청 예제:
req = Request("http://localhost:48888/live", headers={"Content-Type": "application/json"}, method="POST")
data = b'{"live": [], "history": []}'
with urlopen(req, data) as f
f.read()
# ...
application/x-www-form-urlencoded
문자열 인자(application/x-www-form-urlencoded
MIME)는 urllib.parse.parse_qsl
또는 urllib.parse.parse_qs
를 사용하여 파싱할 수 있다.
쿼리 문자(URL 인코딩)로 변환하고 싶을 경우 urllib.parse.urlencode
를 사용하면 된다.
urlencode 의 결과로 반환 받는 ParseResult 객체는 urllib.parse.urlunparse
를 사용하면 원본 URL 텍스트로 반환받을 수 있다.
Troubleshooting
CERTIFICATE_VERIFY_FAILED
Python에서 https접속시 아래와 같은 에러가 발생할 수 있다.
/usr/local/c2core/lib/python3.7/site-packages/torch/hub.py:429: UserWarning: torch.hub._download_url_to_file has been renamed to torch.hub.download_url_to_file to be a public API, _download_url_to_file will be removed in after 1.3 release
_download_url_to_file will be removed in after 1.3 release')
Traceback (most recent call last):
File "/usr/local/c2core/lib/python3.7/urllib/request.py", line 1317, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "/usr/local/c2core/lib/python3.7/http/client.py", line 1229, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/c2core/lib/python3.7/http/client.py", line 1275, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/c2core/lib/python3.7/http/client.py", line 1224, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/c2core/lib/python3.7/http/client.py", line 1016, in _send_output
self.send(msg)
File "/usr/local/c2core/lib/python3.7/http/client.py", line 956, in send
self.connect()
File "/usr/local/c2core/lib/python3.7/http/client.py", line 1392, in connect
server_hostname=server_hostname)
File "/usr/local/c2core/lib/python3.7/ssl.py", line 412, in wrap_socket
session=session
File "/usr/local/c2core/lib/python3.7/ssl.py", line 853, in _create
self.do_handshake()
File "/usr/local/c2core/lib/python3.7/ssl.py", line 1117, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "webcam.py", line 80, in <module>
main()
File "webcam.py", line 64, in main
min_image_size=args.min_image_size,
File "/home/aaa/Project/_c2core/script/python/maskrcnn_benchmark/demo/predictor.py", line 117, in __init__
_ = checkpointer.load(cfg.MODEL.WEIGHT)
File "/usr/local/c2core/lib/python3.7/site-packages/maskrcnn_benchmark-0.1-py3.7-linux-x86_64.egg/maskrcnn_benchmark/utils/checkpoint.py", line 61, in load
checkpoint = self._load_file(f)
File "/usr/local/c2core/lib/python3.7/site-packages/maskrcnn_benchmark-0.1-py3.7-linux-x86_64.egg/maskrcnn_benchmark/utils/checkpoint.py", line 129, in _load_file
cached_f = cache_url(f)
File "/usr/local/c2core/lib/python3.7/site-packages/maskrcnn_benchmark-0.1-py3.7-linux-x86_64.egg/maskrcnn_benchmark/utils/model_zoo.py", line 55, in cache_url
_download_url_to_file(url, cached_file, hash_prefix, progress=progress)
File "/usr/local/c2core/lib/python3.7/site-packages/torch/hub.py", line 430, in _download_url_to_file
download_url_to_file(url, dst, hash_prefix, progress)
File "/usr/local/c2core/lib/python3.7/site-packages/torch/hub.py", line 384, in download_url_to_file
u = urlopen(url)
File "/usr/local/c2core/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/c2core/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/local/c2core/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/usr/local/c2core/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/local/c2core/lib/python3.7/urllib/request.py", line 1360, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/local/c2core/lib/python3.7/urllib/request.py", line 1319, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>
개정된 PEP 467에 따라 모든 https 통신은 필요한 인증서와 호스트명을 기본으로 체크하도록 되어 있어서 그렇다. 영향을 받는 라이브러리는 urllib, urllib2, http, httplib 이다.
인증서를 설치하면 된다.
코드로 직접 해결하고 싶다면 ssl._create_unverified_context()
를 urllib.request.urlopen
의 context 파라미터로 넘겨주면 된다:
context = ssl._create_unverified_context()
response = urllib.request.urlopen(requests, data=data.encode('utf-8'), context=context)
또는 기본 컨텍스트를 아래와 같이 변경하면 된다.
import ssl
try:
_create_unverified_https_context = ssl._create_unverified_context
except AttributeError:
# Legacy Python that doesn't verify HTTPS certificates by default
pass
else:
# Handle target environment that doesn't support HTTPS verification
ssl._create_default_https_context = _create_unverified_https_context
curl이 없을 경우 대체 방법
python -c "import urllib.request as r; exit(0 if r.urlopen('http://www.python.org/').status == 201 else 1)"