The Python Requests module is a pretty cool and easy way to establish HTTP/HTTPS connections. However, if you ever try to connect to a server using HTTPS and the certificate is not trusted, you will probably get an error that looks something like this:
>>> import requests
>>> r = requests.get('https://fullyqualifiedurl.com')
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/site-packages/requests/api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 497, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
Before continuing on, I want to reiterate the disclaimer at the top of the page:
DISCLAIMER: You should NOT send any sensitive traffic to untrusted hosts on the Internet. This is method should only be used for troubleshooting or if you have independently verified the identify of the server you are connecting to.
Now that that's taken care of, the documentation for Requests says to pass the verify=False parameter when calling the requests.get method. Let's try that out:
>>> import requests
>>> r = requests.get('https://fullyqualifiedurl.com', verify=False)
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:843: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
Now we get a different error that has a link to documentation for urllib3. Since requests uses urllib3 under the hood, you need to disable certificate checking with urllib3 as well:
>>> import requests
>>> import urllib3
>>> urllib3.disable_warnings()
>>> r = requests.get('https://fullyqualifiedurl.com', verify=False)
>>> r.status_code
200
If you're still getting an error, it could be because your system is using the version of urllib3 that is bundled with requests. This should get things all squared away for you:
>>> import requests
>>> from requests.packages import urllib3
>>> urllib3.disable_warnings()
>>> r = requests.get('https://fullyqualifiedurl.com', verify=False)
>>> r.status_code
200
I think what python is doing behind the scenes is that if urllib3 is already installed, requests will use that version. But, if urllib3 is not already installed, requests will use its bundled version of urllib3. If you are writing code that might be used on different computers and you aren't sure what is installed, you can use this try/except block to import whichever version of urllib3 is applicable:
try:
import urllib3
except ImportError:
from requests.packages import urllib3
