I’m trying to connect to my corporate’s internal webpages through the requests package, but since python does not use the windows default trusted certificates the connection is denied. I found out that wincertstore can be used to fetch the windows default certificates. But I’m still not sure how to use that along with the my request. Below is the code I have tried so far………….
import requests, socket, atexit, ssl, wincertstore from requests.auth import HTTPBasicAuth certfile = wincertstore.CertFile() certfile.addstore("CA") certfile.addstore("ROOT") atexit.register(certfile.close) ssl_sock = ssl.wrap_socket(s,ca_certs=certfile.name, cert_reqs=ssl.CERT_REQUIRED) requests.get(url)
I get the following error……………….
requests.exceptions.SSLError: HTTPSConnectionPool(host='myhost', port=443): Max retries exceeded with url: myurl (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))
I am able to use wget on the same url and download the content.
wget --no check certificate --user=my username --password=my password URL
But I am not interested in downloading the content as I only need to scrape a small portion of the webpage content.
Pythin version = 3.6.5
Wincertstore link – Link
Thanks in advance for your help…………..
I had a similar issue and fixed it using the python-certifi-win32 package:
pip install python-certifi-win32
now you can just use:
and the certificate is checked using the Windows Certificate Store.
This only works if the certificate is installed in the Windows Certificate Store…
This is all explained in the SSL Cert Verification section of the
requests uses the certs from
certifi if present, falling back to whatever
urllib3 thinks is your OS cert store, which itself falls back on whatever Python thinks it is (although in older versions it often didn’t).
Your company apparently has a private, maybe even self-signed, cert, which isn’t going to be in
certifi. It might be in the Windows cert store—in which case
urllib3 should automatically pick it up—but I suspect that it isn’t. Maybe the cert is installed directly into some custom browser setup your IT department forces you to use, instead of into the OS store. Or maybe it’s not installed at all. (You didn’t mention being able to access this site in a browser without seeing a broken-lock icon…)
--no check certificate (or, more likely,
wget, so you’re just not verifying SSL. And if you want to do the same thing in
requests, that’s just:
If you’re pretty sure that you do have the cert installed, even though
wget can’t find it… well, your code isn’t going to work as written. Here’s what would work:
- Ignore the cert and just disable validation, as shown above.
- Figure out where the relevant cert actually is installed and how to load it, and:
- Pass it as the
verifyargument on every
- Set it up somewhere statically and pass it in an environment variable.
- Install it into your default cert store so everything works automatically.
- Write an
HTTPAdapterthat installs it into your
- Pass it as the
First, your code is just trying to get the default cert in exactly the same way Python already does. That
wincertstore module is just a backport of what’s already builtin to Python 3.4+.
Second, all your code is doing is getting a cert, using it to create an SSL socket, ignoring that socket, and telling
requests to do its normal thing. That isn’t going to help anything. If you want to pass a cert to
requests, you either do this:
… or put it in the environment variable
… or do the
HTTPAdapter code that I showed you in chat (and which you found an old, non-working version of somewhere unspecified). See
HTTPAdapter in the docs if you actually want to do that.