Saiweb

Ramblings of a Sys admin

03 06 2009

Python urllib2, basic http authentication

Posted by: Buzz in python

I meant to write this up over a week ago now, basically the need arose for one of my Python scripts to use HTTP Basic authentication when grabbing the output from a URL.

An example script can be seen below:

Quick description, the script will attempt to connect to a URL and read the data supplied by the webserver, if a HTTP 401 error is returned (Authentication required) the script will then go on to attempt to authenticate using the credentials supplied.

Printing out to the console at each point.

Subversion: http://svn.saiweb.co.uk/branches/python/urllib2_httpbasic_auth.py

Highlighted source (at the time of writing)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

#!/usr/bin/env python
"""
Author: David Busby (http://saiweb.co.uk)
Program: Python HTTP Basic Auth Exa
Copyright: David Busby 2009. All rights reserved.
License: http://creativecommons.org/licenses/by-sa/2.0/uk/
"""

import urllib2, base64

""" URL List """
urls = {
0:{"url":"www.saiweb.co.uk/some/fictional/auth/area","user":"someuser","pass":"somepass"}
}

def main():
ulen = len(urls)
for i in range(0,ulen):
url = "http://%s" % (urls[i]["url"])
req = urllib2.Request(url)
try:
res = urllib2.urlopen(req)
headers = res.info().headers
data = res.read()
except IOError, e:
if hasattr(e, 'reason'):
err = "%s ERROR(%s)" % (urls[i]["url"],e.reason)
print err
elif hasattr(e, 'code'):
if e.code != 401:
err = "%s ERROR(%s)" % (urls[i]["url"],e.code)
print err
#401 = auth required error
elif e.code == 401:
base64string = base64.encodestring('%s:%s' % (urls[i]["user"], urls[i]["pass"]))[:-1]
authheader = "Basic %s" % base64string
req.add_header("Authorization", authheader)
try:
res = urllib2.urlopen(req)
headers = res.info().headers
data = res.read()
except IOError, e:
if hasattr(e, 'reason'):
err = "%s:%s@%s ERROR(%s)" % (urls[i]["user"],urls[i]["pass"],urls[i]["url"],e.reason)
print err
elif hasattr(e, 'code'):
err = "%s:%s@%s ERROR(%s)" % (urls[i]["user"],urls[i]["pass"],urls[i]["url"],e.code)
print err
else:
err = "%s query complete" % (urls[i]["url"])
print err
else:
err = "%s query complete" % (urls[i]["url"])
print err

if __name__ == "__main__":
main()

NOTES: This script does not check the authentication type, it always assumes it is HTTP BASIC, HTTP DIGEST for example is not compatible with this script, though there is no reason why you can not get the Auth type form the headers returned by the server and write in a Digest auth method.

Tags: auth, basic, http, python, urllib2

This entry was posted on Wednesday, June 3rd, 2009 at 11:43 am and is filed under python. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.