Fill form values in a web page via a Python script (not testing)

Posted on

Question :

Fill form values in a web page via a Python script (not testing)

I need to fill form values on a target page then click a button via Python. I’ve looked at Selenium and Windmill, but these are testing frameworks – I’m not testing. I’m trying to log into a 3rd party website programatically, then download and parse a file we need to insert into our database. The problem with the testing frameworks is that they launch instances of browsers; I just want a script I can schedule to run daily to retrieve the page I want. Any way to do this?

Answer #1:

You are looking for Mechanize

Form submitting sample:

import re
from mechanize import Browser

br = Browser()"")
# Browser passes through unknown attributes (including methods)
# to the selected HTMLForm (from ClientForm).
br["cheeses"] = ["mozzarella", "caerphilly"]  # (the method here is __setitem__)
response = br.submit()  # submit current form
Answered By: Vinko Vrsalovic

Answer #2:

Have a look on this example which use Mechanize: it will give the basic idea:

import re 
from mechanize import Browser
br = Browser()

# Ignore robots.txt
br.set_handle_robots( False )
# Google demands a user-agent that isn't a robot
br.addheaders = [('User-agent', 'Firefox')]

# Retrieve the Google home page, saving the response "" )

# Select the search box and search for 'foo'
br.select_form( 'f' )
br.form[ 'q' ] = 'foo'

# Get the search results

# Find the link to; why did we run a search?
resp = None
for link in br.links():
    siteMatch = re.compile( '' ).search( link.url )
    if siteMatch:
        resp = br.follow_link( link )

# Print the site
content = resp.get_data()
print content
Answered By: RATHI

Answer #3:

You can use the standard urllib library to do this like so:

import urllib

urllib.urlretrieve("", "somefile.html", lambda x,y,z:0, urllib.urlencode({"username": "xxx", "password": "pass"}))
Answered By: Clueless

Answer #4:

The Mechanize example as suggested seems to work. In input fields where you must enter text, use something like:

br["kw"] = "rowling"  # (the method here is __setitem__)

If some content is generated after you submit the form, as in a search engine, you get it via:

Answered By: Abhranil Das

Answer #5:

For checkboxes, use 1 & 0 as true & false respectively:

br["checkboxname"] = 1 #checked = true
br["checkboxname2"] = 0 #checked = false
Answered By: Ritesh Khandekar

Leave a Reply

Your email address will not be published. Required fields are marked *