Firefox 29 and the HTTP Response Timeout

I’ve been using Selenium with Python for some time to automate some work in the browser that I absolutely can’t accomplish any other way (and believe me, I wish I could).  So when Firefox 29 came out, I upgraded, ran a quick test, and everything seemed fine.  However, my script that ran overnight bombed during the browser phase.

Before continuing, I should describe a little bit of what I expect the browser to do during this script.  Here is the basic sequence of events:

1. Open a vendor’s website and log in.

2. Initiate a database backup by either clicking a link or directly running the Javascript the link runs (either way works).

3. Wait (60 minutes).

4. Initiate a download of the backed up database by either clicking a link or directly running the Javascript the link runs (again, either way works).

5. Wait (20 minutes).

6. Close the browser and pass control back to the main script.

The script was throwing an unknown exception at step 4.  On investigation, I found out the reason for this was the browser generating a timeout message a few minutes into the wait period at step 3, which made step 4 impossible.  The message was something like this:

Naturally, I started scouting around for all the timeout settings in Firefox, trying to figure out which one needed to change.  There are several timeout settings in Firefox (see here and here, for example), but none of these seemed to solve the problem.  While searching, I stumbled on this document showing changes between Firefox 28 and 29, and it turned out to be gold.  In particular, notice this change:

So, the network.http.response.timeout went from not being set in version 28 to a default limit of 300 (5 minutes) in version 29.  You probably won’t notice this, unless (like me) you work with a page that has a long-running process that you have to wait more than 5 minutes for a response from.  To solve this, you can update this setting manually in about:config.  Better yet, you can do it programmatically through Selenium with this code:

ffprofile = webdriver.FirefoxProfile()

ffprofile.set_preference("network.http.response.timeout", 30000) #replace 30000 with the number of seconds you want the max delay to be

browser = webdriver.Firefox(firefox_profile=ffprofile)

Problem solved.