Writing automated tests with Django&Selenium

on Apr 25, 2013 by Ramona

     Getting our Selenium tests to run faster can be complicated at times, because there are a number of factors to be taken into consideration. Debugging these tests is time consuming and the end result is almost always the same one - a large part of the tests needs to be refactored.

Tags: testing django selenium | Category: testing , python , django Back To Top

Testing your django app with webtest

on Apr 25, 2013 by Danu

     I’ve been watching both presentations that Carl Meyer held at Pycon 2012/13 and I highly recommend them if you want a deep dive into writing tests with django. They outline some very good principles for writing effective and maintainable tests. They also highlight a suite of test utilities and frameworks which help you in writing better tests. Among the others, Webtest caught my attention via django-webtest for writing integration/functional tests.

Tags: django functional webtest integration | Category: testing , python , django Back To Top

Running python tests with GhostDriver in Travis CI

on Feb 18, 2013 by Ramona

Selenium is great by itself, but its API grows every day - it’s not obvious anymore whether to use “type” or “typeKeys”. 

So Google has come up with a new API - WebDriver - which is integrated within Selenium 2.0. Rather than being a JavaScript application running within the browser, WebDriver uses whichever mechanism is most appropriate to control the browser - for Firefox, it’s an extension; for IE, WebDriver makes use of IE's Automation controls and so on. By changing the way the browser is controlled, WebDriver manages to overcome some of the restrictions put in place by the JavaScript security model.

1. PhantomJS

• ​On Windows:
  • Download PhantomJS from here and extract it.
  • Add the folder to the system path and run phantomjs --version in a new command prompt. You should see the actual version (just to verify there are no errors so far).

• On Linux (Ubuntu) - command line:

cd /usr/local/share
wget http://phantomjs.googlecode.com/files/phantomjs-1.8.1-linux-x86_64.tar.bz2
tar xjf phantomjs-1.8.1-linux-x86_64.tar.bz2
sudo ln -s /usr/local/share/phantomjs-1.8.1-linux-x86_64/bin/phantomjs /usr/local/share/phantomjs
sudo ln -s /usr/local/share/phantomjs-1.8.1-linux-x86_64/bin/phantomjs /usr/local/bin/phantomjs
sudo ln -s /usr/local/share/phantomjs-1.8.1-linux-x86_64/bin/phantomjs /usr/bin/phantomjs

It is important that you have installed version 1.8*, because GhostDriver is not integrated into PhantomJS’s older versions.

2. Start GhostDriver by setting phantomjs --webdriver=PORT, where PORT can be any available port. You should see a successful message, indicating that GhostDriver is running on port PORT.
3. Configure your test environment to use PhantomJS:

• For example, create a folder structure like:
  • headless_testing, containing a subfolder test, where I will add my selenium scripts written with Python
  • Add the following two files inside test folder:
    • base_test.py (but delete the first two lines - there's no need for these as we will place all py files in the same folder):

import sys
sys.path.insert(0, "utils")

• properties.py

• Modify the base_test.py file as follows:
  • replace port 8080 with the port where GhostDriver is running for you (line 59)
  • replace open("../config.ini") with open("test/config.ini") (line 40)
• ​​Add your own selenium python script to the same test folder. Make sure your methods are named as test_*, so that you can run them using unittest module.
• Add a config.ini file in the test folder, containing driver=phantomjs. In this way, you’re telling Selenium it should use the headless PhantomJS browser. If you don't want your config.ini file to be mixed up with the python scripts, you could add it to a separate folder, and modify the base_test.py file accordingly.

• pip install discover ( the test discovery mechanism and load_tests protocol for unittest backported from Python 2.7 to work with Python 2.4 or more recent (including Python 3).)

• Log in to Travis with GitHub and enable service hooks for the repositories you want added to Travis. More details here.
• On your local machine, drill inside headless_testing folder and add a file called .travis.yml - this is the config file that will tell Travis how to run your selenium python scripts:

language: python

python:
- "2.7"

install: "pip install -r requirements.txt --use-mirrors"

script:
- "python -m unittest discover -s test --pattern=test*.py"

before_script:
- "python test/start_ghostdriver.py"

• Add this to the requirements.txt file (separate lines): selenium==2.28.0 and discover==0.4.0
• The start_ghostdriver.py script is a script I created in order to be able to start GhostDriver and get the control back (so that you are able to run other commands from the command line as well). The content of this script is:

import os
os.system('phantomjs --webdriver=PORT 2>&1 > log.txt &')
#os.system('phantomjs --webdriver=PORT 2>&1 > /dev/null &')

• Travis comes with PhantomJS configured, so no other action is needed in this direction.​​
• Make sure to add the start_ghostdriver.py file inside headless_testing/test folder. What it does is that it starts a process in the background and it captures the output in the log.txt file. Adding the & ensures the fact that you will gain control back in the command line. Only two lines will be written inside the log file, so you shouldn’t worry about it too much.
• Push the content of the headless_testing folder to GitHub (you might want to use a .gitignore file as well).

Tags: travis testing python selenium automated ci | Category: testing , python Back To Top

Functional tests with lettuce and django test client

on Feb 01, 2013 by Danu

     When running a functional test, you fire up “browser” and do the "same" actions as a real user (or API client). There are different “browsers” for testing your applications, some of them are real, like selenium and some of them are less real, like django test client. Depending on the context, each of them has its pros and cons.

- django test client is very fast, since you don’t need a browser engine, real http listening and so on, but JavaScript and/or Ajax views are not tested

- selenium is a browser-driving library, opens a real browser and tests rendered HTML alongside with behavior of Web pages. Write selenium tests for Ajax, other JS/server interactions.

A complete test suite should contain both test types.

Functional tests and Django

Write functional/integration/system (has more names than it needs) tests for views. Unit testing the view is hard because views have many dependencies (templates, db, middleware, url routing etc). 

You should definitely have django functional tests but chances are you should have fewer than you have now.  See the software testing pyramid by Alister Scott which provides solid approach to automated testing and shows the mix of testing a team should aim for.

Functional tests:

- test that the whole integrated system works, catch regressions

- they tend to be slow

- will catch bugs that unit tests will not, but it's harder to debug

- write fewer (more unit tests)

So what can you test with django-test client ?

- the correct view is executed for a given url

- simulate post, get, head, put etc. requests

- the returned content has the expected values (you can use beautiful soup, lxml or html5lib for parsing the content)

Example of functional test with lettuce and django test client

Feature: Register
    In order to get access to app
    A user should be able to register
 
 
Scenario: User registers
    Given I go to the "/register/" URL
    When I fill register form with:
        | username | email       | password1 | password2 |
        | danul    | dan@dan.com | test123   | test123   |
    And I submit the data
    Then I should see "Check your email"
    And I should receive an email at "dan@dan.com" with the subject "Activate your djangoproject.com account - you have 7 days!"
    And I activate the account
    Then I should see "Congratulations!"
 
 
Scenario: Users login
    Given following users exist
      | username | password |
      | danu     | test123  |
      | lulu     | test123  |
    When I go to the "/login/" URL
    And I login as user "danu"
    Then I should see "Welcome, danu. Thanks for logging in."

Steps

import re

from bs4 import BeautifulSoup

from lettuce import *
from django.core import mail
from nose.tools import assert_equals

from django.contrib.auth.models import User


@step(u'I go to the "(.*)" URL')
def i_go_to_the_url(step, url):
    response = world.browser.get(url)
    assert_equals(response.status_code, 200)
    world.html = BeautifulSoup(response.content)

@step(u'When I fill register form with:')
def when_i_fill_in_user_data_with(step):
    for data in step.hashes:
        world.data = data
    assert_equals(len(world.html.select('form')), 1)
    assert_equals(len(world.html.find_all('input', 'required')), 4)

@@step(u'And I submit the data')
def and_i_submit_the_data(step):
    world.response = world.browser.post('/register/', world.data, follow=True)
    assert_equals(
        User.objects.filter(username=world.data['username']).exists(), True
    )
    assert_equals(world.response.status_code, 200)

@step(u'I should see "(.*)"')
def i_should_see(step, expected_response):
    html = BeautifulSoup(world.response.content)
    expected_text = html.find('h1').get_text()
    assert_equals(expected_text, expected_response)

@step(u'And I should receive an email at "([^"]*)" with the subject "([^"]*)"')
def i_should_receive_email_with_subject(step, address, subject):
    assert_equals(mail.outbox[0].to[0], address)
    assert_equals(mail.outbox[0].subject, subject)

@step(u'And I activate the account')
def and_i_activate_the_account(step):
    activation_url = re.findall(
        r'http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+',
        mail.outbox[0].body
    )
    world.response = world.browser.get(activation_url[0], follow=True)
    assert_equals(world.response.status_code, 200)

@step(u'Given following users exist')
def given_following_users_exist(step):
    for user_hash in step.hashes:
        user_data = user_hash.copy()
        password = user_data.pop("password", None)
        user, created = User.objects.get_or_create(**user_data)
        user.set_password(password)
        user.save()

@step(u'And I login as user "([^"]*)"')
def and_i_login_as_user_group1(step, username):
    if not User.objects.filter(username=username).exists():
        User.objects.create_user(
            username=username,
            password='test123',
        )
    assert_equals(len(world.html.select('form')), 1)
    assert_equals(len(world.html.find_all('input')), 3)
    world.response = world.browser.post(
        '/login/',
        {'username': username, 'password': 'test123'},
        follow=True
    )
    assert_equals(world.response.status_code, 200)

terrain

import logging

from django.conf import settings
from django.core.management import call_command
from django.test import Client
from django.db import connection
#from django.test.utils import setup_test_enviroment, teardown_test_environment

from lettuce import *


@before.all
def initial_setup():
    logging.info("Loading django's test client ...\n")
    world.browser = Client()

    logging.info("Setting up a test database ...\n")

    try:
        from south.management.commands import patch_for_test_db_setup
        patch_for_test_db_setup()
    except ImportError:
        pass

    # Setup test environment / called by harvest.py
    #setup_test_enviroment()

    world.testdb = connection.creation.create_test_db(
        verbosity=2,
        autoclobber=False,
    )


@after.all
def teardown_browser(total):
    logging.info("Destroy test database ...\n")
    connection.creation.destroy_test_db(world.testdb, verbosity=2)

    # Tear Down the test environment / called by harvest.py
    #teardown_test_enviroment()

What's next ?

Well ... WebTest  :) (http://pyvideo.org/video/699/testing-and-django)

Be a good person and write functional tests. Functional testing is something that every app needs, no testing strategy is complete without high-level tests to ensure the entire programming system works together.

Source code for this article:

https://github.com/danclaudiupop/django-lettuce-labtests

Tags: test lettuce django client bs4 selenium bdd | Category: testing , python Back To Top

Testing registration scenario against django with lettuce and selenium webdriver

on Jan 14, 2013 by Danu

Installation

Settings

webdriver

We’ll use Firefox by default since it has webdriver support built-in. If you want to use Chrome you’ll need to download the chrome driver separately. 

lettuce_settings

First let's extend django default settings with lettuce specific configurations. This way we avoid running tests against dev db and declare another db to run tests against. This is actually bad, the lack of test database integration with django and the fact that lettuce server is used over LiveServerTestCase may cause reliability issues in the future (see django test database). Add lettuce.django to INSTALLED_APPS. You can also specify the app names in LETTUCE_APPS to run tests against, so that you won’t need to type the app name as command-line parameters. 

from settings import *


DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'lettuce.db',                   # Or path to database file if using sqlite3.
        'USER': '',                             # Not used with sqlite3.
        'PASSWORD': '',                         # Not used with sqlite3.
        'HOST': '',                             # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '',                             # Set to empty string for default. Not used with sqlite3.
    }
}

INSTALLED_APPS += (
    'lettuce.django',
)

LETTUCE_APPS = (
    'registerap',
)

Test structure

Lettuce will look for a features folder inside every installed app. For each app you want to create lettuce tests you have to create a sub folder called features. Our features are stored under registerapp folder. The "feature" file contains scenarios, while ".py" file contains python code executed for steps used in scenarios.

register.feature

Feature: Register
    In order to get access to app 
    A user should be able to register


Scenario: User registers
    Given I go to the "/register" URL
    When I fill in "username" with "danul"
    And I fill in "email" with "danclaudiupop@gmail.com"
    And I fill in "password1" with "test123"
    And I fill in "password2" with "test123"
    And I press "submit"
    Then I should see "Check your email"
    And I should receive an email at "danclaudiupop@gmail.com" with the subject "Activate your djangoproject.com account - you have 7 days!"
    And I activate the account
    Then I should see "Congratulations!"

Scenario: User logs in successfully
    Given I go to the "/login" URL
    When I fill in "username" with "danul"
    And I fill in "password" with "test123"
    And I press "submit"
    Then I should see "Welcome, danul. Thanks for logging in."

register.py

We are importing django_url utility function so we can easily access web pages within steps using relative urls. We also importing the email backend that sends mails to a multiprocessing queue. Because Django server runs in a different process than lettuce you are not able to use the EMAIL_BACKEND specified in settings file, not to mention that harvest.py is using django.test.utils.setup_test_environment that overrides email backend to locmem

import re

from lettuce import step, world
from lettuce.django import django_url
from lettuce.django import mail
from nose.tools import assert_equals


@step(u'I go to the "(.*)" URL')
def i_go_to_the_url(step, url):
    world.response = world.browser.get(django_url(url))

@step(u'I fill in "(.*)" with "(.*)"')
def i_fill_in(step, field, value):
    world.browser.find_element_by_name(field).send_keys(value)

@step(u'I press "(.*)"')
def i_press(step, button_id):
    button = world.browser.find_element_by_id(button_id)
    button.click()

@step(u'I should see "(.*)"')
def i_should_see(step, expected_response):
    h1 = world.browser.find_element_by_tag_name('h1')
    assert_equals(h1.text, expected_response)

@step(u'And I should receive an email at "([^"]*)" with the subject "([^"]*)"')
def i_should_receive_email_with_subject(step, address, subject):
    message = mail.queue.get(True, timeout=5)
    world.email_body = message.body
    assert_equals(message.subject, subject)
    assert_equals(message.recipients(), [address])

@step(u'And I activate the account')
def and_i_activate_the_account(step):
    activation_url = re.findall(
        r'http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+',
        world.email_body
    )
    world.response = world.browser.get(activation_url)

terrain.py 

Its about setup and teardown. We setup the browser instance using webdriver. We prepare the test environment by running the django syncdb command which creates the database tables as well loading any fixtures named initial_data. Teardown test environment by destroying the test database. Different hooks and commands can be used, for e.g: before each scenario load specific fixtures using loaddata.

from django.core.management import call_command
from django.db import connection
from django.conf import settings

from lettuce import before, after, world
from selenium import webdriver


@before.all
def initial_setup():
    call_command('syncdb', interactive=False, verbosity=1)
    world.browser = webdriver.Firefox()

@after.all
def teardown_browser(total):
    connection.creation.destroy_test_db(settings.DATABASES['default']['NAME'])
    world.browser.quit()

Running lettuce tests

All that’s left is running the tests:

python manage.py harvest tests/lettuce_selenium_tests --settings=conf.settings_lettuce

Source code for this project can be found on github:

https://github.com/danclaudiupop/django-lettuce-labtests

See also:

Web development fun with Lettuce and Django

Tags: webdriver django selenium lettuce bdd | Category: testing , python Back To Top

Latests Posts

• Writing automated tests with Django&Selenium

---

• Testing your django app with webtest

---

• Running python tests with GhostDriver in Travis CI

---

• Functional tests with lettuce and django test client

---

• Testing registration scenario against django with lettuce and selenium webdriver

---

Quick links

• Context Driven Testing

---

• Nat On Testing

---

• Test Obsessed by Elisabeth Hendrickson

---

• CheezyWorld by Jeff Morgan

---

• WatirMelon by Alister Scott

---

• Agile Testing with Lisa Crispin

---

• Google Testing Blog

---