Testing

Guidelines

  • All functionality should be covered by automated tests to shorten our release process and to guarantee good product quality.
  • Test code should be the same quality as production code.
  • The test runs should be as fast as possible.
  • Go to great lengths to write unit tests instead of twill or windmill tests.

Test Servers

  • We need dedicated test servers to test all major platforms to cover differences due to different versions of Python, trac and other system components.
  • Currently the test server executes all tests using 'python setup.py test'.
  • We want to switch to nosetests as test runner in the future because it provides a nice additions over Python's unittest module (e.g. run tests in parallel to use multiple CPUs, skip tests, select tests with command line criteria). To make this possible, please make sure all test files end with '_test.py'
  • Currently all tests must be runnable with our old, self-written testfinder class.

Executing unit tests locally

Python:

  • You need to install 'nose' (aka 'nosetests') locally. 0.10 is known to work, theoretically 0.11 should work as well.
  • To execute the tests go to the root directory of the project and either run
    • nosetests without any commands to run all tests (unittests, twill, windmill) or
    • nosetests --classattr="testtype=unittest" where you can replace unittest by twill or windmill to run only those tests in that layer (you can have multiple --classattr arguments if you want to run multiple layers)
    • If you are running only unittests you can also add --processes=4 to get a nice speedup by executing the tests in parallel (probably doesn't work for twill and windmill tests though as they share environments across tests. This should give about a 2x speedup to the unittests.
  • If you want to execute all tests like the buildserver does, you can execute python setup.py test (replace with setup_pro.py for agilo pro). (This also executes windmill, which are blocked on the build server by the SKIP_WINDMILL environment variable.
  • Many acceptance tests also have a if __file__ == __main__: blurb at the bottom and can be executed directly with python.
  • If you want to set a breakpoint in a test, the recommended way is with from nose.tools import set_trace; set_trace() as that works even if nose is started with output buffering. (You wouldn't see the debugger prompt otherwise)

Javascript:

  • There are two testrunners, one for open and one for pro, both testsuites execute and auto-reload as soon as you open their testriver html page. To get to them, append chrome/agilo/tests/testrunner.html and chrome/agilo_pro/js/tests/testrunner.html to your environment url. (Then either bookmark those or put them on your wiki-start page)

custom nose plugins

  • We have custom nose plugins (agilo.test.nose_testselector) - these are automatically found because they are declared in our setup.py.
Last modified 7 years ago Last modified on 06/21/2010 11:40:07 AM

1.3.15 © 2008-2016 Agilo Software all rights reserved (this page was served in: 0.14815 sec.)