Testing: The Developer Strikes Back

Presented by Sandy

1. Testing is hard to do right.

   • Delegate test responsibilities correctly.
   • Won’t (always) get it right on the first try.

2. Unit test organization strategies for Django projects

   What is “Unit Testing”? A method by which individual units of source code are tested.

   • Each app needs it’s own test module.

     • Each module should have (many) submodules.

       • Each contains classes that test units of code.

   • Organize your test suite however you want

     • Just be consistent.

3. Dealing with increasingly complex test scenarios

   • Separate code and service/infrastructure testing
   • Don’t keep all the tests in one tests.py
   • Use sub-module and root tests.py files sparingly.
   • Give Mock (http://python-mok.sourceforge.net/) a try.

4. Beyond the business logic

   • Testing 3rd party APIs or cache

   • These should be tested separately

     • Better speed of tests if you’re not constantly making outbound calls.

   • Dealing with cache:

     • Dev env doesn’t usually have long-term or complete sets of cached data.
     • Cache should be agnostic to model changes.
     • Do object structure comparison when pulling from cache.
     • Testing how your app deals with ‘old’ data, not the infrastructure.

5. Writing tests can improve coding habits

   • Smaller, modular code is testable, 200-line functions aren’t.

   • Write more tests

     • Discover easily-testable coding patterns.
     • Functions should perform a single function.

6. Tests are alive

7. How do I enforce testing on my team?

   • Most people don’t like 2AM fires.
   • Iterate faster with confidence.
   • Git pre-commit hooks, stop checkins without tests.
   • Coverage.py can make it a game.
   • Public shaming?

8. How much testing is enough testing?

   • No simple answer for that.

9. Junction between Unit and Integration

   • Difficult areas to test, because behavior is dependant upon environment.

10. Testing a virgin codebase

    • You will refactor code as you write tests

11. Successful strategies

    • Require unit tests for all code going forward

    • Reserve time to clear out test debt

      • Good for new team members
      • Everyone takes a turn

    • Establish a good foundation to build on

    • Every bug is two bugs:

      • One in the code that broke
      • One is the test that would/will catch the bug

12. Test data

    • Fixtures, mocks, ObjectCreator
    • Use SQLite

13. Should I get test data from cache?

    • No. Adds unnecessary complexity to unit tests.
    • If tests only succeed with certain caches, probably some code that can be refactored.

14. Graceful code degredation

    • Devs need to think outside their dev instance
    • Service unavailable shouldn’t mean your site is unavailable
    • If failure is catastrophic, you should know how and why it happened.

15. Forcing awareness

    • Code should not be written in such a way that it won’t work (at least in some fashion) if certain services or infrastructers are missing/different.

16. Test infrastructe

    • Staging should be identical to production.
    • Run Solr or RabbitMQ on staging, but not dev.
    • Don’t overdo it on logging.

17. Testing tools

    • Nose
    • Coverage
    • Mock

Code not tested is broken by design