What is Smoke Testing?

Smoke testing takes its name from electrical engineering. After working on a device, technicians would turn it on and see if it starts to smoke. The meaning in software development and other professions is the same: A smoke test is a quick test to make sure your product doesn’t immediately burst into flames.

It is not to be confused with more detailed testing concepts. A smoke test is not meant to tell you if every detail is working as expected. It will only tell you that your system is not so badly broken that more fine-grained tests make no sense.

Typically testing will be done by the developers before they check in their changes. Alternatively/additionally, testers will start their test run with a smoke test to make sure that the product is not so fundamentally broken that a detailed test session would be a waste of ressources.

Smoke Tests Need to be Cheap and Fast

When designing test cases, put the focus on quick execution. These tests will be run over and over again. They are meant as a basic safety measure, not an extensive quality assurance process.

One should not have to judge whether it is worth to execute a smoke test or not - running the test should be a no-brainer. If it takes more than 30 minutes to manually execute the test suite, you are doing it wrong.

Example Test Cases

As an example, here are a few test cases of our smoke test suite for Checkpanel:

Note how they only cover the basic functionality. For each test case, you could cover a lot more details. For example, when logging in you could also test how the login form renders optically, how it handles wrong credentials, how it handles malformed input, etc. But this would not be the point of a smoke test.

Automating Smoke Tests

Smoke tests can be either executed manually or be automated. The most important thing is that they are done.

That being said, smoke tests are very well suited for automation. They are repeated extremely often so the payback for setting up automation is huge. They are also extremely basic so usually they can be implemented fairly quickly.

Even if you do not have a full blown test automation process in place, see if a tool like Selenium IDE can be used for your test cases. For simple test cases, such a record-and-replay tool is often enough.

Pitfalls

When designing your test suite, look out for some common pitfalls:

• Testing too broad
  Don’t try to test every possible function of your system. Broad tests are important, but smoke testing is not the place for it.
• Testing too detailed
  When testing your main features, only verify basic functionality. E.g. do not test for correct line spacing or margins.
• Confusing the concept with regression testing
  Regression testing, while important, is way more detailed.
• Confusing the concept with sanity testing
  Sanity testing is very similar but focuses on a new functionality which is being built.

Tools

The following tools will help you making your smoke testing efforts more efficient:

• Checkpanel (obviously)
  A great test management tool like Checkpanel will help keeing your test cases organized. It will protocol the results of each test run. You can set up reminders so that you won’t forget your smoke tests ever again.
• Selenium IDE
  Selenium IDE lets you automate your browser with a simple interface. Once set up, the browser can run your tests for you with the click of a single button.
• Xenu’s Link Sleuth
  A simple tool which will automatically find all broken links on your site.