Smoke tests

Introduction to smoke tests

Smoke tests are preliminary tests to reveal simple failures severe enough to, for example, reject an upcoming software release. In the Software Factory, you can use smoke tests to ensure a basic quality level of SQL queries in your application. They will discover the following problems in advance instead of at runtime:

• Malformed queries - SQL queries that are not placed on the database as application logic may be malformed.
• Editable views without RDBMS editing support, instead-of triggers, or handlers.
• Outdated parameterization of logic.
• Some forms of outdated dependencies for views, triggers, and stored procedure logic on the database.

Items tested by smoke tests

note

Procedure subroutines, handlers, and tasks are not included in smoke tests, since knowledge of the parameterization is required to execute these without causing errors. You can use unit tests to achieve the desired coverage.

The following items are tested by smoke tests:

• Via a select query:
  • SQL prefilters
  • Views
  • Calculated columns
  • Calculated columns (function)
  • Expression columns
  • Column default expressions
  • Deviating default expressions at a variant level
  • Task parameter default expressions
  • Report parameter default expressions
  • Process variable default expressions
  • Scalar functions
  • Table-valued functions
  • Domain input constraints (sql-typed).
• Within a transaction that is rolled back, using null values as input:
  • Default logic
  • Layout logic
  • Context logic
  • Badge logic
  • Change detection logic
• Within a transaction that is rolled back, firing with 0 affected rows:
  • Insert triggers
  • Instead-of insert triggers
  • Update triggers
  • Instead-of update triggers
  • Delete triggers
  • Instead-of delete triggers.

Run the smoke tests

Before running a smoke test, note that:

• Smoke tests only support SQL server as a database platform.
• Smoke tests can be run as a single unit. You can also re-run a single test step. Note that this is only possible for existing steps. It may cause a step to disappear. Running a full smoke test on a regular basis is required to create new steps for newly introduced testable scenarios.
• Each branch can have only one set of smoke test results. Rerunning a smoke test will replace the previous results.
• If a test has not finished after 10 seconds, it will be aborted. This can happen, for example, for complex calculations. In that case, it generally means the code is compiled correctly.

To run the available smoke tests:

menu Quality > Smoke tests

You can either execute all smoke tests as a single unit or re-run selected smoke test step(s).

To execute all smoke tests as a single unit:

1. Execute the Start smoke test task.
2. If necessary, change the Runtime configuration.
3. Click Execute.

To re-run selected smoke test step(s):

1. Select the smoke test step(s) that you want to run again.
2. Execute the Re-run selected smoke test step(s) task.
3. Click Execute.

The results are displayed in tab List. You can view the following information about the smoke test:

• The full duration of each smoke test step is stored and displayed. Note that a prefilter or expression test may be affected by the performance of an underlying view or function.
• If any of the steps in a smoke test have failed, this will be shown in the Error message column. A warning due to a timed-out step is not considered a failure.

Prefilters are available to show only the unit tests that need attention:

• Use the prefilter Warning and failed (ALT+W) to show smoke tests with a warning, or that have failed.
• Use the prefilter Failed (ALT+F), to only show smoke tests that have failed.

The details for all tests are available in tab Form. Here, you can consult failed or skipped steps, and a test's SQL code for manual execution.

Results of the smoke tests