How do I turn the TIAF on/off in Automated Review?

Method 1: Using the TIAF CMake Switches

This method will switch off TIAF at the CMake option level, leaving all tests currently opted-in to TIAF to be run by CTest. This method is the least intrusive and thus only if this method fails should the following method be used.

When using this method, the TIAF CMake function o3de_test_impact_apply_test_labels will remove the label REQUIRES_tiaf from any native or Python test target that has been disabled with the O3DE_TEST_IMPACT_NATIVE_TEST_TARGETS_ENABLED or O3DE_TEST_IMPACT_PYTHON_TEST_TARGETS_ENABLED CMake variables. As such, it requires no further modification at either the CMake or Jenkins level to gracefully pass the burden of test running back over to CTest.

1. Navigate to scripts\build\Platform\Windows\build_config.json.
2. Locate test_cpu_profile under CMAKE_OPTIONS set O3DE_TEST_IMPACT_NATIVE_TEST_TARGETS_ENABLED (for native C++ tests) and/or O3DE_TEST_IMPACT_PYTHON_TEST_TARGETS_ENABLED (for Python tests) to FALSE.

Method 2: Removing the TIAF Stages from AR

This method will remove TIAF completely from AR. It should be used as a method of last resort as it requires intrusive edits to the AR Jenkins configuration.

1. Navigate to scripts\build\Platform\Windows\build_config.json.
2. Locate profile_pipe and under steps remove test_impact_analysis_profile_native and test_impact_analysis_profile_python.
3. Locate test_cpu_profile and under CTEST_OPTIONS change (REQUIRES_gpu|REQUIRES_tiaf) to (REQUIRES_gpu).

How do I enroll a test into TIAF?

All tests enrolled into the TIAF are run by the appropriate TIAF runtime (native for native C++ tests, python for Python tests) instead of CTest. Likewise, all tests not enrolled into the TIAF are run instead by CTest. In order to enroll a test target into TIAF, simply add the REQUIRES_tiaf label when registering the test, like so:

ly_add_googletest(
    NAME AZ::MyExample.Tests
    LABELS REQUIRES_tiaf
)

Caution:

It is advised that you place your LABELS after the test target name and before any other custom attributes.

How do I enrol a native test for test sharding optimization?

Test sharding is an optional optimization that can greatly boost the speed at which test targets are run (see NativeInstrumentedTestRunner for more information). As the question implies, the test sharding optimization is only available for native tests. In order to opt a test target into test sharding, simply add either TIAF_shard_test or TIAF_shard_fixture label when registering the test, like so:

ly_add_googletest(
    NAME AZ::MyExample.Tests
    LABELS REQUIRES_tiaf;TIAF_shard_test
)

When using the TIAF_shard_test label, each individual test will be interleaved across the available shards. This delivers the greatest performance boost but increases the brittleness of the sharded test target as resource race conditions for badly behaving tests are more likely to manifest. On the contrary, TIAF_shard_fixture interleaves each individual fixture across the available shards, delivering less performance than TIAF_shard_test but less brittleness. Always test your targets when enrolling them into test sharding optimization to ensure that they behave well when running as shards.

Note:

The test target must be enrolled into native TIAF in order to benefit from test sharding, else the test target will not be run by TIAF.

What are the command line options for the TIAF runtimes?

The following table details the command line options common to both the native and Python runtimes:

The native runtime has the following additional options:

What are the command line options for the TIAF AR scripts?

The following table details the command line options common to both the native and Python AR scripts: