July 28, 2017 - Shuah Khan
Kselftest for Linux 4.13 to Include TAP13
TAP13 defines a human friendly output format for tests. Kselftest is run in test rings and is widely used for Linux kernel stable release regression testing. It’s important to make it easier to identify run-to-run differences; TAP13 adaption makes it easier to understand the test results, and helps pin point differences between one run to another run of the test suite.
Credit goes to Tim Bird for recommending TAP13 as a suitable format, and to Greg KH for kick starting the work with help from Paul Elder and Alice Ferrazzi.
The first phase of the TAP13 conversion is included in Linux 4.13. Future releases will include updates to rest of the tests. The following shows membarrier test results before and after TAP 13 conversion:
membarrier MEMBARRIER_CMD_QUERY syscall available.
membarrier: MEMBARRIER_CMD_SHARED success.
membarrier: tests done!
TAP version 13
ok 1 sys_membarrier available
ok 2 sys membarrier invalid command test: command = -1, flags = 0. Failed as expected
ok 3 sys membarrier MEMBARRIER_CMD_QUERY invalid flags test: flags = 1. Failed as expected
ok 4 sys membarrier MEMBARRIER_CMD_SHARED test: flags = 0
Each test result is preceded with “ok,” “not ok,” or “skip” followed by the test number to clearly indicate the result of a test.
The Kselftest API has been enhanced to add new interfaces, and existing interfaces have been updated to print output in TAP13 format to abstract TAP13 details from the individual tests.
static inline void ksft_print_header(void);
static inline void ksft_print_msg(const char *msg, ...);
static inline void ksft_test_result_pass(const char *msg, ...);
static inline void ksft_test_result_fail(const char *msg, ...);
static inline void ksft_test_result_skip(const char *msg, ...);
static inline void ksft_print_cnts(void);
static inline int ksft_exit_pass(void);
static inline int ksft_exit_fail(void);
static inline int ksft_exit_xfail(void);
static inline int ksft_exit_xpass(void);
static inline int ksft_exit_skip(void);
With the framework in place, work is now underway to convert existing tests to TAP13 and ensure that new tests conform to the format.
About Shuah Khan
Shuah Khan is a Senior Linux Kernel Developer at Samsung's Open Source Group. She is a Linux Kernel Maintainer and Contributor who focuses on Linux Media Core and Power Management. She maintains Kernel Selftest framework. She has contributed to IOMMU, and DMA areas. In addition, she is helping with stable release kernel testing. She authored Linux Kernel Testing and Debugging paper published on the Linux Journal and writes Linux Journal kernel news articles. She has presented at several Linux conferences and Linux Kernel Developer Keynote Panels. She served on the Linux Foundation Technical Advisory Board. Prior to joining Samsung, she worked as a kernel and software developer at HP and Lucent.
Image Credits: OSDC