diff options
Diffstat (limited to 'Documentation/dev-tools/kunit/usage.rst')
| -rw-r--r-- | Documentation/dev-tools/kunit/usage.rst | 1155 |
1 files changed, 888 insertions, 267 deletions
diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst index 607758a66a99..ebd06f5ea455 100644 --- a/Documentation/dev-tools/kunit/usage.rst +++ b/Documentation/dev-tools/kunit/usage.rst @@ -1,57 +1,13 @@ .. SPDX-License-Identifier: GPL-2.0 -=========== -Using KUnit -=========== - -The purpose of this document is to describe what KUnit is, how it works, how it -is intended to be used, and all the concepts and terminology that are needed to -understand it. This guide assumes a working knowledge of the Linux kernel and -some basic knowledge of testing. - -For a high level introduction to KUnit, including setting up KUnit for your -project, see :doc:`start`. - -Organization of this document -============================= - -This document is organized into two main sections: Testing and Isolating -Behavior. The first covers what unit tests are and how to use KUnit to write -them. The second covers how to use KUnit to isolate code and make it possible -to unit test code that was otherwise un-unit-testable. - -Testing -======= - -What is KUnit? --------------- - -"K" is short for "kernel" so "KUnit" is the "(Linux) Kernel Unit Testing -Framework." KUnit is intended first and foremost for writing unit tests; it is -general enough that it can be used to write integration tests; however, this is -a secondary goal. KUnit has no ambition of being the only testing framework for -the kernel; for example, it does not intend to be an end-to-end testing -framework. - -What is Unit Testing? ---------------------- - -A `unit test <https://martinfowler.com/bliki/UnitTest.html>`_ is a test that -tests code at the smallest possible scope, a *unit* of code. In the C -programming language that's a function. - -Unit tests should be written for all the publicly exposed functions in a -compilation unit; so that is all the functions that are exported in either a -*class* (defined below) or all functions which are **not** static. - Writing Tests -------------- +============= Test Cases -~~~~~~~~~~ +---------- The fundamental unit in KUnit is the test case. A test case is a function with -the signature ``void (*)(struct kunit *test)``. It calls a function to be tested +the signature ``void (*)(struct kunit *test)``. It calls the function under test and then sets *expectations* for what should happen. For example: .. code-block:: c @@ -65,18 +21,19 @@ and then sets *expectations* for what should happen. For example: KUNIT_FAIL(test, "This test never passes."); } -In the above example ``example_test_success`` always passes because it does -nothing; no expectations are set, so all expectations pass. On the other hand -``example_test_failure`` always fails because it calls ``KUNIT_FAIL``, which is -a special expectation that logs a message and causes the test case to fail. +In the above example, ``example_test_success`` always passes because it does +nothing; no expectations are set, and therefore all expectations pass. On the +other hand ``example_test_failure`` always fails because it calls ``KUNIT_FAIL``, +which is a special expectation that logs a message and causes the test case to +fail. Expectations ~~~~~~~~~~~~ -An *expectation* is a way to specify that you expect a piece of code to do -something in a test. An expectation is called like a function. A test is made -by setting expectations about the behavior of a piece of code under test; when -one or more of the expectations fail, the test case fails and information about -the failure is logged. For example: +An *expectation* specifies that we expect a piece of code to do something in a +test. An expectation is called like a function. A test is made by setting +expectations about the behavior of a piece of code under test. When one or more +expectations fail, the test case fails and information about the failure is +logged. For example: .. code-block:: c @@ -86,28 +43,28 @@ the failure is logged. For example: KUNIT_EXPECT_EQ(test, 2, add(1, 1)); } -In the above example ``add_test_basic`` makes a number of assertions about the -behavior of a function called ``add``; the first parameter is always of type -``struct kunit *``, which contains information about the current test context; -the second parameter, in this case, is what the value is expected to be; the +In the above example, ``add_test_basic`` makes a number of assertions about the +behavior of a function called ``add``. The first parameter is always of type +``struct kunit *``, which contains information about the current test context. +The second parameter, in this case, is what the value is expected to be. The last value is what the value actually is. If ``add`` passes all of these expectations, the test case, ``add_test_basic`` will pass; if any one of these -expectations fail, the test case will fail. +expectations fails, the test case will fail. -It is important to understand that a test case *fails* when any expectation is -violated; however, the test will continue running, potentially trying other -expectations until the test case ends or is otherwise terminated. This is as -opposed to *assertions* which are discussed later. +A test case *fails* when any expectation is violated; however, the test will +continue to run, and try other expectations until the test case ends or is +otherwise terminated. This is as opposed to *assertions* which are discussed +later. -To learn about more expectations supported by KUnit, see :doc:`api/test`. +To learn about more KUnit expectations, see Documentation/dev-tools/kunit/api/test.rst. .. note:: - A single test case should be pretty short, pretty easy to understand, - focused on a single behavior. + A single test case should be short, easy to understand, and focused on a + single behavior. -For example, if we wanted to properly test the add function above, we would -create additional tests cases which would each test a different property that an -add function should have like this: +For example, if we want to rigorously test the ``add`` function above, create +additional tests cases which would test each property that an ``add`` function +should have as shown below: .. code-block:: c @@ -133,56 +90,88 @@ add function should have like this: KUNIT_EXPECT_EQ(test, INT_MIN, add(INT_MAX, 1)); } -Notice how it is immediately obvious what all the properties that we are testing -for are. - Assertions ~~~~~~~~~~ -KUnit also has the concept of an *assertion*. An assertion is just like an -expectation except the assertion immediately terminates the test case if it is -not satisfied. - -For example: +An assertion is like an expectation, except that the assertion immediately +terminates the test case if the condition is not satisfied. For example: .. code-block:: c - static void mock_test_do_expect_default_return(struct kunit *test) + static void test_sort(struct kunit *test) { - struct mock_test_context *ctx = test->priv; - struct mock *mock = ctx->mock; - int param0 = 5, param1 = -5; - const char *two_param_types[] = {"int", "int"}; - const void *two_params[] = {¶m0, ¶m1}; - const void *ret; - - ret = mock->do_expect(mock, - "test_printk", test_printk, - two_param_types, two_params, - ARRAY_SIZE(two_params)); - KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ret); - KUNIT_EXPECT_EQ(test, -4, *((int *) ret)); + int *a, i, r = 1; + a = kunit_kmalloc_array(test, TEST_LEN, sizeof(*a), GFP_KERNEL); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, a); + for (i = 0; i < TEST_LEN; i++) { + r = (r * 725861) % 6599; + a[i] = r; + } + sort(a, TEST_LEN, sizeof(*a), cmpint, NULL); + for (i = 0; i < TEST_LEN-1; i++) + KUNIT_EXPECT_LE(test, a[i], a[i + 1]); } -In this example, the method under test should return a pointer to a value, so -if the pointer returned by the method is null or an errno, we don't want to -bother continuing the test since the following expectation could crash the test -case. `ASSERT_NOT_ERR_OR_NULL(...)` allows us to bail out of the test case if -the appropriate conditions have not been satisfied to complete the test. +In this example, we need to be able to allocate an array to test the ``sort()`` +function. So we use ``KUNIT_ASSERT_NOT_ERR_OR_NULL()`` to abort the test if +there's an allocation error. + +.. note:: + In other test frameworks, ``ASSERT`` macros are often implemented by calling + ``return`` so they only work from the test function. In KUnit, we stop the + current kthread on failure, so you can call them from anywhere. + +.. note:: + Warning: There is an exception to the above rule. You shouldn't use assertions + in the suite's exit() function, or in the free function for a resource. These + run when a test is shutting down, and an assertion here prevents further + cleanup code from running, potentially leading to a memory leak. + +Customizing error messages +-------------------------- + +Each of the ``KUNIT_EXPECT`` and ``KUNIT_ASSERT`` macros have a ``_MSG`` +variant. These take a format string and arguments to provide additional +context to the automatically generated error messages. + +.. code-block:: c + + char some_str[41]; + generate_sha1_hex_string(some_str); + + /* Before. Not easy to tell why the test failed. */ + KUNIT_EXPECT_EQ(test, strlen(some_str), 40); + + /* After. Now we see the offending string. */ + KUNIT_EXPECT_EQ_MSG(test, strlen(some_str), 40, "some_str='%s'", some_str); + +Alternatively, one can take full control over the error message by using +``KUNIT_FAIL()``, e.g. + +.. code-block:: c + + /* Before */ + KUNIT_EXPECT_EQ(test, some_setup_function(), 0); + + /* After: full control over the failure message. */ + if (some_setup_function()) + KUNIT_FAIL(test, "Failed to setup thing for testing"); + Test Suites ~~~~~~~~~~~ -Now obviously one unit test isn't very helpful; the power comes from having -many test cases covering all of a unit's behaviors. Consequently it is common -to have many *similar* tests; in order to reduce duplication in these closely -related tests most unit testing frameworks - including KUnit - provide the -concept of a *test suite*. A *test suite* is just a collection of test cases -for a unit of code with a set up function that gets invoked before every test -case and then a tear down function that gets invoked after every test case -completes. +We need many test cases covering all the unit's behaviors. It is common to have +many similar tests. In order to reduce duplication in these closely related +tests, most unit testing frameworks (including KUnit) provide the concept of a +*test suite*. A test suite is a collection of test cases for a unit of code +with optional setup and teardown functions that run before/after the whole +suite and/or every test case. + +.. note:: + A test case will only run if it is associated with a test suite. -Example: +For example: .. code-block:: c @@ -197,62 +186,100 @@ Example: .name = "example", .init = example_test_init, .exit = example_test_exit, + .suite_init = example_suite_init, + .suite_exit = example_suite_exit, .test_cases = example_test_cases, }; kunit_test_suite(example_test_suite); -In the above example the test suite, ``example_test_suite``, would run the test -cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``, -each would have ``example_test_init`` called immediately before it and would -have ``example_test_exit`` called immediately after it. -``kunit_test_suite(example_test_suite)`` registers the test suite with the -KUnit test framework. +In the above example, the test suite ``example_test_suite`` would first run +``example_suite_init``, then run the test cases ``example_test_foo``, +``example_test_bar``, and ``example_test_baz``. Each would have +``example_test_init`` called immediately before it and ``example_test_exit`` +called immediately after it. Finally, ``example_suite_exit`` would be called +after everything else. ``kunit_test_suite(example_test_suite)`` registers the +test suite with the KUnit test framework. .. note:: - A test case will only be run if it is associated with a test suite. + The ``exit`` and ``suite_exit`` functions will run even if ``init`` or + ``suite_init`` fail. Make sure that they can handle any inconsistent + state which may result from ``init`` or ``suite_init`` encountering errors + or exiting early. + +``kunit_test_suite(...)`` is a macro which tells the linker to put the +specified test suite in a special linker section so that it can be run by KUnit +either after ``late_init``, or when the test module is loaded (if the test was +built as a module). + +For more information, see Documentation/dev-tools/kunit/api/test.rst. + +.. _kunit-on-non-uml: + +Writing Tests For Other Architectures +------------------------------------- + +It is better to write tests that run on UML to tests that only run under a +particular architecture. It is better to write tests that run under QEMU or +another easy to obtain (and monetarily free) software environment to a specific +piece of hardware. + +Nevertheless, there are still valid reasons to write a test that is architecture +or hardware specific. For example, we might want to test code that really +belongs in ``arch/some-arch/*``. Even so, try to write the test so that it does +not depend on physical hardware. Some of our test cases may not need hardware, +only few tests actually require the hardware to test it. When hardware is not +available, instead of disabling tests, we can skip them. + +Now that we have narrowed down exactly what bits are hardware specific, the +actual procedure for writing and running the tests is same as writing normal +KUnit tests. + +.. important:: + We may have to reset hardware state. If this is not possible, we may only + be able to run one test case per invocation. + +.. TODO(brendanhiggins@google.com): Add an actual example of an architecture- + dependent KUnit test. -For more information on these types of things see the :doc:`api/test`. +Common Patterns +=============== Isolating Behavior -================== - -The most important aspect of unit testing that other forms of testing do not -provide is the ability to limit the amount of code under test to a single unit. -In practice, this is only possible by being able to control what code gets run -when the unit under test calls a function and this is usually accomplished -through some sort of indirection where a function is exposed as part of an API -such that the definition of that function can be changed without affecting the -rest of the code base. In the kernel this primarily comes from two constructs, -classes, structs that contain function pointers that are provided by the -implementer, and architecture specific functions which have definitions selected -at compile time. +------------------ + +Unit testing limits the amount of code under test to a single unit. It controls +what code gets run when the unit under test calls a function. Where a function +is exposed as part of an API such that the definition of that function can be +changed without affecting the rest of the code base. In the kernel, this comes +from two constructs: classes, which are structs that contain function pointers +provided by the implementer, and architecture-specific functions, which have +definitions selected at compile time. Classes -------- +~~~~~~~ Classes are not a construct that is built into the C programming language; -however, it is an easily derived concept. Accordingly, pretty much every project -that does not use a standardized object oriented library (like GNOME's GObject) -has their own slightly different way of doing object oriented programming; the -Linux kernel is no exception. +however, it is an easily derived concept. Accordingly, in most cases, every +project that does not use a standardized object oriented library (like GNOME's +GObject) has their own slightly different way of doing object oriented +programming; the Linux kernel is no exception. The central concept in kernel object oriented programming is the class. In the kernel, a *class* is a struct that contains function pointers. This creates a contract between *implementers* and *users* since it forces them to use the -same function signature without having to call the function directly. In order -for it to truly be a class, the function pointers must specify that a pointer -to the class, known as a *class handle*, be one of the parameters; this makes -it possible for the member functions (also known as *methods*) to have access -to member variables (more commonly known as *fields*) allowing the same -implementation to have multiple *instances*. - -Typically a class can be *overridden* by *child classes* by embedding the -*parent class* in the child class. Then when a method provided by the child -class is called, the child implementation knows that the pointer passed to it is -of a parent contained within the child; because of this, the child can compute -the pointer to itself because the pointer to the parent is always a fixed offset -from the pointer to the child; this offset is the offset of the parent contained -in the child struct. For example: +same function signature without having to call the function directly. To be a +class, the function pointers must specify that a pointer to the class, known as +a *class handle*, be one of the parameters. Thus the member functions (also +known as *methods*) have access to member variables (also known as *fields*) +allowing the same implementation to have multiple *instances*. + +A class can be *overridden* by *child classes* by embedding the *parent class* +in the child class. Then when the child class *method* is called, the child +implementation knows that the pointer passed to it is of a parent contained +within the child. Thus, the child can compute the pointer to itself because the +pointer to the parent is always a fixed offset from the pointer to the child. +This offset is the offset of the parent contained in the child struct. For +example: .. code-block:: c @@ -268,7 +295,7 @@ in the child struct. For example: int rectangle_area(struct shape *this) { - struct rectangle *self = container_of(this, struct shape, parent); + struct rectangle *self = container_of(this, struct rectangle, parent); return self->length * self->width; }; @@ -280,8 +307,8 @@ in the child struct. For example: self->width = width; } -In this example (as in most kernel code) the operation of computing the pointer -to the child from the pointer to the parent is done by ``container_of``. +In this example, computing the pointer to the child from the pointer to the +parent is done by ``container_of``. Faking Classes ~~~~~~~~~~~~~~ @@ -290,14 +317,11 @@ In order to unit test a piece of code that calls a method in a class, the behavior of the method must be controllable, otherwise the test ceases to be a unit test and becomes an integration test. -A fake just provides an implementation of a piece of code that is different than -what runs in a production instance, but behaves identically from the standpoint -of the callers; this is usually done to replace a dependency that is hard to -deal with, or is slow. - -A good example for this might be implementing a fake EEPROM that just stores the -"contents" in an internal buffer. For example, let's assume we have a class that -represents an EEPROM: +A fake class implements a piece of code that is different than what runs in a +production instance, but behaves identical from the standpoint of the callers. +This is done to replace a dependency that is hard to deal with, or is slow. For +example, implementing a fake EEPROM that stores the "contents" in an +internal buffer. Assume we have a class that represents an EEPROM: .. code-block:: c @@ -306,7 +330,7 @@ represents an EEPROM: ssize_t (*write)(struct eeprom *this, size_t offset, const char *buffer, size_t count); }; -And we want to test some code that buffers writes to the EEPROM: +And we want to test code that buffers writes to the EEPROM: .. code-block:: c @@ -319,7 +343,7 @@ And we want to test some code that buffers writes to the EEPROM: struct eeprom_buffer *new_eeprom_buffer(struct eeprom *eeprom); void destroy_eeprom_buffer(struct eeprom *eeprom); -We can easily test this code by *faking out* the underlying EEPROM: +We can test this code by *faking out* the underlying EEPROM: .. code-block:: c @@ -446,148 +470,745 @@ We can now use it to test ``struct eeprom_buffer``: destroy_eeprom_buffer(ctx->eeprom_buffer); } -.. _kunit-on-non-uml: +Testing Against Multiple Inputs +------------------------------- -KUnit on non-UML architectures -============================== +Testing just a few inputs is not enough to ensure that the code works correctly, +for example: testing a hash function. -By default KUnit uses UML as a way to provide dependencies for code under test. -Under most circumstances KUnit's usage of UML should be treated as an -implementation detail of how KUnit works under the hood. Nevertheless, there -are instances where being able to run architecture specific code or test -against real hardware is desirable. For these reasons KUnit supports running on -other architectures. +We can write a helper macro or function. The function is called for each input. +For example, to test ``sha1sum(1)``, we can write: -Running existing KUnit tests on non-UML architectures ------------------------------------------------------ +.. code-block:: c -There are some special considerations when running existing KUnit tests on -non-UML architectures: + #define TEST_SHA1(in, want) \ + sha1sum(in, out); \ + KUNIT_EXPECT_STREQ_MSG(test, out, want, "sha1sum(%s)", in); -* Hardware may not be deterministic, so a test that always passes or fails - when run under UML may not always do so on real hardware. -* Hardware and VM environments may not be hermetic. KUnit tries its best to - provide a hermetic environment to run tests; however, it cannot manage state - that it doesn't know about outside of the kernel. Consequently, tests that - may be hermetic on UML may not be hermetic on other architectures. -* Some features and tooling may not be supported outside of UML. -* Hardware and VMs are slower than UML. + char out[40]; + TEST_SHA1("hello world", "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed"); + TEST_SHA1("hello world!", "430ce34d020724ed75a196dfc2ad67c77772d169"); -None of these are reasons not to run your KUnit tests on real hardware; they are -only things to be aware of when doing so. +Note the use of the ``_MSG`` version of ``KUNIT_EXPECT_STREQ`` to print a more +detailed error and make the assertions clearer within the helper macros. -The biggest impediment will likely be that certain KUnit features and -infrastructure may not support your target environment. For example, at this -time the KUnit Wrapper (``tools/testing/kunit/kunit.py``) does not work outside -of UML. Unfortunately, there is no way around this. Using UML (or even just a -particular architecture) allows us to make a lot of assumptions that make it -possible to do things which might otherwise be impossible. +The ``_MSG`` variants are useful when the same expectation is called multiple +times (in a loop or helper function) and thus the line number is not enough to +identify what failed, as shown below. -Nevertheless, all core KUnit framework features are fully supported on all -architectures, and using them is straightforward: all you need to do is to take -your kunitconfig, your Kconfig options for the tests you would like to run, and -merge them into whatever config your are using for your platform. That's it! +In complicated cases, we recommend using a *table-driven test* compared to the +helper macro variation, for example: -For example, let's say you have the following kunitconfig: +.. code-block:: c -.. code-block:: none + int i; + char out[40]; - CONFIG_KUNIT=y - CONFIG_KUNIT_EXAMPLE_TEST=y + struct sha1_test_case { + const char *str; + const char *sha1; + }; -If you wanted to run this test on an x86 VM, you might add the following config -options to your ``.config``: + struct sha1_test_case cases[] = { + { + .str = "hello world", + .sha1 = "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed", + }, + { + .str = "hello world!", + .sha1 = "430ce34d020724ed75a196dfc2ad67c77772d169", + }, + }; + for (i = 0; i < ARRAY_SIZE(cases); ++i) { + sha1sum(cases[i].str, out); + KUNIT_EXPECT_STREQ_MSG(test, out, cases[i].sha1, + "sha1sum(%s)", cases[i].str); + } -.. code-block:: none - CONFIG_KUNIT=y - CONFIG_KUNIT_EXAMPLE_TEST=y - CONFIG_SERIAL_8250=y - CONFIG_SERIAL_8250_CONSOLE=y +There is more boilerplate code involved, but it can: -All these new options do is enable support for a common serial console needed -for logging. +* be more readable when there are multiple inputs/outputs (due to field names). -Next, you could build a kernel with these tests as follows: + * For example, see ``fs/ext4/inode-test.c``. +* reduce duplication if test cases are shared across multiple tests. -.. code-block:: bash + * For example: if we want to test ``sha256sum``, we could add a ``sha256`` + field and reuse ``cases``. - make ARCH=x86 olddefconfig - make ARCH=x86 +* be converted to a "parameterized test". -Once you have built a kernel, you could run it on QEMU as follows: +Parameterized Testing +~~~~~~~~~~~~~~~~~~~~~ -.. code-block:: bash +To run a test case against multiple inputs, KUnit provides a parameterized +testing framework. This feature formalizes and extends the concept of +table-driven tests discussed previously. - qemu-system-x86_64 -enable-kvm \ - -m 1024 \ - -kernel arch/x86_64/boot/bzImage \ - -append 'console=ttyS0' \ - --nographic +A KUnit test is determined to be parameterized if a parameter generator function +is provided when registering the test case. A test user can either write their +own generator function or use one that is provided by KUnit. The generator +function is stored in ``kunit_case->generate_params`` and can be set using the +macros described in the section below. -Interspersed in the kernel logs you might see the following: +To establish the terminology, a "parameterized test" is a test which is run +multiple times (once per "parameter" or "parameter run"). Each parameter run has +both its own independent ``struct kunit`` (the "parameter run context") and +access to a shared parent ``struct kunit`` (the "parameterized test context"). -.. code-block:: none +Passing Parameters to a Test +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +There are three ways to provide the parameters to a test: - TAP version 14 - # Subtest: example - 1..1 - # example_simple_test: initializing - ok 1 - example_simple_test - ok 1 - example +Array Parameter Macros: -Congratulations, you just ran a KUnit test on the x86 architecture! + KUnit provides special support for the common table-driven testing pattern. + By applying either ``KUNIT_ARRAY_PARAM`` or ``KUNIT_ARRAY_PARAM_DESC`` to the + ``cases`` array from the previous section, we can create a parameterized test + as shown below: -In a similar manner, kunit and kunit tests can also be built as modules, -so if you wanted to run tests in this way you might add the following config -options to your ``.config``: +.. code-block:: c -.. code-block:: none + // This is copy-pasted from above. + struct sha1_test_case { + const char *str; + const char *sha1; + }; + static const struct sha1_test_case cases[] = { + { + .str = "hello world", + .sha1 = "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed", + }, + { + .str = "hello world!", + .sha1 = "430ce34d020724ed75a196dfc2ad67c77772d169", + }, + }; - CONFIG_KUNIT=m - CONFIG_KUNIT_EXAMPLE_TEST=m + // Creates `sha1_gen_params()` to iterate over `cases` while using + // the struct member `str` for the case description. + KUNIT_ARRAY_PARAM_DESC(sha1, cases, str); -Once the kernel is built and installed, a simple + // Looks no different from a normal test. + static void sha1_test(struct kunit *test) + { + // This function can just contain the body of the for-loop. + // The former `cases[i]` is accessible under test->param_value. + char out[40]; + struct sha1_test_case *test_param = (struct sha1_test_case *)(test->param_value); + + sha1sum(test_param->str, out); + KUNIT_EXPECT_STREQ_MSG(test, out, test_param->sha1, + "sha1sum(%s)", test_param->str); + } + + // Instead of KUNIT_CASE, we use KUNIT_CASE_PARAM and pass in the + // function declared by KUNIT_ARRAY_PARAM or KUNIT_ARRAY_PARAM_DESC. + static struct kunit_case sha1_test_cases[] = { + KUNIT_CASE_PARAM(sha1_test, sha1_gen_params), + {} + }; -.. code-block:: bash +Custom Parameter Generator Function: - modprobe example-test + The generator function is responsible for generating parameters one-by-one + and has the following signature: + ``const void* (*)(struct kunit *test, const void *prev, char *desc)``. + You can pass the generator function to the ``KUNIT_CASE_PARAM`` + or ``KUNIT_CASE_PARAM_WITH_INIT`` macros. -...will run the tests. + The function receives the previously generated parameter as the ``prev`` argument + (which is ``NULL`` on the first call) and can also access the parameterized + test context passed as the ``test`` argument. KUnit calls this function + repeatedly until it returns ``NULL``, which signifies that a parameterized + test ended. -Writing new tests for other architectures ------------------------------------------ + Below is an example of how it works: -The first thing you must do is ask yourself whether it is necessary to write a -KUnit test for a specific architecture, and then whether it is necessary to -write that test for a particular piece of hardware. In general, writing a test -that depends on having access to a particular piece of hardware or software (not -included in the Linux source repo) should be avoided at all costs. +.. code-block:: c -Even if you only ever plan on running your KUnit test on your hardware -configuration, other people may want to run your tests and may not have access -to your hardware. If you write your test to run on UML, then anyone can run your -tests without knowing anything about your particular setup, and you can still -run your tests on your hardware setup just by compiling for your architecture. + #define MAX_TEST_BUFFER_SIZE 8 -.. important:: - Always prefer tests that run on UML to tests that only run under a particular - architecture, and always prefer tests that run under QEMU or another easy - (and monetarily free) to obtain software environment to a specific piece of - hardware. - -Nevertheless, there are still valid reasons to write an architecture or hardware -specific test: for example, you might want to test some code that really belongs -in ``arch/some-arch/*``. Even so, try your best to write the test so that it -does not depend on physical hardware: if some of your test cases don't need the -hardware, only require the hardware for tests that actually need it. - -Now that you have narrowed down exactly what bits are hardware specific, the -actual procedure for writing and running the tests is pretty much the same as -writing normal KUnit tests. One special caveat is that you have to reset -hardware state in between test cases; if this is not possible, you may only be -able to run one test case per invocation. - -.. TODO(brendanhiggins@google.com): Add an actual example of an architecture - dependent KUnit test. + // Example generator function. It produces a sequence of buffer sizes that + // are powers of two, starting at 1 (e.g., 1, 2, 4, 8). + static const void *buffer_size_gen_params(struct kunit *test, const void *prev, char *desc) + { + long prev_buffer_size = (long)prev; + long next_buffer_size = 1; // Start with an initial size of 1. + + // Stop generating parameters if the limit is reached or exceeded. + if (prev_buffer_size >= MAX_TEST_BUFFER_SIZE) + return NULL; + + // For subsequent calls, calculate the next size by doubling the previous one. + if (prev) + next_buffer_size = prev_buffer_size << 1; + + return (void *)next_buffer_size; + } + + // Simple test to validate that kunit_kzalloc provides zeroed memory. + static void buffer_zero_test(struct kunit *test) + { + long buffer_size = (long)test->param_value; + // Use kunit_kzalloc to allocate a zero-initialized buffer. This makes the + // memory "parameter run managed," meaning it's automatically cleaned up at + // the end of each parameter run. + int *buf = kunit_kzalloc(test, buffer_size * sizeof(int), GFP_KERNEL); + + // Ensure the allocation was successful. + KUNIT_ASSERT_NOT_NULL(test, buf); + + // Loop through the buffer and confirm every element is zero. + for (int i = 0; i < buffer_size; i++) + KUNIT_EXPECT_EQ(test, buf[i], 0); + } + + static struct kunit_case buffer_test_cases[] = { + KUNIT_CASE_PARAM(buffer_zero_test, buffer_size_gen_params), + {} + }; + +Runtime Parameter Array Registration in the Init Function: + + For scenarios where you might need to initialize a parameterized test, you + can directly register a parameter array to the parameterized test context. + + To do this, you must pass the parameterized test context, the array itself, + the array size, and a ``get_description()`` function to the + ``kunit_register_params_array()`` macro. This macro populates + ``struct kunit_params`` within the parameterized test context, effectively + storing a parameter array object. The ``get_description()`` function will + be used for populating parameter descriptions and has the following signature: + ``void (*)(struct kunit *test, const void *param, char *desc)``. Note that it + also has access to the parameterized test context. + + .. important:: + When using this way to register a parameter array, you will need to + manually pass ``kunit_array_gen_params()`` as the generator function to + ``KUNIT_CASE_PARAM_WITH_INIT``. ``kunit_array_gen_params()`` is a KUnit + helper that will use the registered array to generate the parameters. + + If needed, instead of passing the KUnit helper, you can also pass your + own custom generator function that utilizes the parameter array. To + access the parameter array from within the parameter generator + function use ``test->params_array.params``. + + The ``kunit_register_params_array()`` macro should be called within a + ``param_init()`` function that initializes the parameterized test and has + the following signature ``int (*)(struct kunit *test)``. For a detailed + explanation of this mechanism please refer to the "Adding Shared Resources" + section that is after this one. This method supports registering both + dynamically built and static parameter arrays. + + The code snippet below shows the ``example_param_init_dynamic_arr`` test that + utilizes ``make_fibonacci_params()`` to create a dynamic array, which is then + registered using ``kunit_register_params_array()``. To see the full code + please refer to lib/kunit/kunit-example-test.c. + +.. code-block:: c + + /* + * Example of a parameterized test param_init() function that registers a dynamic + * array of parameters. + */ + static int example_param_init_dynamic_arr(struct kunit *test) + { + size_t seq_size; + int *fibonacci_params; + + kunit_info(test, "initializing parameterized test\n"); + + seq_size = 6; + fibonacci_params = make_fibonacci_params(test, seq_size); + if (!fibonacci_params) + return -ENOMEM; + /* + * Passes the dynamic parameter array information to the parameterized test + * context struct kunit. The array and its metadata will be stored in + * test->parent->params_array. The array itself will be located in + * params_data.params. + */ + kunit_register_params_array(test, fibonacci_params, seq_size, + example_param_dynamic_arr_get_desc); + return 0; + } + + static struct kunit_case example_test_cases[] = { + /* + * Note how we pass kunit_array_gen_params() to use the array we + * registered in example_param_init_dynamic_arr() to generate + * parameters. + */ + KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init_dynamic_arr, + kunit_array_gen_params, + example_param_init_dynamic_arr, + example_param_exit_dynamic_arr), + {} + }; + +Adding Shared Resources +^^^^^^^^^^^^^^^^^^^^^^^ +All parameter runs in this framework hold a reference to the parameterized test +context, which can be accessed using the parent ``struct kunit`` pointer. The +parameterized test context is not used to execute any test logic itself; instead, +it serves as a container for shared resources. + +It's possible to add resources to share between parameter runs within a +parameterized test by using ``KUNIT_CASE_PARAM_WITH_INIT``, to which you pass +custom ``param_init()`` and ``param_exit()`` functions. These functions run once +before and once after the parameterized test, respectively. + +The ``param_init()`` function, with the signature ``int (*)(struct kunit *test)``, +can be used for adding resources to the ``resources`` or ``priv`` fields of +the parameterized test context, registering the parameter array, and any other +initialization logic. + +The ``param_exit()`` function, with the signature ``void (*)(struct kunit *test)``, +can be used to release any resources that were not parameterized test managed (i.e. +not automatically cleaned up after the parameterized test ends) and for any other +exit logic. + +Both ``param_init()`` and ``param_exit()`` are passed the parameterized test +context behind the scenes. However, the test case function receives the parameter +run context. Therefore, to manage and access shared resources from within a test +case function, you must use ``test->parent``. + +For instance, finding a shared resource allocated by the Resource API requires +passing ``test->parent`` to ``kunit_find_resource()``. This principle extends to +all other APIs that might be used in the test case function, including +``kunit_kzalloc()``, ``kunit_kmalloc_array()``, and others (see +Documentation/dev-tools/kunit/api/test.rst and the +Documentation/dev-tools/kunit/api/resource.rst). + +.. note:: + The ``suite->init()`` function, which executes before each parameter run, + receives the parameter run context. Therefore, any resources set up in + ``suite->init()`` are cleaned up after each parameter run. + +The code below shows how you can add the shared resources. Note that this code +utilizes the Resource API, which you can read more about here: +Documentation/dev-tools/kunit/api/resource.rst. To see the full version of this +code please refer to lib/kunit/kunit-example-test.c. + +.. code-block:: c + + static int example_resource_init(struct kunit_resource *res, void *context) + { + ... /* Code that allocates memory and stores context in res->data. */ + } + + /* This function deallocates memory for the kunit_resource->data field. */ + static void example_resource_free(struct kunit_resource *res) + { + kfree(res->data); + } + + /* This match function locates a test resource based on defined criteria. */ + static bool example_resource_alloc_match(struct kunit *test, struct kunit_resource *res, + void *match_data) + { + return res->data && res->free == example_resource_free; + } + + /* Function to initialize the parameterized test. */ + static int example_param_init(struct kunit *test) + { + int ctx = 3; /* Data to be stored. */ + void *data = kunit_alloc_resource(test, example_resource_init, + example_resource_free, + GFP_KERNEL, &ctx); + if (!data) + return -ENOMEM; + kunit_register_params_array(test, example_params_array, + ARRAY_SIZE(example_params_array)); + return 0; + } + + /* Example test that uses shared resources in test->resources. */ + static void example_params_test_with_init(struct kunit *test) + { + int threshold; + const struct example_param *param = test->param_value; + /* Here we pass test->parent to access the parameterized test context. */ + struct kunit_resource *res = kunit_find_resource(test->parent, + example_resource_alloc_match, + NULL); + + threshold = *((int *)res->data); + KUNIT_ASSERT_LE(test, param->value, threshold); + kunit_put_resource(res); + } + + static struct kunit_case example_test_cases[] = { + KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init, kunit_array_gen_params, + example_param_init, NULL), + {} + }; + +As an alternative to using the KUnit Resource API for sharing resources, you can +place them in ``test->parent->priv``. This serves as a more lightweight method +for resource storage, best for scenarios where complex resource management is +not required. + +As stated previously ``param_init()`` and ``param_exit()`` get the parameterized +test context. So, you can directly use ``test->priv`` within ``param_init/exit`` +to manage shared resources. However, from within the test case function, you must +navigate up to the parent ``struct kunit`` i.e. the parameterized test context. +Therefore, you need to use ``test->parent->priv`` to access those same +resources. + +The resources placed in ``test->parent->priv`` will need to be allocated in +memory to persist across the parameter runs. If memory is allocated using the +KUnit memory allocation APIs (described more in the "Allocating Memory" section +below), you won't need to worry about deallocation. The APIs will make the memory +parameterized test 'managed', ensuring that it will automatically get cleaned up +after the parameterized test concludes. + +The code below demonstrates example usage of the ``priv`` field for shared +resources: + +.. code-block:: c + + static const struct example_param { + int value; + } example_params_array[] = { + { .value = 3, }, + { .value = 2, }, + { .value = 1, }, + { .value = 0, }, + }; + + /* Initialize the parameterized test context. */ + static int example_param_init_priv(struct kunit *test) + { + int ctx = 3; /* Data to be stored. */ + int arr_size = ARRAY_SIZE(example_params_array); + + /* + * Allocate memory using kunit_kzalloc(). Since the `param_init` + * function receives the parameterized test context, this memory + * allocation will be scoped to the lifetime of the parameterized test. + */ + test->priv = kunit_kzalloc(test, sizeof(int), GFP_KERNEL); + + /* Assign the context value to test->priv.*/ + *((int *)test->priv) = ctx; + + /* Register the parameter array. */ + kunit_register_params_array(test, example_params_array, arr_size, NULL); + return 0; + } + + static void example_params_test_with_init_priv(struct kunit *test) + { + int threshold; + const struct example_param *param = test->param_value; + + /* By design, test->parent will not be NULL. */ + KUNIT_ASSERT_NOT_NULL(test, test->parent); + + /* Here we use test->parent->priv to access the shared resource. */ + threshold = *(int *)test->parent->priv; + + KUNIT_ASSERT_LE(test, param->value, threshold); + } + + static struct kunit_case example_tests[] = { + KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init_priv, + kunit_array_gen_params, + example_param_init_priv, NULL), + {} + }; + +Allocating Memory +----------------- + +Where you might use ``kzalloc``, you can instead use ``kunit_kzalloc`` as KUnit +will then ensure that the memory is freed once the test completes. + +This is useful because it lets us use the ``KUNIT_ASSERT_EQ`` macros to exit +early from a test without having to worry about remembering to call ``kfree``. +For example: + +.. code-block:: c + + void example_test_allocation(struct kunit *test) + { + char *buffer = kunit_kzalloc(test, 16, GFP_KERNEL); + /* Ensure allocation succeeded. */ + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buffer); + + KUNIT_ASSERT_STREQ(test, buffer, ""); + } + +Registering Cleanup Actions +--------------------------- + +If you need to perform some cleanup beyond simple use of ``kunit_kzalloc``, +you can register a custom "deferred action", which is a cleanup function +run when the test exits (whether cleanly, or via a failed assertion). + +Actions are simple functions with no return value, and a single ``void*`` +context argument, and fulfill the same role as "cleanup" functions in Python +and Go tests, "defer" statements in languages which support them, and +(in some cases) destructors in RAII languages. + +These are very useful for unregistering things from global lists, closing +files or other resources, or freeing resources. + +For example: + +.. code-block:: C + + static void cleanup_device(void *ctx) + { + struct device *dev = (struct device *)ctx; + + device_unregister(dev); + } + + void example_device_test(struct kunit *test) + { + struct my_device dev; + + device_register(&dev); + + kunit_add_action(test, &cleanup_device, &dev); + } + +Note that, for functions like device_unregister which only accept a single +pointer-sized argument, it's possible to automatically generate a wrapper +with the ``KUNIT_DEFINE_ACTION_WRAPPER()`` macro, for example: + +.. code-block:: C + + KUNIT_DEFINE_ACTION_WRAPPER(device_unregister, device_unregister_wrapper, struct device *); + kunit_add_action(test, &device_unregister_wrapper, &dev); + +You should do this in preference to manually casting to the ``kunit_action_t`` type, +as casting function pointers will break Control Flow Integrity (CFI). + +``kunit_add_action`` can fail if, for example, the system is out of memory. +You can use ``kunit_add_action_or_reset`` instead which runs the action +immediately if it cannot be deferred. + +If you need more control over when the cleanup function is called, you +can trigger it early using ``kunit_release_action``, or cancel it entirely +with ``kunit_remove_action``. + + +Testing Static Functions +------------------------ + +If you want to test static functions without exposing those functions outside of +testing, one option is conditionally export the symbol. When KUnit is enabled, +the symbol is exposed but remains static otherwise. To use this method, follow +the template below. + +.. code-block:: c + + /* In the file containing functions to test "my_file.c" */ + + #include <kunit/visibility.h> + #include <my_file.h> + ... + VISIBLE_IF_KUNIT int do_interesting_thing() + { + ... + } + EXPORT_SYMBOL_IF_KUNIT(do_interesting_thing); + + /* In the header file "my_file.h" */ + + #if IS_ENABLED(CONFIG_KUNIT) + int do_interesting_thing(void); + #endif + + /* In the KUnit test file "my_file_test.c" */ + + #include <kunit/visibility.h> + #include <my_file.h> + ... + MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING"); + ... + // Use do_interesting_thing() in tests + +For a full example, see this `patch <https://lore.kernel.org/all/20221207014024.340230-3-rmoar@google.com/>`_ +where a test is modified to conditionally expose static functions for testing +using the macros above. + +As an **alternative** to the method above, you could conditionally ``#include`` +the test file at the end of your .c file. This is not recommended but works +if needed. For example: + +.. code-block:: c + + /* In "my_file.c" */ + + static int do_interesting_thing(); + + #ifdef CONFIG_MY_KUNIT_TEST + #include "my_kunit_test.c" + #endif + +Injecting Test-Only Code +------------------------ + +Similar to as shown above, we can add test-specific logic. For example: + +.. code-block:: c + + /* In my_file.h */ + + #ifdef CONFIG_MY_KUNIT_TEST + /* Defined in my_kunit_test.c */ + void test_only_hook(void); + #else + void test_only_hook(void) { } + #endif + +This test-only code can be made more useful by accessing the current ``kunit_test`` +as shown in next section: *Accessing The Current Test*. + +Accessing The Current Test +-------------------------- + +In some cases, we need to call test-only code from outside the test file. This +is helpful, for example, when providing a fake implementation of a function, or +to fail any current test from within an error handler. +We can do this via the ``kunit_test`` field in ``task_struct``, which we can +access using the ``kunit_get_current_test()`` function in ``kunit/test-bug.h``. + +``kunit_get_current_test()`` is safe to call even if KUnit is not enabled. If +KUnit is not enabled, or if no test is running in the current task, it will +return ``NULL``. This compiles down to either a no-op or a static key check, +so will have a negligible performance impact when no test is running. + +The example below uses this to implement a "mock" implementation of a function, ``foo``: + +.. code-block:: c + + #include <kunit/test-bug.h> /* for kunit_get_current_test */ + + struct test_data { + int foo_result; + int want_foo_called_with; + }; + + static int fake_foo(int arg) + { + struct kunit *test = kunit_get_current_test(); + struct test_data *test_data = test->priv; + + KUNIT_EXPECT_EQ(test, test_data->want_foo_called_with, arg); + return test_data->foo_result; + } + + static void example_simple_test(struct kunit *test) + { + /* Assume priv (private, a member used to pass test data from + * the init function) is allocated in the suite's .init */ + struct test_data *test_data = test->priv; + + test_data->foo_result = 42; + test_data->want_foo_called_with = 1; + + /* In a real test, we'd probably pass a pointer to fake_foo somewhere + * like an ops struct, etc. instead of calling it directly. */ + KUNIT_EXPECT_EQ(test, fake_foo(1), 42); + } + +In this example, we are using the ``priv`` member of ``struct kunit`` as a way +of passing data to the test from the init function. In general ``priv`` is +pointer that can be used for any user data. This is preferred over static +variables, as it avoids concurrency issues. + +Had we wanted something more flexible, we could have used a named ``kunit_resource``. +Each test can have multiple resources which have string names providing the same +flexibility as a ``priv`` member, but also, for example, allowing helper +functions to create resources without conflicting with each other. It is also +possible to define a clean up function for each resource, making it easy to +avoid resource leaks. For more information, see Documentation/dev-tools/kunit/api/resource.rst. + +Failing The Current Test +------------------------ + +If we want to fail the current test, we can use ``kunit_fail_current_test(fmt, args...)`` +which is defined in ``<kunit/test-bug.h>`` and does not require pulling in ``<kunit/test.h>``. +For example, we have an option to enable some extra debug checks on some data +structures as shown below: + +.. code-block:: c + + #include <kunit/test-bug.h> + + #ifdef CONFIG_EXTRA_DEBUG_CHECKS + static void validate_my_data(struct data *data) + { + if (is_valid(data)) + return; + + kunit_fail_current_test("data %p is invalid", data); + + /* Normal, non-KUnit, error reporting code here. */ + } + #else + static void my_debug_function(void) { } + #endif + +``kunit_fail_current_test()`` is safe to call even if KUnit is not enabled. If +KUnit is not enabled, or if no test is running in the current task, it will do +nothing. This compiles down to either a no-op or a static key check, so will +have a negligible performance impact when no test is running. + +Managing Fake Devices and Drivers +--------------------------------- + +When testing drivers or code which interacts with drivers, many functions will +require a ``struct device`` or ``struct device_driver``. In many cases, setting +up a real device is not required to test any given function, so a fake device +can be used instead. + +KUnit provides helper functions to create and manage these fake devices, which +are internally of type ``struct kunit_device``, and are attached to a special +``kunit_bus``. These devices support managed device resources (devres), as +described in Documentation/driver-api/driver-model/devres.rst + +To create a KUnit-managed ``struct device_driver``, use ``kunit_driver_create()``, +which will create a driver with the given name, on the ``kunit_bus``. This driver +will automatically be destroyed when the corresponding test finishes, but can also +be manually destroyed with ``driver_unregister()``. + +To create a fake device, use the ``kunit_device_register()``, which will create +and register a device, using a new KUnit-managed driver created with ``kunit_driver_create()``. +To provide a specific, non-KUnit-managed driver, use ``kunit_device_register_with_driver()`` +instead. Like with managed drivers, KUnit-managed fake devices are automatically +cleaned up when the test finishes, but can be manually cleaned up early with +``kunit_device_unregister()``. + +The KUnit devices should be used in preference to ``root_device_register()``, and +instead of ``platform_device_register()`` in cases where the device is not otherwise +a platform device. + +For example: + +.. code-block:: c + + #include <kunit/device.h> + + static void test_my_device(struct kunit *test) + { + struct device *fake_device; + const char *dev_managed_string; + + // Create a fake device. + fake_device = kunit_device_register(test, "my_device"); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, fake_device) + + // Pass it to functions which need a device. + dev_managed_string = devm_kstrdup(fake_device, "Hello, World!"); + + // Everything is cleaned up automatically when the test ends. + }
\ No newline at end of file |