summaryrefslogtreecommitdiff
path: root/include/kunit
AgeCommit message (Collapse)Author
2020-06-26kunit: add support for named resourcesAlan Maguire
The kunit resources API allows for custom initialization and cleanup code (init/fini); here a new resource add function sets the "struct kunit_resource" "name" field, and calls the standard add function. Having a simple way to name resources is useful in cases such as multithreaded tests where a set of resources are shared among threads; a pointer to the "struct kunit *" test state then is all that is needed to retrieve and use named resources. Support is provided to add, find and destroy named resources; the latter two are simply wrappers that use a "match-by-name" callback. If an attempt to add a resource with a name that already exists is made kunit_add_named_resource() will return -EEXIST. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-06-26kunit: generalize kunit_resource API beyond allocated resourcesAlan Maguire
In its original form, the kunit resources API - consisting the struct kunit_resource and associated functions - was focused on adding allocated resources during test operation that would be automatically cleaned up on test completion. The recent RFC patch proposing converting KASAN tests to KUnit [1] showed another potential model - where outside of test context, but with a pointer to the test state, we wish to access/update test-related data, but expressly want to avoid allocations. It turns out we can generalize the kunit_resource to support static resources where the struct kunit_resource * is passed in and initialized for us. As part of this work, we also change the "allocation" field to the more general "data" name, as instead of associating an allocation, we can associate a pointer to static data. Static data is distinguished by a NULL free functions. A test is added to cover using kunit_add_resource() with a static resource and data. Finally we also make use of the kernel's krefcount interfaces to manage reference counting of KUnit resources. The motivation for this is simple; if we have kernel threads accessing and using resources (say via kunit_find_resource()) we need to ensure we do not remove said resources (or indeed free them if they were dynamically allocated) until the reference count reaches zero. A new function - kunit_put_resource() - is added to handle this, and it should be called after a thread using kunit_find_resource() is finished with the retrieved resource. We ensure that the functions needed to look up, use and drop reference count are "static inline"-defined so that they can be used by builtin code as well as modules in the case that KUnit is built as a module. A cosmetic change here also; I've tried moving to kunit_[action]_resource() as the format of function names for consistency and readability. [1] https://lkml.org/lkml/2020/2/26/1286 Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-05-22Documentation: test.h - fix warningsLothar Rubusch
Fix warnings at 'make htmldocs', and formatting issues in the resulting documentation. - test.h: Fix annotation in kernel-doc parameter description. - Documentation/*.rst: Fixing formatting issues, and a duplicate label issue due to usage of sphinx.ext.autosectionlabel and identical labels within one document (sphinx warning) Signed-off-by: Lothar Rubusch <l.rubusch@gmail.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-03-26kunit: subtests should be indented 4 spaces according to TAPAlan Maguire
Introduce KUNIT_SUBTEST_INDENT macro which corresponds to 4-space indentation and KUNIT_SUBSUBTEST_INDENT macro which corresponds to 8-space indentation in line with TAP spec (e.g. see "Subtests" section of https://node-tap.org/tap-protocol/). Use these macros in place of one or two tabs in strings to clarify why we are indenting. Suggested-by: Frank Rowand <frowand.list@gmail.com> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Frank Rowand <frank.rowand@sony.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-03-26kunit: add debugfs /sys/kernel/debug/kunit/<suite>/results displayAlan Maguire
add debugfs support for displaying kunit test suite results; this is especially useful for module-loaded tests to allow disentangling of test result display from other dmesg events. debugfs support is provided if CONFIG_KUNIT_DEBUGFS=y. As well as printk()ing messages, we append them to a per-test log. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Reviewed-by: Frank Rowand <frank.rowand@sony.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-01-09kunit: allow kunit tests to be loaded as a moduleAlan Maguire
As tests are added to kunit, it will become less feasible to execute all built tests together. By supporting modular tests we provide a simple way to do selective execution on a running system; specifying CONFIG_KUNIT=y CONFIG_KUNIT_EXAMPLE_TEST=m ...means we can simply "insmod example-test.ko" to run the tests. To achieve this we need to do the following: o export the required symbols in kunit o string-stream tests utilize non-exported symbols so for now we skip building them when CONFIG_KUNIT_TEST=m. o drivers/base/power/qos-test.c contains a few unexported interface references, namely freq_qos_read_value() and freq_constraints_init(). Both of these could be potentially defined as static inline functions in include/linux/pm_qos.h, but for now we simply avoid supporting module build for that test suite. o support a new way of declaring test suites. Because a module cannot do multiple late_initcall()s, we provide a kunit_test_suites() macro to declare multiple suites within the same module at once. o some test module names would have been too general ("test-test" and "example-test" for kunit tests, "inode-test" for ext4 tests); rename these as appropriate ("kunit-test", "kunit-example-test" and "ext4-inode-test" respectively). Also define kunit_test_suite() via kunit_test_suites() as callers in other trees may need the old definition. Co-developed-by: Knut Omang <knut.omang@oracle.com> Signed-off-by: Knut Omang <knut.omang@oracle.com> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Acked-by: Theodore Ts'o <tytso@mit.edu> # for ext4 bits Acked-by: David Gow <davidgow@google.com> # For list-test Reported-by: kbuild test robot <lkp@intel.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-01-09kunit: hide unexported try-catch interface in try-catch-impl.hAlan Maguire
Define function as static inline in try-catch-impl.h to allow it to be used in kunit itself and tests. Also remove unused kunit_generic_try_catch Co-developed-by: Knut Omang <knut.omang@oracle.com> Signed-off-by: Knut Omang <knut.omang@oracle.com> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-01-09kunit: move string-stream.h to lib/kunitAlan Maguire
string-stream interfaces are not intended for external use; move them from include/kunit to lib/kunit accordingly. Co-developed-by: Knut Omang <knut.omang@oracle.com> Signed-off-by: Knut Omang <knut.omang@oracle.com> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-09-30kunit: fix failure to build without printkBrendan Higgins
Previously KUnit assumed that printk would always be present, which is not a valid assumption to make. Fix that by removing call to vprintk_emit, and calling printk directly. This fixes a build error[1] reported by Randy. For context this change comes after much discussion. My first stab[2] at this was just to make the KUnit logging code compile out; however, it was agreed that if we were going to use vprintk_emit, then vprintk_emit should provide a no-op stub, which lead to my second attempt[3]. In response to me trying to stub out vprintk_emit, Sergey Senozhatsky suggested a way for me to remove our usage of vprintk_emit, which led to my third attempt at solving this[4]. In my third version of this patch[4], I completely removed vprintk_emit, as suggested by Sergey; however, there was a bit of debate over whether Sergey's solution was the best. The debate arose due to Sergey's version resulting in a checkpatch warning, which resulted in a debate over correct printk usage. Joe Perches offered an alternative fix which was somewhat less far reaching than what Sergey had suggested and importantly relied on continuing to use %pV. Much of the debated centered around whether %pV should be widely used, and whether Sergey's version would result in object size bloat. Ultimately, we decided to go with Sergey's version. Reported-by: Randy Dunlap <rdunlap@infradead.org> Link[1]: https://lore.kernel.org/linux-kselftest/c7229254-0d90-d90e-f3df-5b6d6fc0b51f@infradead.org/ Link[2]: https://lore.kernel.org/linux-kselftest/20190827174932.44177-1-brendanhiggins@google.com/ Link[3]: https://lore.kernel.org/linux-kselftest/20190827234835.234473-1-brendanhiggins@google.com/ Link[4]: https://lore.kernel.org/linux-kselftest/20190828093143.163302-1-brendanhiggins@google.com/ Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> Cc: Joe Perches <joe@perches.com> Cc: Tim.Bird@sony.com Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Acked-by: Randy Dunlap <rdunlap@infradead.org> # build-tested Reviewed-by: Petr Mladek <pmladek@suse.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-09-30kunit: test: add the concept of assertionsBrendan Higgins
Add support for assertions which are like expectations except the test terminates if the assertion is not satisfied. The idea with assertions is that you use them to state all the preconditions for your test. Logically speaking, these are the premises of the test case, so if a premise isn't true, there is no point in continuing the test case because there are no conclusions that can be drawn without the premises. Whereas, the expectation is the thing you are trying to prove. It is not used universally in x-unit style test frameworks, but I really like it as a convention. You could still express the idea of a premise using the above idiom, but I think KUNIT_ASSERT_* states the intended idea perfectly. Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Stephen Boyd <sboyd@kernel.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-09-30kunit: test: add support for test abortBrendan Higgins
Add support for aborting/bailing out of test cases, which is needed for implementing assertions. An assertion is like an expectation, but bails out of the test case early if the assertion is not met. The idea with assertions is that you use them to state all the preconditions for your test. Logically speaking, these are the premises of the test case, so if a premise isn't true, there is no point in continuing the test case because there are no conclusions that can be drawn without the premises. Whereas, the expectation is the thing you are trying to prove. Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Stephen Boyd <sboyd@kernel.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-09-30kunit: test: add the concept of expectationsBrendan Higgins
Add support for expectations, which allow properties to be specified and then verified in tests. Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Stephen Boyd <sboyd@kernel.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-09-30kunit: test: add assertion printing libraryBrendan Higgins
Add `struct kunit_assert` and friends which provide a structured way to capture data from an expectation or an assertion (introduced later in the series) so that it may be printed out in the event of a failure. Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Reviewed-by: Stephen Boyd <sboyd@kernel.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-09-30kunit: test: add string_stream a std::stream like string builderBrendan Higgins
A number of test features need to do pretty complicated string printing where it may not be possible to rely on a single preallocated string with parameters. So provide a library for constructing the string as you go similar to C++'s std::string. string_stream is really just a string builder, nothing more. Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Stephen Boyd <swboyd@chromium.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-09-30kunit: test: add test resource management APIBrendan Higgins
Create a common API for test managed resources like memory and test objects. A lot of times a test will want to set up infrastructure to be used in test cases; this could be anything from just wanting to allocate some memory to setting up a driver stack; this defines facilities for creating "test resources" which are managed by the test infrastructure and are automatically cleaned up at the conclusion of the test. Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Stephen Boyd <sboyd@kernel.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-09-30kunit: test: add KUnit test runner coreBrendan Higgins
Add core facilities for defining unit tests; this provides a common way to define test cases, functions that execute code which is under test and determine whether the code under test behaves as expected; this also provides a way to group together related test cases in test suites (here we call them test_modules). Just define test cases and how to execute them for now; setting expectations on code will be defined later. Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org> Reviewed-by: Stephen Boyd <sboyd@kernel.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>