| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arek@hiler.eu>
Reviewed-by: Swati Sharma <swati2.sharma@intel.com>
|
|
The gcc compiler has a feature that enables checking the code coverage
in runtime[1].
[1] See https://www.kernel.org/doc/html/latest/dev-tools/gcov.html
The Linux Kernel comes with an option to enable such feature:
./scripts/config -e DEBUG_FS -e GCOV_KERNEL
The driver's Makefile also needs change to enable it. For instance, in
order to enable GCOV for all DRM drivers, one would need to run:
for i in $(find drivers/gpu/drm/ -name Makefile); do
sed '1 a GCOV_PROFILE := y' -i $i
done
This patch adds support for it by:
a) Implementing a logic to cleanup the code coverage counters via sysfs;
b) Calling a script responsible for collecging code coverage data.
The implementation works with two modes:
1) It zeroes the counters, run all IGT tests and collects the code
coverage results at the end.
This implies that no tests would crash the driver, as otherwise the
results won't be collected;
This is faster, as collecting code coverage data can take several
seconds.
2) For each test, it will clean the code coverage counters, run the and
collect the results.
This is more reliable, as a Kernel crash/OOPS won't affect the
results of the previously ran tests.
Reviewed-by: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
|
|
Without root permissions, most IGT tests won't actually run, but they
would be displayed at the runner's output as if everything went fine.
In order to avoid that, check if one attempts to run IGT without root
permission. Such check can be disbled with a new command line option:
--allow-non-root
As runner_tests runs as non-root, most unit tests need to pass
--allow-non-root in order for them to not return an error.
Reviewed-by: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
|
|
Disk usage limit is a limit of disk space taken, per (dynamic)
subtest. If the test's output, kernel log included, exceeds this
limit, the test is killed, similarly to killing the test when the
kernel gets tainted.
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
Noticed while running some tests that adding any kind of spaces into the
name of a test run would stop igt_resume from working for said test run.
Turns out that when we parse test metadata, we accidentally use the
'%ms' specifier with fscanf() which finishes parsing strings when any
kind of whitespace is encountered.
So, fix this by using the proper %m[^\n] specifier, which dynamically
allocates it's result and doesn't stop reading the string until a
newline is encountered. Additionally, add a test for this.
Signed-off-by: Lyude Paul <lyude@redhat.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>
|
|
Now that the IGT tests have a mechanism for signaling broken testing
conditions we can stop the run on the first test that has noticed it,
and possibly has triggered that state.
Traditionally run would have continued with that test failing and the
side effects would trickle down into the other tests causing a lot of
skip/fails.
v2: extra explanations, small cleanup (Petri)
Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>
|
|
to make this bit of code more readable and to reuse it in the following patch
Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>
|
|
A new config option, --per-test-timeout, sets a time a single test
cannot exceed without getting itself killed. The time resets when
starting a subtest or a dynamic subtest, so an execution with
--per-test-timeout=20 can indeed go over 20 seconds a long as it
launches a dynamic subtest within that time.
As a bonus, verbose log level from runner now also prints dynamic
subtest begin/result.
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
|
|
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
See also: commit 0e6457f1bfe2 ("lib: Don't dump log buffer when
dynamic subtest failure is inherited")
This is quite an explicit top-to-bottom test that we don't get an
incorrect warn result for an innocent dynamic subtest. It is tested
here in runner_test instead of testing in lib/tests that the extra
lines are not printed, because not printing the extra lines is an
implementation detail that might change later. The issue is after all
about test results parsed by igt_runner.
v2: Squash adding the new mockup test binary to this commit
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
We have some handcrafted test binaries in runner/testdata/ for runner
testing, and hardcoded numbers for the total amount of subtests and
test binaries all over the runner's unit tests. Replace magic numbers
with clear defines so new tests can easily be added.
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
Currently, runner already collects all subtest names into job_list.
--list-all allows to output it to stdout.
--blacklist option takes a filename as an argument and adds all regexes
from that file to the exclusion list.
v2:
- Update exclude/include regex matches for tests without
subtests to be matched with pigtit-like name (Petri)
- Replace relative paths with those formatted with testdatadir (Petri)
- Minor codestyle changes
v3:
- Print test names in lowercase (Petri)
v4:
- Replaced custom str_to_lower with generate_piglit_name (Petri)
- Minor codestyle changes
Cc: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Oleg Vasilev <oleg.vasilev@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>
|
|
Some libraries (looking at you, GnuTLS) may leave fds opened after the
implicitly called library constructor. We don't have full control over
them as they may be dependencies of our dependencies and may get pulled
in if the user's and distribution's compile/configure/USE are just
right.
Let's close all the non-standard fds ahead of executing anything in the
igt_runner tests, so we can test for descriptor leakage caused by any of
the igt_runner code-paths exercised there.
Cc: Antonio Argenziano <antonio.argenziano@intel.com>
Cc: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Antonio Argenziano <antonio.argenziano@intel.com>
Reviewed-by: Simon Ser <simon.ser@intel.com>
|
|
Quoting lib/igt_core.c:
* - Code blocks with magic control flow are implemented with setjmp() and
* longjmp(). This applies to #igt_fixture and #igt_subtest blocks and all the
* three variants to finish test: igt_success(), igt_skip() and igt_fail().
* Mostly this is of no concern, except when such a control block changes
* stack variables defined in the same function as the control block resides.
* Any store/load behaviour after a longjmp() is ill-defined for these
* variables. Avoid such code.
*
* Quoting the man page for longjmp():
*
* "The values of automatic variables are unspecified after a call to
* longjmp() if they meet all the following criteria:"
* - "they are local to the function that made the corresponding setjmp() call;
* - "their values are changed between the calls to setjmp() and longjmp(); and
* - "they are not declared as volatile."
We have quite a few igt_subtest_group that define local variables to
share the state between subtest and fixture (e.g. set open() in subtest,
close() in the following fixture), causing the contents to be undefined.
We can allocate some of them using malloc and make the others volatile
to be back on the defined grounds.
Cc: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>
|
|
This switch allows users to select which dmesg log level is treated as
warning resulting in overriding the test results to
dmesg-fail/dmesg-warn.
Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>
|
|
To aid testing function parsing metadata.txt is split into outer helper
that operates on dirfd and inner function that operates on FILE*.
This allows us to test the parsing using fmemopen(), limiting the amount
of necessary boilerplate.
Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>
|
|
v2: Adjust tests accordingly
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Cc: Tomi Sarvela <tomi.p.sarvela@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
Actually implement --dry-run to not execute tests. With dry-run
active, attempting to execute will figure out the list of things to
execute, serialize them along with settings, and stop. This will be
useful for CI that wants to post-mortem on failed test rounds to
generate a list of tests that should have been executed and produce
json result files (full of 'notrun') for proper statistics.
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Andi Shyti <andi.shyti@intel.com>
Cc: Martin Peres <martin.peres@linux.intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Cc: Tomi Sarvela <tomi.p.sarvela@intel.com>
Reviewed-by: Martin Peres <martin.peres@linux.intel.com>
|
|
Deviating a bit from the piglit command line flag, igt_runner takes an
optional comma-separated list as an argument to
--abort-on-monitored-error for the list of conditions to abort
on. Without a list all possible conditions will be checked.
Two conditions implemented:
- "taint" checks the kernel taint level for TAINT_PAGE, TAINT_DIE and
TAINT_OOPS
- "lockdep" checks the kernel lockdep status
Checking is done after every test binary execution, and if an abort
condition is met, the reason is printed to stderr (unless log level is
quiet) and the runner doesn't execute any further tests. Aborting
between subtests (when running in --multiple-mode) is not done.
v2:
- Remember to fclose
- Taints are unsigned long (Chris)
- Use getline instead of fgets (Chris)
v3:
- Fix brainfart with lockdep
v4:
- Rebase
- Refactor the abort condition checking to pass down strings
- Present the abort result in results.json as a pseudo test result
- Unit tests for the pseudo result
v5:
- Refactors (Chris)
- Don't claim lockdep was triggered if debug_locks is not on
anymore. Just say it's not active.
- Dump lockdep_stats when aborting due to lockdep (Chris)
- Use igt@runner@aborted instead for the pseudo result (Martin)
v6:
- If aborting after a test, generate results.json. Like was already
done for aborting at startup.
- Print the test that would be executed next as well when aborting,
as requested by Tomi.
v7:
- Remove the resolved TODO item from commit message
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Cc: Tomi Sarvela <tomi.p.sarvela@intel.com>
Cc: Martin Peres <martin.peres@linux.intel.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
const where possible, and cast away const when passing argv to
parse_options, which expects non-const strings, because it passes them
to getopt_long, which expects non-const strings...
getopt and getopt_long take the argv array as char * const *, or in
other words, as pointer-to-const-pointer-to-char. In C, pointer-to-T
implicitly converts to pointer-to-const-T and for a char **, the T is
char* and "const T" is char * const, ergo char ** converts to char *
const *, not const char **. The only const-correctness getopt and
getopt_long can really do is char * const * or they lose the ability
to directly pass in main()'s arguments, which are an array of
non-const pointers to non-const char for legacy reasons.
For testing the argument handling, it's very convenient to use an
array of string literals, which are of type const char[N], convertible
to const char *. To get such an array into getopt, the choices are:
1) Cast away the const in the pointer-to-pointer
2) Cast away the const in the string literal
3) Don't cast anything and eat the compiler warning
Option 1 looked cleanest out of all those.
tl;dr: Choices made in 1972 force our hand.
v2:
- Augment commit message
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
With --overall-timeout $foo, the runner will stop executing new tests
when $foo seconds have already been used.
A resumed run will start over with no time used, using the same
timeout. This allows for executing a long list of tests piecemeal, in
about $foo length executions.
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=106127
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Cc: Tomi Sarvela <tomi.p.sarvela@intel.com>
Cc: Martin Peres <martin.peres@linux.intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
Pretty much needed, as proven.
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
Actually implement what was already commented to work.
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Acked-by: Tomi Sarvela <tomi.p.sarvela@intel.com> #irc
|
|
With the flag, dmesg handling is done exactly as piglit does it: Level
5 (info) and higher dmesg lines, if they match a regexp, cause test
result to change to dmesg-*.
The default is false (use new method).
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Cc: Martin Peres <martin.peres@linux.intel.com>
Acked-by: Martin Peres <martin.peres@linux.intel.com>
|
|
Instead of just matching the binary/subtest name.
Originally not implemented to get the runner landed faster. Turned out
to be simple enough.
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Tomi Sarvela <tomi.p.sarvela@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
|
absolute_path() tends to return NULL if more than the last element of
the path is nonexistent. That behavior is confusing the callers, which
use NULL as a convention for something not being set at all.
Let's fix that by sprinkling a little bit of recursion onto
absolute_path() and wrapping POSIX in some sanity.
Cc: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>
|
|
TODO: Unit tests for the results.json file contents.
v2:
- Avoid writing the nul character to mock files
- Properly set up tmpdirs
v3:
- Restore the resume-related changes that were lost in rebase
v4:
- Better teardown for temporary directories
- Build with autotools
Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
|
