summaryrefslogtreecommitdiff
path: root/runner
AgeCommit message (Collapse)Author
2019-07-22runner: Make sure that we are closing watchdogs on signalsArkadiusz Hiler
There are few short windows of opportunity when watchdogs are primed but there is no signal handling in place, so the process may exit without proper shutdown sequence. This patch rearranges the existing code so that we set up the signalfd and BLOCK the signals before setting up watchdogs and UNBLOCK only after the watchdogs are closed properly. If igt_runner exits due to signal, non-zero status code is returned. v2: more error handling and minor touch ups (Simon) Cc: Petri Latvala <petri.latvala@intel.com> Cc: Simon Ser <simon.ser@intel.com> Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Simon Ser <simon.ser@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-07-22runner: Warn when watchdogs are being closed from the exit handlerArkadiusz Hiler
instead of being closed normally on a graceful code path Cc: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Simon Ser <simon.ser@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-07-22runner: Make sure we don't close watchdogs twiceArkadiusz Hiler
Setting the watchdog fd lists to NULL for extra fireworks if accessed unintentionally. Cc: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Simon Ser <simon.ser@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-07-16build: rename Meson optionsSimon Ser
Meson emits a warning because we use options beginning with "build_": DEPRECATION: Option uses prefix "build_", which is reserved for Meson. This will become an error in the future. Rename our options so that we don't use the Meson-reserved prefix. Signed-off-by: Simon Ser <simon.ser@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2019-06-24runner/settings: Be consistent with empty blacklistArkadiusz Hiler
If provided an empty blacklist let's fail instead of returning a value of uninitialized variable. Cc: Oleg Vasilev <oleg.vasilev@intel.com> Cc: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-06-24runner/resultgen: Don't pass NULL to str*() functionsArkadiusz Hiler
If we don't get values from the JSON strlen() and strcmp() on them would dereference NULL pointer. Cc: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-06-24runner/executor: Make sure that intervals_left is always initializedArkadiusz Hiler
intervals_left got initialized only when when we had a timeout exceeding watchdog capabilities, meaning we had to use multiple shorter intervals by moving intervals_left = timeout_intervals down we are always initializing it Cc: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-06-24runner: Log which signal was used to terminate the runnerArkadiusz Hiler
Feed the curious ones, aid the troubleshooters. Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Simon Ser <simon.ser@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-06-24runner: Handle SIGHUP tooArkadiusz Hiler
Default handler for SIGHUP is also terminating the process, so let's mask it and handle it manually, like the rest of the bunch. Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Simon Ser <simon.ser@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-06-24runner: Log when watchdog handling failsArkadiusz Hiler
If write or ioctl on a watchdog ever fails it will be logged. Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Simon Ser <simon.ser@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-06-19runner: add --list-all and --blacklistOleg Vasilev
Currently, runner already collects all subtest names into job_list. --list-all allows to output it to stdout. --blacklist option takes a filename as an argument and adds all regexes from that file to the exclusion list. v2: - Update exclude/include regex matches for tests without subtests to be matched with pigtit-like name (Petri) - Replace relative paths with those formatted with testdatadir (Petri) - Minor codestyle changes v3: - Print test names in lowercase (Petri) v4: - Replaced custom str_to_lower with generate_piglit_name (Petri) - Minor codestyle changes Cc: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Oleg Vasilev <oleg.vasilev@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-06-05runner_tests: Fix the file descriptor leakage subtestArkadiusz Hiler
Some libraries (looking at you, GnuTLS) may leave fds opened after the implicitly called library constructor. We don't have full control over them as they may be dependencies of our dependencies and may get pulled in if the user's and distribution's compile/configure/USE are just right. Let's close all the non-standard fds ahead of executing anything in the igt_runner tests, so we can test for descriptor leakage caused by any of the igt_runner code-paths exercised there. Cc: Antonio Argenziano <antonio.argenziano@intel.com> Cc: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Antonio Argenziano <antonio.argenziano@intel.com> Reviewed-by: Simon Ser <simon.ser@intel.com>
2019-06-03runner: Cope with repeated subtests in journalPetri Latvala
As seen in fallout with the i915 engine discovery patchset, subtests can in very dire cases be entered multiple times, when the for loop they're in doesn't progress. That is of course a bug in IGT that needs to be fixed, but if (when) that happens again, don't make results take a million years to process. This patch has been tested to produce good results from the corresponding bad runs. Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
2019-05-23meson: Start using 'feature' optionsArkadiusz Hiler
Meson 0.47 comes with a new type of option called 'feature' so instead of: type : 'combo', value : 'auto', choices : ['auto', 'true', 'false'], We can: type : 'feature', The main difference is that the feature takes auto, enabled and disabled instead of auto, true and false. get_option() on a feature returns opaque object that can be passed as a 'required' argument of a dependency. Auto is equivalent to 'required : false', enabled is equivalent to 'required : true' and disabled introduces new behavior forcing the dependency to be considered not found. This allows us to streamline a lot of logic regarding optional IGT features. This patch bumps required meson version to 0.47.0 Cc: Daniel Vetter <daniel.vetter@ffwll.ch> Cc: Petri Latvala <petri.latvala@intel.com> Cc: Simon Ser <simon.ser@intel.com> Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2019-05-10runner: Use glib's regex utilities instead of POSIX ERELyude Paul
POSIX ERE, while pretty standard is also very old and severely limited. In fact, it's so limited that Intel CI's own test blacklist, tests/intel-ci/blacklist.txt, doesn't even work with it due to the lack of support for backreferences. Since we're already using glib in other parts of igt, let's just start using glib's regular expression matching instead of the POSIX regex API. This gives us full perl compatible regular expressions and in turn, also gives us python compatible regular expressions to match piglit. Reviewed-by: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Lyude Paul <lyude@redhat.com>
2019-05-10runner: Disable building if tests are not builtPetri Latvala
Signed-off-by: Petri Latvala <petri.latvala@intel.com> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=110248 Cc: Eero Tamminen <eero.t.tamminen@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2019-05-10runner_tests: Operate within defined behaviorArkadiusz Hiler
Quoting lib/igt_core.c: * - Code blocks with magic control flow are implemented with setjmp() and * longjmp(). This applies to #igt_fixture and #igt_subtest blocks and all the * three variants to finish test: igt_success(), igt_skip() and igt_fail(). * Mostly this is of no concern, except when such a control block changes * stack variables defined in the same function as the control block resides. * Any store/load behaviour after a longjmp() is ill-defined for these * variables. Avoid such code. * * Quoting the man page for longjmp(): * * "The values of automatic variables are unspecified after a call to * longjmp() if they meet all the following criteria:" * - "they are local to the function that made the corresponding setjmp() call; * - "their values are changed between the calls to setjmp() and longjmp(); and * - "they are not declared as volatile." We have quite a few igt_subtest_group that define local variables to share the state between subtest and fixture (e.g. set open() in subtest, close() in the following fixture), causing the contents to be undefined. We can allocate some of them using malloc and make the others volatile to be back on the defined grounds. Cc: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-04-16runner/job_list: print error when job list is emptySimon Ser
Using a filter that doesn't match any test name resulted in the runner silently failing. Print an error message so that the user understands why the runner fails. Signed-off-by: Simon Ser <simon.ser@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-04-12runner: Make sure oom-killer doesn't kill the runnerPetri Latvala
Tests that eat all of the RAM and then some to invoke the oom-killer deliberately sometimes cause extra casualties. Make sure the runner stays alive. Signed-off-by: Petri Latvala <petri.latvala@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
2019-04-01runner: Add --dmesg-warn-level switchArkadiusz Hiler
This switch allows users to select which dmesg log level is treated as warning resulting in overriding the test results to dmesg-fail/dmesg-warn. Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-04-01runner: Refactor metadata parsingArkadiusz Hiler
To aid testing function parsing metadata.txt is split into outer helper that operates on dirfd and inner function that operates on FILE*. This allows us to test the parsing using fmemopen(), limiting the amount of necessary boilerplate. Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-04-01runner: Reinitialize compiled dmesg regexp each parsing sessionArkadiusz Hiler
Which regexp gets compiled is settings specific, depending whether we run piglit-style or not. If it's optimized to be initialized only once and it is a global variable, it will be "stuck" in the mode we have selected with the first run, which may break tests. Let's remove this optimization and initialize it each time, as it takes less 0.002s on my hardware. Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Simon Ser <simon.ser@intel.com>
2019-03-27runner: Make taint abort messages more verboseArkadiusz Hiler
Since not everyone is familiar with kernel taints, and it is easy to get confused and attribute an abort to an innocent TAINT_USER caused by an unsafe module option, which is usually the first thing people find greping dmesg for "taint", we should provide more guidance. This patch extends the abort log by printing the taint names, as found in the kernel, along with a short explanation, so people know what to look for in the dmesg. v2: rebase, reword Cc: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com> Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2019-03-25runner/executor: refactor error handlingSimon Ser
* Refactor to use goto error handling * Make execute_test_process noreturn to remove uninitialized variable warning * Check fork() return value Signed-off-by: Simon Ser <simon.ser@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2019-02-21runner: Exit with 0 on dry-runPetri Latvala
v2: Adjust tests accordingly Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2019-02-19runner: Exit with 2 when overall timeout exceededPetri Latvala
That leaves exitcode 1 for aborts and initialization failures. Should maybe differentiate those as well. Not to mention document the exit codes. Also fix igt_resume to follow suit to igt_runner: Generate results.json even when aborting or exceeding overall-timeout. Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2019-02-15lib: Drop IGT_EXIT_TIMEOUTDaniel Vetter
We use the timeout status for when the runner had to kill a testcase, which indicates a more sever issue than an operation failing that we expected to complete within seconds. Since it's unused, drop it. Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
2019-01-29automake: Drop runner build supportDaniel Vetter
Main use case here is CI, which already builds using meson. Acked-by: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
2019-01-22runner: Implement --dry-runPetri Latvala
Actually implement --dry-run to not execute tests. With dry-run active, attempting to execute will figure out the list of things to execute, serialize them along with settings, and stop. This will be useful for CI that wants to post-mortem on failed test rounds to generate a list of tests that should have been executed and produce json result files (full of 'notrun') for proper statistics. Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Andi Shyti <andi.shyti@intel.com> Cc: Martin Peres <martin.peres@linux.intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Martin Peres <martin.peres@linux.intel.com>
2018-12-20runner: Add explicit "notrun" results for tests that were not executedPetri Latvala
When possible, all tests we know we were going to attempt to execute now appear in the results as "notrun". The only known case where it's not possible to add an explicit "notrun" is when running in multiple-mode, because "no subtests" and "run all subtests, we didn't list them beforehand" are represented the same. v2: Rebase and adjust to already landed json changes Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Martin Peres <martin.peres@linux.intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Acked-by: Martin Peres <martin.peres@linux.intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-12-12runner: Produce json with results_version=10Petri Latvala
New piglit bumped its results_version to 10, making glxinfo and pals optional in practice, not just by accident. Unfortunately reading results with newer piglit attempts to convert the results to version 10, reading glxinfo and pals, and thus fails. In a hilarious summary: A commit to piglit making glxinfo optional makes it mandatory for us. v2: json unit tests confirmed to be working... Reported-by: Andi Shyti <andi.shyti@intel.com> Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Cc: Andi Shyti <andi.shyti@intel.com> Tested-by: Andi Shyti <andi.shyti@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-12-12runner: Handle escaped characters from kmsgPetri Latvala
Characters in kernel logs, when read from /dev/kmsg, are escaped as \xNN if they are not between 32 and 127 of if they are "\". Decode what we can when creating results.json. Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-11-27lib: Show stacktrace when terminated by runnerChris Wilson
The igt_runner sends a SIGTERM to ask the test to cleanly exit upon an external timeout. It is useful to know what the code was doing when the timeout occurred, just in case it was unexpectedly stuck. However, since we use SIGTERM internally to marshal helper processes, we want to keep SIGTERM quiet, and so opt to use SIGQUIT for the timeout request instead. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Petri Latvala <petri.latvala@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2018-11-27runner: Flip kill argsChris Wilson
int kill(pid_t pid, int sig)! Fixes: a6b514d242bd ("runner: Be patient for processes to die") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Petri Latvala <petri.latvala@intel.com> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2018-11-27Revert "runner: Treat dmesg warnings as pure warnings"Chris Wilson
Accidentally pushed, believing this was the kill(child, 0) switcheroo. This reverts commit 0be3f3e7c1613dcaf27267fce778025ea46a36c1. Acked-by: Petri Latvala <petri.latvala@intel.com>
2018-11-27runner: Treat dmesg warnings as pure warningsChris Wilson
I have whinged on for ages about the dmesg-warnings being an expected part of kernel testing (where else is the kernel meant to log its errors?) and should be treated the same as our stderr for the test. That is if a test fails, it fails and does not need to be conflated with whether or not there was a dmesg warning (just as the test saying why it failed on stderr does not need flagging), and that a passing test with a dmesg warning is simply a warn. The effect is that we simply remove the "dmesg-" flagging from results names, as the err/dmesg output is simply collated for the error report already. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Petri Latvala <petri.latvala@intel.com> Cc: Martin Peres <martin.peres@linux.intel.com>
2018-11-27runner: Use %lx for printing unsigned longsPetri Latvala
../runner/executor.c:555:54: warning: format ‘%x’ expects argument of type ‘unsigned int’, but argument 3 has type ‘long unsigned int’ [-Wformat=] fprintf(stderr, "Child refuses to die, tainted %x. Aborting.\n", ~^ Fixes: a6b514d242bd ("runner: Be patient for processes to die") Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
2018-11-27runner: Be patient for processes to dieChris Wilson
Some machines are very slow and some processes hog a lot of resources and so take much longer than a mere 2s to be terminated. Be patient. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108073 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Petri Latvala <petri.latvala@intel.com>
2018-11-20runner: Exit with a nonzero status if abortingPetri Latvala
Signed-off-by: Petri Latvala <petri.latvala@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-11-15runner: Implement --abort-on-monitored-errorPetri Latvala
Deviating a bit from the piglit command line flag, igt_runner takes an optional comma-separated list as an argument to --abort-on-monitored-error for the list of conditions to abort on. Without a list all possible conditions will be checked. Two conditions implemented: - "taint" checks the kernel taint level for TAINT_PAGE, TAINT_DIE and TAINT_OOPS - "lockdep" checks the kernel lockdep status Checking is done after every test binary execution, and if an abort condition is met, the reason is printed to stderr (unless log level is quiet) and the runner doesn't execute any further tests. Aborting between subtests (when running in --multiple-mode) is not done. v2: - Remember to fclose - Taints are unsigned long (Chris) - Use getline instead of fgets (Chris) v3: - Fix brainfart with lockdep v4: - Rebase - Refactor the abort condition checking to pass down strings - Present the abort result in results.json as a pseudo test result - Unit tests for the pseudo result v5: - Refactors (Chris) - Don't claim lockdep was triggered if debug_locks is not on anymore. Just say it's not active. - Dump lockdep_stats when aborting due to lockdep (Chris) - Use igt@runner@aborted instead for the pseudo result (Martin) v6: - If aborting after a test, generate results.json. Like was already done for aborting at startup. - Print the test that would be executed next as well when aborting, as requested by Tomi. v7: - Remove the resolved TODO item from commit message Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Cc: Martin Peres <martin.peres@linux.intel.com> Cc: Daniel Vetter <daniel.vetter@ffwll.ch> Cc: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-11-06runner: Unit tests for json generationPetri Latvala
Test the results.json generation with a top-down approach: With a directory of test run intermediary logs, check that the resulting json would match a reference json file. Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-11-06runner/resultgen: Split json generation and file writingPetri Latvala
This allows testing to skip the file writing. Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-11-05runner: const-correct the unit testsPetri Latvala
const where possible, and cast away const when passing argv to parse_options, which expects non-const strings, because it passes them to getopt_long, which expects non-const strings... getopt and getopt_long take the argv array as char * const *, or in other words, as pointer-to-const-pointer-to-char. In C, pointer-to-T implicitly converts to pointer-to-const-T and for a char **, the T is char* and "const T" is char * const, ergo char ** converts to char * const *, not const char **. The only const-correctness getopt and getopt_long can really do is char * const * or they lose the ability to directly pass in main()'s arguments, which are an array of non-const pointers to non-const char for legacy reasons. For testing the argument handling, it's very convenient to use an array of string literals, which are of type const char[N], convertible to const char *. To get such an array into getopt, the choices are: 1) Cast away the const in the pointer-to-pointer 2) Cast away the const in the string literal 3) Don't cast anything and eat the compiler warning Option 1 looked cleanest out of all those. tl;dr: Choices made in 1972 force our hand. v2: - Augment commit message Signed-off-by: Petri Latvala <petri.latvala@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-11-05runner: Fix declaration of timeofday_doublePetri Latvala
Fixes compiler warning: function declaration isn’t a prototype Signed-off-by: Petri Latvala <petri.latvala@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-10-23runner: Add "root" also to the "totals" objectPetri Latvala
Turns out the same information is looked up in different places in different code paths. Piglit's summary module looks up total counts in ['totals']['root'], CI looks up ['totals']['']. The latter is inherited from piglit, so this has probably changed at some point. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108486 Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Cc: Andi Shyti <andi.shyti@intel.com> Tested-by: Andi Shyti <andi.shyti@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-10-19runner: Add --overall-timeoutPetri Latvala
With --overall-timeout $foo, the runner will stop executing new tests when $foo seconds have already been used. A resumed run will start over with no time used, using the same timeout. This allows for executing a long list of tests piecemeal, in about $foo length executions. Signed-off-by: Petri Latvala <petri.latvala@intel.com> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=106127 Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Cc: Martin Peres <martin.peres@linux.intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-10-15runner: Fix results when there's warns and dmesg-warnsPetri Latvala
After setting the result object text, the string retrieved from the old object is invalid. Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Martin Peres <martin.peres@linux.intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2018-10-04runner: Add unit test for resume when enough subtests are already runPetri Latvala
Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-10-04runner: Don't try to execute an empty set of subtestsPetri Latvala
If we resume a test run with igt_resume, or if resume is done automatically from a test timeout, the runner will execute the last attempted test with the subtest selection set to original set minus the subtests already journaled to have started. If this results in an empty set, we get a harmless but misleading message from the test saying "igt_core-WARNING: Unknown subtest: subtest-name,!subtest-name" If the journal already contains as many subtests as we have requested (when we know the set), assume we have them all already and move to the next job list entry instead. Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Martin Peres <martin.peres@linux.intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
2018-10-03runner: Add a unit test that checks for fd leaksPetri Latvala
Pretty much needed, as proven. Signed-off-by: Petri Latvala <petri.latvala@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-12 00:14:36 +0000