docs/code_coverage.md: document the code coverage filter script

Add documentation bits for the code_coverage_parse_info.

While here, also drop the extensions from the scripts that were renamed.

Reviewed-by: Ch Sai Gowtham <sai.gowtham.ch@intel.com>
Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

1 files changed, 224 insertions, 14 deletions

diff --git a/docs/code_coverage.md b/docs/code_coverage.md
index 34d56d28..915e800e 100644
--- a/docs/code_coverage.md
+++ b/docs/code_coverage.md
@@ -132,15 +132,15 @@ Kernel that was built at the same machine, at the directory `~/linux`,
 and wants to capture one file per test, it would use:
 
 ```
-./scripts/run-tests.sh -T my.testlist -k ~/linux -c scripts/code_cov_capture.sh -P
+./scripts/run-tests.sh -T my.testlist -k ~/linux -c code_cov_capture -P
 ```
 
 ### Code Coverage Collect script
 
 While any script could in thesis be used, currently, there are two ones
-under `scripts/`:
+under the IGT's `scripts/` source directory:
 
-- `scripts/code_cov_capture.sh`:
+- `code_cov_capture`:
 
   Assumes that the Kernel was built at the same machine, and uses
   the lcov tool to generate GCC-independent code coverage data,
@@ -150,7 +150,7 @@ under `scripts/`:
 
   Such script requires `lcov` tool to be installed at the test machine.
 
-- `scripts/code_cov_gather_on_test.py`:
+- `code_cov_gather_on_test`:
 
   Generates a gzipped tarbal with the code coverage counters in
   binary format. Such kind of output should then be parsed at
@@ -162,12 +162,12 @@ For each script, the igt_runner passes just one parameter: the results
 directory + the test name.
 
 For instance, if it is needed to run a test called
-`debugfs_test (read_all_entries)` using `scripts/code_cov_capture.sh`
+`debugfs_test (read_all_entries)` using `code_cov_capture`
 parameter, e. g.:
 
 ```
 $ echo "igt@debugfs_test@read_all_entries" > my.testlist
-$ ./scripts/run-tests.sh -T my.testlist -k ~/linux -c scripts/code_cov_capture.sh -P
+$ ./scripts/run-tests.sh -T my.testlist -k ~/linux -c code_cov_capture -P
 Found test list: "/basedir/igt/build/tests/test-list.txt"
 [31410.499969] [1/1] debugfs_test (read_all_entries)
 [31411.060446] Storing code coverage results...
@@ -178,7 +178,7 @@ Done.
 The script will be called as:
 
 ```
-scripts/code_cov_capture.sh /basedir/igt/results/code_cov/debugfs_test_read_all_entries
+code_cov_capture results/code_cov/debugfs_test_read_all_entries
 ```
 
 Please notice that any character that it is not a number nor a letter at the
@@ -190,6 +190,73 @@ as titles at the lcov files.
 If any extra global parameters are needed by the script, those can be sent
 via shell's environment var.
 
+## Parsing data from code coverage *.info files
+
+The `*.info` files generated by `lcov` are plain text files that list the
+tests that were executed in runtime.
+
+The `code_cov_parse_info` script has some logic on it that allows
+printing the called functions stored inside the `*.info` file. It can also
+optionally apply the following filters. Its main options are:
+
+- `--stat` or `--statistics`
+
+  Prints code coverage statistics.
+
+  It displays function, line, branch and file coverage percentage.
+
+  The statistics report is affected by the applied filters.
+
+- `--print-coverage`, `--print` or `-p`
+
+
+  Prints the functions that were executed in runtime and how many times
+  they were reached.
+
+  The function coverage report is affected by the applied filters.
+
+- `--print-unused` or `-u`
+
+  Prints the functions that were never reached.
+
+  The function coverage report is affected by the applied filters.
+
+- `--show-lines` or `--show_lines`
+
+  When printing per-function code coverage data, always output the source
+  file and the line number where the function is defined.
+
+- `--output` *output file* or `-o` *output file*
+
+  Produces an output file merging all input files.
+
+  The generated output file is affected by the applied filters.
+
+- `--show-files` or `--show_files`
+
+   Shows the list of files that were useed to produce the code coverage
+   results.
+
+- It also has a set of parameters that filters the code coverage results:
+  `--only-drm`, `--only-i915`, `--func-filters`, `--source-filters`,
+  `--ignore-unused`.
+  When used, all coverage displayed reports, and the stored output file
+  will be affected by such filters.
+
+More details can be seen by calling:
+
+```
+code_cov_parse_info --help
+
+```
+
+or:
+
+```
+code_cov_parse_info --man
+
+```
+
 ### The `*.info` file format
 
 The `*.info` files contain several fields on it, grouped into records.
@@ -224,7 +291,7 @@ So, the above example means that, inside
 ## Generating code coverage documentation
 
 The `lcov` package contains the needed tools to parse and generate code
-coverage documentation. It is used by `scripts/code_cov_capture.sh` script
+coverage documentation. It is used by `code_cov_capture` script
 to convery from compiler-dependent `*.gcno` counters into a
 compiler-independent format (`*.info`).
 
@@ -243,11 +310,16 @@ source files. Some optional arguments can be used at the command line, or
 can be stored at `/etc/lcovrc` or `~/.lcovrc` files.
 
 As generating the documentation depends wheather the results were generated
-as with a single or multiple `*.info` files by `scripts/code_cov_capture.sh`
+as with a single or multiple `*.info` files by `code_cov_capture`
 or stored in raw formats inside `*.tar.gz` file(s) by
-`scripts/code_cov_gather_on_test.py`, there's a script that does all the
+`code_cov_gather_on_test`, there's a script that does all the
 required steps to build the code coverage html reports:
-`scripts/code_cov_gen_report.sh`.
+`code_cov_gen_report`. Besides its own command line arguments, it
+also accepts arguments to be passed to `code_cov_parse_info`.
+
+If a `code_cov_parse_info` command line parameter is passed, it will
+also call the script, in order to use a filtered `*.info` file to be
+used when generating the HTML reports.
 
 It requires the following arguments:
 
@@ -276,18 +348,156 @@ It requires the following arguments:
 
   The files specified by `--read` are gzipped tarballs containing all
   `*.gcno` files and all `*.gcda` softlinks from the `/sys/kernel/debug/gcov/`
-  directory at the test machine, created by
-  `scripts/code_cov_gather_on_test.py` script.
+  directory at the test machine, created by `code_cov_gather_on_test` script.
 
 - `--force-override`
 
   Allow using a non-empty directory for `--output-dir`.
 
+- It also accepts `--print`, `--only-drm`, `--only-i915` and `--ignore-unused`
+  options from `code_cov_parse_info`.
+
 `--info` and `--tar` are mutually exclusive and at least one of them should
 be specified.
 
+## Code coverage capture script example
+
+### Capture, parse and generate code coverage html data
+
+The script below provides a simple yet powerful script using code
+coverage capture on a test machine that also contains the Linux
+Kernel source and objects. It assumes that LGT was installed.
+
+```
+#/bin/bash -e
+
+TESTLIST="my_tests.testlist"
+OUT_DIR="${HOME}/results"
+
+mkdir -p $OUT_DIR/html
+
+echo "igt@debugfs_test@read_all_entries" > $TESTLIST
+echo "igt@core_auth@basic-auth" >> $TESTLIST
+echo "igt@gem_exec_basic@basic" >> $TESTLIST
+
+sudo IGT_KERNEL_TREE="${HOME}/linux" igt_runner -s -o --coverage-per-test \
+                  --collect-script code_cov_capture --test-list $TESTLIST \
+                  /usr/local/libexec/igt-gpu-tools $OUT_DIR/ | sed s,$HOME/,,
+
+sudo chown -R $(id -u):$(id -g) $OUT_DIR/
+
+for i in $OUT_DIR/code_cov/*.info; do
+        echo -e "\n$(basename $i):"
+        code_cov_parse_info --only-drm --ignore-unused --stat $i
+done
+echo -e "\nTOTAL:"
+code_cov_parse_info --only-drm --stat --output $OUT_DIR/results.info \
+        $OUT_DIR/code_cov/*.info
+
+cd $OUT_DIR/html
+genhtml -q -s --legend --branch-coverage $OUT_DIR/results.info
+```
+
+Running such script produces the following output:
+
+```
+[3622.993304] [1/3] debugfs_test (read_all_entries)
+[3631.95]     Code coverage wrote to results/code_cov/debugfs_test_read_all_entries.info
+[3626.217016] Storing code coverage results...
+[3631.957998] [2/3] core_auth (basic-auth)
+[3638.03]     Code coverage wrote to results/code_cov/core_auth_basic_auth.info
+[3632.116024] Storing code coverage results...
+[3638.070869] [3/3] gem_exec_basic (basic)
+[3644.24]     Code coverage wrote to results/code_cov/gem_exec_basic_basic.info
+[3638.366790] Storing code coverage results...
+Done.
+
+core_auth_basic_auth.info:
+  lines......: 11.7% (8217 of 70257 lines)
+  functions..: 7.1% (776 of 10971 functions)
+  branches...: 7.0% (3596 of 51041 branches)
+Ignored......: non-drm headers and source files where none of its code ran.
+Source files.: 23.27% (165 of 709 total), 29.57% (165 of 558 filtered)
+
+debugfs_test_read_all_entries.info:
+  lines......: 19.3% (20266 of 104802 lines)
+  functions..: 17.5% (1922 of 10971 functions)
+  branches...: 12.7% (9462 of 74555 branches)
+Ignored......: non-drm headers and source files where none of its code ran.
+Source files.: 34.70% (246 of 709 total), 44.09% (246 of 558 filtered)
+
+gem_exec_basic_basic.info:
+  lines......: 17.1% (14964 of 87503 lines)
+  functions..: 13.0% (1422 of 10971 functions)
+  branches...: 10.1% (6446 of 63758 branches)
+Ignored......: non-drm headers and source files where none of its code ran.
+Source files.: 30.89% (219 of 709 total), 39.25% (219 of 558 filtered)
+
+TOTAL:
+  lines......: 15.5% (25821 of 166849 lines)
+  functions..: 22.1% (2429 of 10971 functions)
+  branches...: 10.5% (11869 of 112665 branches)
+Ignored......: non-drm headers.
+Source files.: 78.70% (558 of 709 total)
+```
+
+### Reporting detailed function coverage stored on *.info files
+
+The `code_cov_parse_info` script can be used alone in order to provide
+a text file output containing code coverage data obtained from a *.info
+file. For example, listing code coverage usage for all functions whose name
+contains "edid_" can be done with:
+
+```
+$ echo edid_ >filter.txt
+$ code_cov_parse_info --func-filters filter.txt results/results.info -p -u --stat
+TEST: Code_coverage_tests
+__drm_get_edid_firmware_path(): unused
+__drm_set_edid_firmware_path(): unused
+displayid_iter_edid_begin(): executed 10 times
+drm_add_edid_modes(): executed 2 times
+drm_add_override_edid_modes(): unused
+drm_connector_attach_edid_property(): unused
+drm_connector_update_edid_property(): executed 8 times
+drm_dp_send_real_edid_checksum(): unused
+drm_edid_are_equal(): executed 4 times
+drm_edid_block_valid(): executed 8 times
+drm_edid_duplicate(): unused
+drm_edid_get_monitor_name(): unused
+drm_edid_header_is_valid(): executed 4 times
+drm_edid_is_valid(): executed 2 times
+drm_edid_to_eld(): executed 2 times
+drm_edid_to_sad(): unused
+drm_edid_to_speaker_allocation(): unused
+drm_find_edid_extension(): executed 22 times
+drm_get_edid_switcheroo(): unused
+drm_load_edid_firmware(): executed 2 times
+edid_firmware_get(): unused
+edid_firmware_set(): unused
+edid_fixup_preferred(): unused
+edid_get_quirks(): executed 6 times
+edid_load(): unused
+edid_open(): executed 4 times
+edid_show() from linux/drivers/gpu/drm/drm_debugfs.c: executed 4 times
+edid_show() from linux/drivers/gpu/drm/drm_sysfs.c: unused
+edid_vendor(): executed 348 times
+edid_write(): unused
+intel_panel_edid_downclock_mode(): unused
+intel_panel_edid_fixed_mode(): unused
+is_edid_digital_input_dp(): unused
+  lines......: 5.5% (5 of 91 lines)
+  functions..: 42.4% (14 of 33 functions)
+  branches...: 1.9% (1 of 52 branches)
+Ignored......: unmatched functions m/(?^:edid_)/ and source files where none of its code ran.
+Source files.: 0.90% (5 of 558 total), 55.56% (5 of 9 filtered)
+```
+
+When the function is unique, it will just display the function name and how
+many times the IGT test(s) executed it. When the same function name exists
+on multiple files (like the `edid_show()` on the above example), it will
+display multiple lines, one for each different function/file combination.
+
 ## References
 
 More information is available at Kernel gcov documentation:
 [Using gcov with the Linux kernel](https://www.kernel.org/doc/html/latest/dev-tools/gcov.html).
-