igt/media-bench.pl: Media workload analyzer

The high level goal of this script is to programatically analyze
the simulated media workloads (gem_wsim) by finding an optimal
load balancing strategy, and also detecting any possible
shortcomings of the same.

When run without command line arguments script will run through
both of its phases.

In the first phase it will be running all the known balancers
against the all the known workloads, and for each combination
look for a point where aggregated system throughput cannot be
increased by running more parallel workload instances.

At that point each balancer gets a score proportional to the
throughput achieved, which is added to the running total for the
complete phase.

Several different score boards are kept - total throughput, per
client throughput and combined (total + per client). Weighted
scoreboards are also kept where scores are weighted based on the
total variance detected for a single workload. This means scores
for workloads which respond well to being balanced will be worth
more than of the ones which do not balance well in neither of
the configurations.

Based on the first phase a "best" balancing strategy will be
selected based on the combined weighted scoreboard.

Second phase will then proceed to profile all the selected
workloads with this balancer and look at potential problems with
GPU engines not being completely saturated.

If none of the active engine is saturated the workload will be
flagged, as it will if the only saturated engine is one of the
ones which can be balanced, but the other one in the same class
is under-utilized.

Flagged workloads then need to be analyzed which can be achieved
by looking at the html of the engine timelines which are
generated during this phase. (These files are all put in the
current working directory.)

It is quite possible that something flagged by the script as
suspect is completely fine and just a consequence of the
workload in question being fundementally unbalanced.

It is possible to skip directly to the second phase of the
evaluation by using the -b command line option. This option must
contain a string exactly as understood by gem_wsim's -b option.
For example '-b "-b rtavg -R"'.

Apart from being run with no arguments, script also supports a
selection of command line switches to enable fine tuning.

For example, also including the complete output from the script
in order to be more illustrative:

-8<---
+ scripts/media-bench.pl -n 642317 -r 2 -B rand,rtavg -W media_load_balance_hd12.wsim,media_load_balance_fhd26u7.wsim
Workloads:
  media_load_balance_hd12.wsim
  media_load_balance_fhd26u7.wsim
Balancers: rand,rtavg,
Target workload duration is 2s.
Calibration tolerance is 0.01.
Nop calibration is 642317.

Evaluating 'media_load_balance_hd12.wsim'... 2s is 990 workloads. (error=0.00750000000000006)
  Finding saturation points for 'media_load_balance_hd12.wsim'...
    rand balancer ('-b rand'): 6 clients (1412.576 wps, 235.429333333333 wps/client).
    rand balancer ('-b rand -R'): 6 clients (1419.639 wps, 236.6065 wps/client).
    rtavg balancer ('-b rtavg'): 5 clients (1430.143 wps, 286.0286 wps/client).
    rtavg balancer ('-b rtavg -H'): 5 clients (1339.775 wps, 267.955 wps/client).
    rtavg balancer ('-b rtavg -R'): 5 clients (1386.384 wps, 277.2768 wps/client).
    rtavg balancer ('-b rtavg -R -H'): 6 clients (1365.943 wps, 227.657166666667 wps/client).
  Best balancer is '-b rtavg'.

Evaluating 'media_load_balance_fhd26u7.wsim'... 2s is 52 workloads. (error=0.002)
  Finding saturation points for 'media_load_balance_fhd26u7.wsim'...
    rand balancer ('-b rand'): 3 clients (46.532 wps, 15.5106666666667 wps/client).
    rand balancer ('-b rand -R'): 3 clients (46.242 wps, 15.414 wps/client).
    rtavg balancer ('-b rtavg'): 6 clients (61.232 wps, 10.2053333333333 wps/client).
    rtavg balancer ('-b rtavg -H'): 4 clients (57.608 wps, 14.402 wps/client).
    rtavg balancer ('-b rtavg -R'): 6 clients (61.793 wps, 10.2988333333333 wps/client).
    rtavg balancer ('-b rtavg -R -H'): 7 clients (60.697 wps, 8.671 wps/client).
  Best balancer is '-b rtavg -R'.

Total wps rank:
===============
  1: '-b rtavg' (1)
  2: '-b rtavg -R' (0.989191465637926)
  3: '-b rtavg -R -H' (0.973103630772601)
  4: '-b rtavg -H' (0.938804458876241)
  5: '-b rand -R' (0.874465740398305)
  6: '-b rand' (0.874342391093453)

Total weighted wps rank:
========================
  1: '-b rtavg -R' (1)
  2: '-b rtavg' (0.998877134022041)
  3: '-b rtavg -R -H' (0.982849160383224)
  4: '-b rtavg -H' (0.938950446314292)
  5: '-b rand' (0.80507369080098)
  6: '-b rand -R' (0.80229656623594)

Per client wps rank:
====================
  1: '-b rtavg -H' (1)
  2: '-b rand' (0.977356849770376)
  3: '-b rand -R' (0.976222085591368)
  4: '-b rtavg' (0.888825068013012)
  5: '-b rtavg -R' (0.875653417817828)
  6: '-b rtavg -R -H' (0.726389466714194)

Per client weighted wps rank:
=============================
  1: '-b rand' (1)
  2: '-b rand -R' (0.996866139192282)
  3: '-b rtavg -H' (0.986348733324348)
  4: '-b rtavg' (0.811593544774355)
  5: '-b rtavg -R' (0.805704548552663)
  6: '-b rtavg -R -H' (0.671567075453688)

Combined wps rank:
==================
  1: '-b rtavg' (1)
  2: '-b rtavg -R' (0.989191465637926)
  3: '-b rtavg -H' (0.972251783752137)
  4: '-b rtavg -R -H' (0.949708930404222)
  5: '-b rand' (0.914594701126905)
  6: '-b rand -R' (0.914312395840401)

Combined weighted wps rank:
===========================
  1: '-b rtavg' (1)
  2: '-b rtavg -R' (0.995945739226824)
  3: '-b rtavg -H' (0.984347862855008)
  4: '-b rtavg -R -H' (0.956920992185625)
  5: '-b rand' (0.899001713089319)
  6: '-b rand -R' (0.896984246540919)

Balancer is '-b rtavg'.
Idleness tolerance is 2%.

Profiling 'media_load_balance_hd12.wsim'...
      2s is 992 workloads. (error=0.00150000000000006)
      Saturation at 6 clients (1434.207 workloads/s).
    Pass [ 0: 0.57%, 2: 22.59%, 3: 23.30%, ]

Profiling 'media_load_balance_fhd26u7.wsim'...
      2s is 52 workloads. (error=0.001)
      Saturation at 6 clients (61.823 workloads/s).
    WARN [ 0: 7.77%, 2: 0.66%, 3: 28.70%, ]

Problematic workloads were:
   media_load_balance_fhd26u7.wsim -c 6 -r 52 [ 0: 7.77%, 2: 0.66%, 3: 28.70%, ]

-8<---

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

1 files changed, 1 insertions, 1 deletions

diff --git a/scripts/Makefile.am b/scripts/Makefile.am
index 64171529..e26a39e2 100644
--- a/scripts/Makefile.am
+++ b/scripts/Makefile.am
@@ -1,2 +1,2 @@
-dist_noinst_SCRIPTS = intel-gfx-trybot who.sh run-tests.sh trace.pl
+dist_noinst_SCRIPTS = intel-gfx-trybot who.sh run-tests.sh trace.pl media-bench.pl
 noinst_PYTHON = throttle.py