buildroot.git/support/scripts/graph-depends, branch master

graph-depends: split off get_rdepends from get_depends

2017-02-06T11:28:00+00:00

The use of a 'rule' variable that can contain 'show-depends' or 'show-rdepends' is not logical if get_depends is considered as a reusable function from various scripts. The name of these rules are too much an implementation detail. Therefore, split the existing get_depends into two separate functions get_depends and get_rdepends, while keeping code duplication to a minimum. Signed-off-by: Thomas De Schampheleire Reviewed-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

graph-depends: split off get_version/get_depends into pkgutil.py

2017-02-06T11:26:55+00:00

Functions to obtain the version and dependencies of a package from Python can be useful for several scripts. Extract this logic out of graph-depends into pkgutil.py. Signed-off-by: Thomas De Schampheleire Reviewed-by: "Yann E. MORIN" [Thomas: remove shebang from pkgutil.py, noticed by Yann E. Morin.] Signed-off-by: Thomas Petazzoni

graph-depends: avoid use of global var 'rule' in get_depends

2017-02-05T14:20:02+00:00

Function get_depends was recently changed to support both normal dependencies as reverse dependencies, via a global variable 'rule' that equals 'show-depends' or 'show-rdepends'. As a subsequent function will extract this function get_depends to a separate file, the use of globals is problematic. Instead, pass the global as an argument. Signed-off-by: Thomas De Schampheleire Signed-off-by: Thomas Petazzoni

core/graph-depends: add option to graph reverse dependencies

2016-10-25T20:59:05+00:00

Now that we can dump the reverse dependencies of a package, add the ability to graph those. It does not make sense to do a full reverse graph, as it would be semantically equivalent to the direct graph. So we only provide a per-package reverse graph. Signed-off-by: "Yann E. MORIN" Cc: Arnout Vandecappelle Cc: Thomas Petazzoni Signed-off-by: Thomas Petazzoni

support/scripts: fix graph-depends when run with python3

2016-04-18T09:34:33+00:00

Make graph-depends script opening the output file in text mode since only ascii characters will be written. This change fixes the following error occuring when the default host python interpreter is python3: make: Entering directory '/opt/buildroot' Getting targets Getting dependencies for ['toolchain-external', 'toolchain', 'busybox', ...] Getting dependencies for ['host-python3', 'host-pkgconf', 'host-gettext', ...] Getting dependencies for ['host-libxml2', 'host-swig', 'host-m4', ...] Getting version for ['toolchain-external', 'toolchain', 'busybox', ...] Traceback (most recent call last): File "/opt/buildroot/support/scripts/graph-depends", line 425, in outfile.write("digraph G {\n") TypeError: a bytes-like object is required, not 'str' Makefile:807: recipe for target 'graph-depends' failed make[1]: *** [graph-depends] Error 1 Makefile:84: recipe for target '_all' failed make: *** [_all] Error 2 make: Leaving directory '/opt/buildroot' While with python2, adding 'b' to the openning mode has no effect on Linux (c.f. [2]), the above error is expected with python3 (c.f. [1]). Therefore, just open the outfile in default (i.e. text) mode. [1] https://docs.python.org/3/library/functions.html#open [2] https://docs.python.org/2/library/functions.html#open Signed-off-by: Samuel Martin Acked-by: Arnout Vandecappelle (Essensium/Mind) Signed-off-by: Thomas Petazzoni

graph-depends: add support for excluding host packages

2016-02-08T20:29:06+00:00

Just like the --stop-on and --exclude options allow to stop on or exclude virtual packages from the list by passing the "virtual" magic value, this commit extends the graph-depends logic to support a "host" magic value for --stop-on and --exclude. This will allow to draw the graph by stopping on host packages, or by excluding host packages. Signed-off-by: Thomas Petazzoni Tested-by: "Yann E. MORIN" Reviewed-by: "Yann E. MORIN" [Thomas: minor code beautification suggested by Yann E. Morin.] Signed-off-by: Thomas Petazzoni

graph-depends: fix handling of "virtual" in exclude_list

2016-02-08T20:28:19+00:00

The condition to determine if a virtual package should be excluded from the list due to "virtual" being passed in --exclude is under a loop iterating over each entry of the exclude_list, but it doesn't use the iterator of this list. Indeed, the condition contains: "virtual" in exclude_list which checks automatically if "virtual" was passed in the list. Due to this, there is no need for this check to be within the "for p in exclude_list" iteration. This commit fixes that by moving the check outside of the loop. Signed-off-by: Thomas Petazzoni Tested-by: "Yann E. MORIN" Acked-by: "Yann E. MORIN" Signed-off-by: Thomas Petazzoni

support/graph-depends: teach it to only check dependencies

2016-02-07T22:21:40+00:00

Add an option to graph-depends to only do the dependency checks and not generate the dot program. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Samuel Martin Signed-off-by: Thomas Petazzoni

support/graph-depends: detect circular dependencies

2016-02-07T22:20:38+00:00

Currently, if there is a circular dependency in the packages, the graph-depends script just errors out with a Python RuntimeError which is not caught, resulting in a very-long backtrace which does not provide any hint as what the real issue is (even if "RuntimeError: maximum recursion depth exceeded" is a pretty good hint at it). We fix that by recursing the dependency chain of each package, until we either end up with a package with no dependency, or with a package already seen along the current dependency chain. We need to introduce a new function, check_circular_deps(), because we can't re-use the existing ones: - remove_mandatory_deps() does not iterate, - remove_transitive_deps() does iterate, but we do not call it for the top-level package if it is not 'all' - it does not make sense to use those functions anyway, as they were not designed to _check_ but to _act_ on the dependency chain. Since we've had time-related issues in the past, we do not want to introduce yet another time-hog, so here are timings with the circular dependency check: $ time python -m cProfile -s cumtime support/scripts/graph-depends [...] 28352654 function calls (20323050 primitive calls) in 87.292 seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.012 0.012 87.292 87.292 graph-depends:24() 21 0.000 0.000 73.685 3.509 subprocess.py:473(_eintr_retry_call) 7 0.000 0.000 73.655 10.522 subprocess.py:768(communicate) 7 73.653 10.522 73.653 10.522 {method 'read' of 'file' objects} 5/1 0.027 0.005 43.488 43.488 graph-depends:164(get_all_depends) 5 0.003 0.001 43.458 8.692 graph-depends:135(get_depends) 1 0.001 0.001 25.712 25.712 graph-depends:98(get_version) 1 0.001 0.001 13.457 13.457 graph-depends:337(remove_extra_deps) 1717 1.672 0.001 13.050 0.008 graph-depends:290(remove_transitive_deps) 9784086/2672326 5.079 0.000 11.363 0.000 graph-depends:274(is_dep) 2883343/1980154 2.650 0.000 6.942 0.000 graph-depends:262(is_dep_uncached) 1 0.000 0.000 4.529 4.529 graph-depends:121(get_targets) 2883343 1.123 0.000 1.851 0.000 graph-depends:246(is_dep_cache_insert) 9784086 1.783 0.000 1.783 0.000 graph-depends:255(is_dep_cache_lookup) 2881580 0.728 0.000 0.728 0.000 {method 'update' of 'dict' objects} 1 0.001 0.001 0.405 0.405 graph-depends:311(check_circular_deps) 12264/1717 0.290 0.000 0.404 0.000 graph-depends:312(recurse) [...] real 1m27.371s user 1m15.075s sys 0m12.673s The cumulative time spent in check_circular_deps is just below 0.5s, which is largely less than 1% of the total run time. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Samuel Martin Signed-off-by: Thomas Petazzoni

support/graph-depends: add option to specify output file

2016-02-07T22:15:30+00:00

Currently, graph-depends outputs the dotfile program to stdout, and uses stderr to trace the dependencies it is currently looking for. Redirection was done because the output was directly piped into the dot program to generate the final PDF/SVG/... dependency graph, but that meant that an error in the graph-depends script was never caught (because shell pipes only return the final command exit status, and an empty dot program is perfectly valid so dot would not complain). Add an option to tell graph-depends where to store the generated dot program, and keep stdout as the default if not specified. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Samuel Martin [Thomas: rename metavar from DOT_FILE to OUT_FILE for consistency with the rest of the new option naming.] Signed-off-by: Thomas Petazzoni