Configuration time and execution time
Build phases describes the phases of every Gradle build. Let’s zoom into the configuration and execution phases of a multi-project build. Configuration here means evaluating the build script file of a project, which includes downloading all plugins and build script dependencies. By default, the configuration of all projects happens before any task is executed. This means that when a single task, from a single project is requested, all projects of a multi-project build are configured first. The reason every project needs to be configured is to support the flexibility of accessing and changing any part of the Gradle project model.
With more and more CPU cores available on developer desktops and CI servers, it is important that Gradle is able to fully utilise these processing resources. More specifically, parallel execution attempts to:
Reduce total build time for a multi-project build where execution is IO bound or otherwise does not consume all available CPU resources.
Provide faster feedback for execution of small projects without awaiting completion of other projects.
Although Gradle already offers parallel test execution via Test.setMaxParallelForks(int) the feature described in this section is parallel execution at a project level.
Parallel project execution allows the separate projects in a decoupled multi-project build to be executed in parallel (see also Decoupled projects). While parallel execution does not strictly require decoupling at configuration time, the long-term goal is to provide a powerful set of features that will be available for fully decoupled projects. Such features include:
Configuration of projects in parallel.
Re-use of configuration for unchanged projects.
Project-level up-to-date checks.
Using pre-built artifacts in the place of building dependent projects.
How does parallel execution work? First, you need to tell Gradle to use parallel mode.
You can use the
--parallel command line argument or configure your build environment (Gradle properties).
Unless you provide a specific number of parallel threads, Gradle attempts to choose the right number based on available CPU cores.
Every parallel worker exclusively owns a given project while executing a task.
Task dependencies are fully supported and parallel workers will start executing upstream tasks first.
Bear in mind that the alphabetical ordering of decoupled tasks, as can be seen during sequential execution, is not guaranteed in parallel mode.
In other words, in parallel mode tasks will run as soon as their dependencies complete and a task worker is available to run them, which may be earlier than they would start during a sequential build.
You should make sure that task dependencies and task inputs/outputs are declared correctly to avoid ordering issues.
Gradle allows any project to access any other project during both the configuration and execution phases. While this provides a great deal of power and flexibility to the build author, it also limits the flexibility that Gradle has when building those projects. For instance, this effectively prevents Gradle from correctly building multiple projects in parallel, configuring only a subset of projects, or from substituting a pre-built artifact in place of a project dependency.
Two projects are said to be decoupled if they do not directly access each other’s project model. Decoupled projects may only interact in terms of declared dependencies: project dependencies and/or task dependencies. Any other form of project interaction (i.e. by modifying another project object or by reading a value from another project object) causes the projects to be coupled. The consequence of coupling during the configuration phase is that if gradle is invoked with the 'configuration on demand' option, the result of the build can be flawed in several ways. The consequence of coupling during execution phase is that if gradle is invoked with the parallel option, one project task runs too late to influence a task of a project building in parallel. Gradle does not attempt to detect coupling and warn the user, as there are too many possibilities to introduce coupling.
A very common way for projects to be coupled is by using configuration injection.
It may not be immediately apparent, but using key Gradle features like the
subprojects keywords automatically cause your projects to be coupled.
This is because these keywords are used in a
build.gradle file, which defines a project.
Often this is a “root project” that does nothing more than define common configuration, but as far as Gradle is concerned this root project is still a fully-fledged project, and by using
allprojects that project is effectively coupled to all other projects.
Coupling of the root project to subprojects does not impact configuration on-demand, but using the
subprojects in any subproject’s
build.gradle file will have an impact.
This means that using any form of shared build script logic or configuration injection (
subprojects, etc.) will cause your projects to be coupled.
As we extend the concept of project decoupling and provide features that take advantage of decoupled projects, we will also introduce new features to help you to solve common use cases (like configuration injection) without causing your projects to be coupled.
In order to make good use of cross project configuration without running into issues for parallel and 'configuration on demand' options, follow these recommendations:
Avoid a subproject’s build script referencing other subprojects; preferring cross configuration from the root project.
Avoid changing the configuration of other projects at execution time.
The Configuration injection feature and access to the complete project model are possible because every project is configured before the execution phase. Yet, this approach may not be the most efficient in a very large multi-project build. There are Gradle builds with a hierarchy of hundreds of subprojects. The configuration time of huge multi-project builds may become noticeable.
Configuration on demand attempts to configure only projects that are relevant for requested tasks, i.e. it only executes the build script file of projects that are participating in the build. This way, the configuration time of a large multi-project build can be reduced.
The configuration on demand feature is incubating, so not every build is guaranteed to work correctly. The feature should work very well for multi-project builds that have decoupled projects. In “configuration on demand” mode, projects are configured as follows:
The root project is always configured.
The project in the directory where the build is executed is also configured, but only when Gradle is executed without any tasks. This way the default tasks behave correctly when projects are configured on demand.
The standard project dependencies are supported and makes relevant projects configured. If project A has a compile dependency on project B then building A causes configuration of both projects.
The task dependencies declared via task path are supported and cause relevant projects to be configured. Example:
A task requested via task path from the command line (or Tooling API) causes the relevant project to be configured. For example, building 'project-a:project-b:someTask' causes configuration of project-b.