We are asking early adopters to try out a new API for creating and configuring tasks to iron out any issues and gather feedback. This chapter provides a quick introduction to the feature and explains some guidelines for migrating your build to use the new API. We intend for this new API to eventually replace the existing API through our usual deprecation process over several major releases. As of Gradle 5.0, we believe the API will become the recommended approach to creating tasks in the next minor release. The core part of the API is identical back to Gradle 4.9. There may be some convenience methods added in later versions of Gradle that do not exist in older releaes.

The new Task configuration API is currently incubating. Please be aware that the DSL or API may change in minor Gradle versions.

What is the new API good for?

In a nutshell, the new API allows builds to avoid the cost of creating and configuring tasks during Gradle’s configuration phase when those tasks will never be executed. This is part of our approach to evolving the Gradle API to reduce configuration time. We identified task configuration as a contributor to overall configuration time. Configuration time affects every build, so making it faster is good for everyone. For example, if you’re only compiling, Gradle doesn’t need to also configure code quality, testing and publishing tasks. For tasks that aren’t used often by developers, configuration can be expensive (e.g., running git status or querying a webservice) and can slow down builds that do not need to execute them. This new API avoids configuring those tasks if they are not needed.

This feature is designed to be backwards compatible when mixed with the old and new APIs, so you can gradually migrate your plugins and build scripts to the new API. Instead of immediately creating a task and running all configuration actions, Gradle exposes a Provider to a task. A task will only be created and configured when the task is required. If at any point, a task registered with the new API is accessed through the old API, the task will be created and configured just as if it had been created with the old API.

Many built-in Gradle plugins have been converted to use this new API already, so you may see some benefits without changing anything.

What do we want you to do?

We’ve been using this feature in many of the built-in Gradle plugins, internal Gradle projects and the Gradle build itself. Now we would appreciate your help in evaluating it with your builds as well.

We’d like for you to take a few steps:

  • Try your build with the latest nightly (see the version number at the top of this page).

  • How many tasks do you see registered with the new API? See How do I know it’s working? for how to find this.

  • Which task types are most numerous? Which ones are created with the old API? Which ones are registered with the new API, but have been created/configured anyways?

  • Following Pitfalls and migration guidelines, change your build scripts to use the new API to avoid creating/configuring tasks that are from the built-in Gradle plugins. Is there an impact to the number of tasks that are created?

  • Convert some of your custom plugins/tasks to use the new API. Is there an impact to the number of tasks using the new API and the number of new API tasks that are created?

  • Run the gradle-profiler to measure the difference between a build with less task configuration and a build where all tasks are configured. What do you see? Use the scenario file below.

Guidelines

How do I defer task creation?

This feature requires build authors to opt-in by migrating task creation from the TaskContainer.create(java.lang.String) APIs to the TaskContainer.register(java.lang.String) APIs. The register(…​) API registers a task to be created at a later time if and only if the task is needed. The create(…​) API continues to eagerly create and configure tasks when it is called.

Using the new creation API may not be enough to avoid all task configuration completely. You may need to change other code that configures tasks by name or by type, as explained in the following sections.

How do I defer task configuration?

Existing APIs like DomainObjectCollection.all(org.gradle.api.Action) and DomainObjectCollection.withType(java.lang.Class, org.gradle.api.Action) will immediately create and configure any registered tasks. To defer task configuration, you will need to migrate to a new API equivalent. See the table below to identify the new API alternative.

How do I reference a task without creating/configuring it?

Calling Provider.get() or looking up a task by name with TaskCollection.getByName(java.lang.String) will cause the task to be created and configured. Methods like Task.dependsOn(java.lang.Object…​) and ConfigurableFileCollection.builtBy(java.lang.Object...) work with TaskProvider in the same way as Task, so you do not need to unwrap a Provider for explicit dependencies to continue to work.

If you are configuring a task by name, you will need to use the new API equivalent. See the table below to identify the new API alternative.

How to get an instance of a Task?

In the event you still need to get access to a Task instance, you can use TaskCollection.named(java.lang.String) and call Provider.get(). This will cause the task to be created/configured, but everything should work as it has with the eager APIs.

Please contact us if you find tricky build logic that requires you to get a Task instance.

How do I know if it’s working?

We provide a couple of internal flags to print out information after a build. This will eventually be captured and displayed in an easier to digest form.

gradle -Dorg.gradle.internal.tasks.stats help
Task counts: Old API 188, New API 587, total 775 # 1
Task counts: created 613, avoided 162, %-lazy 21 # 2

Task types that were created with the old API # 3
class org.gradle.api.DefaultTask 31
class org.jetbrains.kotlin.gradle.tasks.KotlinCompile 28
class org.gradle.api.tasks.JavaExec 24
class org.jlleitschuh.gradle.ktlint.KtlintCheck 24
class org.gradle.api.tasks.bundling.Jar 16
class org.gradle.kotlin.dsl.accessors.tasks.PrintAccessors 16
class org.gradle.plugin.devel.tasks.ValidateTaskProperties 15
class org.gradle.another.CoolTask 13
class org.gradle.plugin.devel.tasks.GeneratePluginDescriptors 10
class org.gradle.plugin.devel.tasks.PluginUnderTestMetadata 10
class org.gradle.kotlin.dsl.accessors.tasks.UpdateProjectSchema 1

Task types that were registered with the new API but were created anyways # 4
class org.gradle.api.DefaultTask 142
class org.gradle.api.tasks.Delete 106
class org.gradle.api.tasks.compile.JavaCompile 32
class org.gradle.language.jvm.tasks.ProcessResources 32
class org.gradle.api.tasks.testing.Test 16
class org.gradle.api.tasks.javadoc.Javadoc 16
class org.gradle.plugins.ide.idea.GenerateIdeaModule 15
class org.gradle.plugins.ide.eclipse.GenerateEclipseClasspath 15
class org.gradle.plugins.ide.eclipse.GenerateEclipseJdt 15
class org.gradle.plugins.ide.eclipse.GenerateEclipseProject 15
class org.gradle.api.tasks.compile.GroovyCompile 10
class org.gradle.api.tasks.javadoc.Groovydoc 5
class org.gradle.api.plugins.quality.Checkstyle 4
class org.gradle.api.plugins.quality.CodeNarc 2
  1. 188 tasks are created with the old API and 587 tasks are registered with the new API for a total of 775 tasks.

  2. 613 tasks were created and configured and only 162 tasks were avoided (never created or configured). 21% of all tasks were avoided (higher is better).

  3. Lists of the type of tasks that were created with the old API. This is a good list to work down to increase the amount of possible avoidable task configuration.

  4. Lists of the type of tasks that were created with the new API but were created/configured anyways. This is a good list to work down to increase the amount of task configuration that is avoided.

These statistics are printed out once per build. Projects with buildSrc and composite builds will display this information multiple times. In a build that uses the new APIs perfectly, we should see 0 tasks created with the old API and only 1 created/configured task because we are only executing the help task. If you run other tasks (like build), you should expect many more tasks to be created and configured.

You can use the list of task types to guide which tasks would provide the biggest bang for your buck when you migrate to the new API.

To approximate the time it takes to configure a build without executing tasks, you can run gradle help. Please use the Gradle Profiler to measure your build as described further on.

Migration Guide

The following sections will go through some general guidelines to adhere when migrating the build logic as well as the steps we recommend following. We also cover some troubleshooting and pitfalls to help you work around some issues you may encountered during the migration.

General

  1. Use help task as a benchmark during the migration. The help task is the perfect candidate to benchmark your migration process. In a completely lazy build, a build scan should show no task created immediately or created during configuration. Only a single task, the help task, should be created during task graph calculation. Be mindful of the pitfall around the lazy API usage in the build scan plugin.

  2. Only mutate the current task inside configuration action. Because the task configuration action can now run immediately, later or never, mutating anything other than the current task will expose flaky-like behavior in your build. Consider the following code:

    def check = tasks.register("check")
    tasks.register("verificationTask") { verificationTask ->
        // Configure verificationTask
    
        // Run verificationTask when someone runs check
        check.get().dependsOn verificationTask
    }
    val check = tasks by registering
    tasks.register("verificationTask") {
        // Configure verificationTask
    
        // Run verificationTask when someone runs check
        check.get().dependsOn(this)
    }

    Executing the gradle check task should execute verificationTask, but with this example, it won’t. This is because the dependency between verificationTask and check only happens when verificationTask is realized. To avoid issues like this, you must only modify the task associated with the configuration action. Other tasks should be modified in their own configuration action outside of any other possibly "lazy" configuration actions. The code would become:

    def check = tasks.register("check")
    def verificationTask = tasks.register("verificationTask") {
        // Configure verificationTask
    }
    check.configure {
        dependsOn verificationTask
    }
    val check = tasks by registering
    val verificationTask = tasks by registering {
        // Configure verificationTask
    }
    check {
        dependsOn(verificationTask)
    }

    In the future, Gradle will consider these flaky-mutations as failures. It also means the configuration needs to flow from the extensions to the tasks. Thus, the task should be configured by the extension and not the other way around.

  3. Prefer small incremental changes. Smaller changes are easier to sanity check. If you ever break your build logic, it will be easier to analyze the changelog since the last successful verification.

  4. Ensure a good plan is established for validating the build logic. Usually, a simple build task invocation should do the trick to validate all corner of your build logic. However, if all automation aspect is centralized around Gradle, you may need to validate other task execution path.

  5. Prefer automatic testing to manual testing. It’s good practice to write integration test for your build logic using TestKit.

  6. Avoid referencing a task by name. In the majority of the case, referencing a task by name is a fragile pattern and should be avoided. Although task name is available on the TaskProvider, effort should be made to use references from a strongly typed model instead.

  7. Use the new task API as much as possible. The lazy task API is an opt-in feature. Eagerly realizing some tasks may cause a cascade of other tasks to be realized. Using TaskProvider helps create an indirection that protects transitive realization.

  8. Some APIs may be disallowed if you try to access them from the new API’s configuration blocks. For example, Project.afterEvaluate() cannot be called when configuring a task registered with the new API. Since afterEvaluate is used to delay configuring a Project, mixing delayed configuration with the new API can cause hard to track down configuration errors because tasks registered with the new API are not always configured, but an afterEvaluate block may be expected to always execute.

Migration Steps

The first part of the migration process is to go through the code and manually migrate eager task creation and configuration to use lazy APIs. The following explores the recommended steps for a successful migration. While going through those steps, keep in mind the guidelines.

Using the new API in a plugin will require users to use Gradle 4.9 or newer. Plugin authors should refer to Supporting older versions of Gradle section.

  1. Migrate task configuration that affects all tasks (tasks.all {}) or subsets by type (tasks.withType(…​) {}). This will cause your build to create fewer built-in Gradle task types. Note that despite some built-in Gradle task aren’t using the lazy APIs, the configuration affecting those task type should still be migrated. We recommend using regex code search capability offered by modern IDE. For example, the Intellij regex \.all\s*((\(?\s*\{)|\(\s*new) will identify every call to .all of a DomainObjectCollection and subtype. We will need to go through those result and identify and migrate only the method calls affecting the TaskContainer or TaskCollection. Be aware of the common pitfall around deferred configuration.

  2. Migrate tasks configured by name. Just like the previous point, it will cause your build to create fewer built-in Gradle tasks. For example, logic that uses TaskContainer#getByName(String, Closure/Action) would be converted to TaskContainer#named(String).configure(Closure/Action). It also include task configuration as DSL blocks. Be aware of the common pitfall around deferred configuration.

  3. Migrate tasks creation to register(…​). It will cause your build to create fewer tasks in general. Be aware of the common pitfall around deferred configuration.

At this point, we should see a decent improvement regarding build configuration. We recommend profiling your build and note the improvement. Under some circumstance, you may notice no improvement. It’s unfortunate, and we feel your frustration. Keep reading onto the next section.

Troubleshooting

  • The build became unstable after migrating to lazy task configuration. It’s a common issue that happens during the migration. Follow these next steps to find out the issue:

    1. Is the build succeed when all tasks are eagerly realized on creation using the command line flag (-Dorg.gradle.internal.tasks.eager=true)? Verify that each task configuration only mutating the task.

    2. Was there any build logic accidentally deleted? Verify your change log for disparity in the configuration that may have been deleted.

    3. It may be a real issue. For example, the issue of FileCollection using TaskProvider was a real discrepancy found while migrating a build to lazy task API. As a rule of thumb, a TaskProvider should be accepted everywhere a Task instance is accepted.

  • Where a task was realized? As we keep developing the feature, more reporting, and troubleshooting information will be made available to answer this question. In the meantime, build scan is the best way to answer this question. Follow these steps:

    1. Create a build scan. Execute the Gradle command using the --scan flag.

    2. Navigate to the configuration performance tab.

      taskConfigurationAvoidance navigate to performance
      Figure 1. Navigate to configuration performance tab in build scan
      1. Navigate to the performance card from the left side menu.

      2. Navigate to the configuration tab from the top of the performance card.

    3. All the information requires will be presented.

      taskConfigurationAvoidance performance annotated
      Figure 2. Configuration performance tab in build scan annotated
      1. Total tasks present when each task are created or not.

        • Created immediately represent all the task created using the eager task APIs.

        • Created during configuration represent all the tasks that were created using the lazy task APIs, but was realized explicitly (via TaskProvider#get()) or implicitly using the eager task query APIs.

        • Both "Created immediately" and "Created during configuration" numbers are considered the "bad" numbers that should be minimized as much as possible.

        • Created during task graph calculation number represent all the task created while building the execution task graph. Typically, this number should be equal to the number of tasks that is executed.

        • Not created number represents all the tasks that were avoided in this build session.

      2. The next section helps answer the question of where a task was realized. For each scripts, plugins and lifecycle callbacks, the last column represents the tasks that were created either immediately or during configuration. Ideally, this column should be empty. The migration should be focusing on the build logic under your control.

      3. Extending any scripts, plugins, or lifecycle callbacks subsection shows a break down in which project it was applied.

  • The build performance haven’t improved after migrating to lazy API. It is important to note that lazy task configuration is an opt-in feature. It means the entire configuration chain needs to be lazy to get the benefit. If at any point, an eager API is used for a specific named task, task type or, the worst, the entire task container, all benefits are mitigated. The first step is to have a look at the statistics to see how much more the build can be improved. If the statistic shows the build is completely lazy (work-avoided) please contact us so we can discuss where this feature falls short and improve it for the next Gradle version.

Pitfalls

  • Lazy task configuration is an opt-in feature. Any eager realization of a task will mitigate the benefit of this new API. In the far future, eager realizations will become an error, until them, it’s up to the build author to avoid eager realization.

  • Beware of the hidden eager task realization. Gradle has lots of ways to configure a task eagerly. TaskContainer#getByName(String, Action) is an explicit configuration by name. Configuring a task as a DSL block is an alias to the previous explicit configuration, meaning a lazy task will be realized:

    // Given a task lazily created with
    tasks.register("someTask")
    
    // Some time later, the task is configured using a DSL block like
    someTask {
        // The task is realized immediately, and this closure is executed immediately
    }

    You will have to migrate the DSL task configuration to the following:

    tasks.named("someTask").configure {
        // ...
        // Beware of the pitfalls here
    }

    While migrating to the lazy task configuration, you may still have eager task configuration that references a task by name using Groovy’s property to task reference syntactic sugar. If this lookup is done from anything other than a lazy configuration action, the lazy task reference by the name will be prematurely realized:

    tasks.register("someTask")
    
    // Sometime later, an eager task is configured like
    task anEagerTask {
        // The task is realized immediately as it will be treated as a property of the project and will result in an eager task lookup by name
        dependsOn someTask
    }

    You can solve this in three ways:

    • Use TaskProvider variable. Useful when the task is referenced multiple times in the same build script.

      def someTask = tasks.register("someTask")
      
      task anEagerTask {
          dependsOn someTask
      }
      val someTask = tasks by registering
      
      task("anEagerTask") {
          dependsOn(someTask)
      }
    • Migrate the consumer task to the new API.

      tasks.register("someTask")
      
      tasks.register("anEagerTask") {
          dependsOn someTask
      }
    • Lookup the task lazily. Useful when the tasks are not created by the same plugin.

      tasks.register("someTask")
      
      task anEagerTask {
          dependsOn tasks.named("someTask")
      }
      tasks.register("someTask")
      
      task("anEagerTask") {
          dependsOn(tasks.named("someTask"))
      }
  • The build scan plugin buildScanPublishPrevious task is eager until version 1.15. Upgrade the build scan plugin in your build to use the latest version.

Supporting older versions of Gradle

This section describes two ways to keep your plugin backward compatible with older version of Gradle if you must maintain compatibility with versions of Gradle older than 4.9. Most of the new API methods are available from Gradle 4.9 upwards.

Although backward compatibility is good for users, we still recommended to upgrade to newer Gradle releases in a timely manner. This will reduce your maintenance burden.

The first method to maintain compatibility is to compile your plugin against the Gradle 4.9 API and conditionally call the right APIs with Groovy (example).

The second method is to use Java reflection to cope with the fact that the APIs are unavailable during compilation (example).

It is highly recommended to have cross-version test coverage using TestKit and multiple versions of Gradle.

Existing vs New API overview

  • Methods that take a groovy.lang.Closure are covered in the new API with methods taking org.gradle.api.Action.

  • More convenience methods may be added in the future based on user feedback.

  • Some old API methods may never have a direct replacement in the new API.

  • Some APIs may be restricted when accessed in a lazy configuration block.

Old vs New API Description

Instead of: task myTask(type: MyTask) {}

There is not a shorthand Groovy DSL for using the new API.

Use: tasks.register("myTask", MyTask) {}

Use one of the alternatives below.

Use: No direct equivalent.

Use one of the alternatives below.

Use: No direct equivalent.

This returns a TaskProvider instead of a Task.

This returns a TaskProvider instead of a Task.

This returns a TaskProvider instead of a Task.

This returns a TaskProvider instead of a Task.

This returns a TaskProvider instead of a Task.

This returns a TaskProvider instead of a Task.

This returns a TaskProvider instead of a Task.

Use: named(java.lang.String).configure(Action)

Accessing tasks from another project requires a specific ordering of project evaluation.

Use: No direct equivalent.

named(String) is the closest equivalent, but will fail if the task does not exist. Using findByName(String) will cause tasks registered with the new API to be created/configured.

Use: No direct equivalent.

See getByPath(String) above.

Use: No direct equivalent.

This is OK to use because it does not require tasks to be created immediately.

Use: OK

Instead of: withType(java.lang.Class).getByName(java.lang.String)

This returns a TaskProvider instead of a Task.

Use: named(java.lang.String, java.lang.Class)

This returns void, so it cannot be chained.

Use: withType(java.lang.Class).configureEach(org.gradle.api.Action)

This returns void, so it cannot be chained.

This returns void, so it cannot be chained.

This returns void, so it cannot be chained.

Avoid calling this method. matching(Spec) and configureEach(Action) are more appropriate in most cases.

Use: OK, with issues.

This is OK to use because it does not require tasks to be created immediately.

Use: OK

Avoid calling this directly as it’s a Groovy convenience method. The alternative returns a TaskProvider instead of a Task.

Instead of: iterator() or implicit iteration over the Task collection

Avoid doing this as it requires creating and configuring all tasks. See findAll(Closure) above.

Use: OK, with issues.

Instead of: remove(org.gradle.api.Task)

Avoid calling this. The behavior of remove with the new API may change in the future.

Use: OK, with issues.

Avoid calling this. The behavior of replace with the new API may change in the future.

Use: OK, with issues.

Avoid calling this. The behavior of replace with the new API may change in the future.

Use: OK, with issues.

Profiling with a Gradle Profiler Scenario File

The Gradle Profiler is a tool to measure build times for Gradle builds in a predictable and reproducible manner. The tool automates collecting profiling and benchmark information from a Gradle build and mitigates environmental impacts to measuring build time (like JIT warmups and cached dependencies). Clone and build gradle-profiler locally.

To measure the impact of the new API on your build, we’ve included a sample scenario file you can use. This scenario runs gradle help on your build with a special flag to enable/disable the new API to make it easier to measure improvements. gradle help approximates the time it takes Gradle to configure your build by running only a single, simple task.

Save as help.scenario
defaults {
    tasks = ["help"]
    warm-ups = 20
}
eagerHelp = ${defaults} {
    gradle-args = ["-Dorg.gradle.internal.tasks.eager=true"]
}
lazyHelp = ${defaults} {
    gradle-args = ["-Dorg.gradle.internal.tasks.eager=false"]
}

Run gradle-profiler in the root of your build. The results will go into a file called profile-out-N where N is unique for each invocation.

  • When measuring your build with gradle-profiler, you should make sure the machine running the benchmark is not also busy doing other things. You may get false positives/negatives if resources are spent doing other things.

  • Get a baseline for how long your build takes before making any changes. Run

gradle-profiler --benchmark
    --iterations 20
    --gradle-version [some Gradle version]
    --scenario-file help.scenario
    eagerHelp lazyHelp
  • In the profile-out-N directory, gradle-profiler will generate a CSV and a HTML file to display the results of the benchmarking.

  • After making some changes to decrease the number of tasks that are created and configured, re-run the command above.

  • For the Gradle build itself, we saw improvements after 50% of the tasks were no longer configured each time. Your mileage may vary depending on how expensive particular tasks are to create and configure.

  • Please provide feedback on this issue. Or send us an email at performance@gradle.com.