Dependency Caching
Gradle contains a highly sophisticated dependency caching mechanism, which seeks to minimise the number of remote requests made in dependency resolution, while striving to guarantee that the results of dependency resolution are correct and reproducible.
-
Local Cache: Gradle caches dependencies locally to avoid repeated downloads. The cache is located in the
.gradledirectory under the user’s home folder (e.g.,~/.gradle/caches/modules-2). When a dependency is requested, Gradle first checks this local cache before attempting to fetch it from remote repositories. -
Changing Dependencies: By default, Gradle treats dependencies marked as "changing" (e.g., SNAPSHOT or dynamic dependencies) differently and refreshes them more frequently. The caching times for these dependencies can be altered programmatically.
-
Offline Mode: Gradle can run in offline mode, using only the cached dependencies without trying to download anything from remote repositories. You can enable offline mode with the
--offlineflag, ensuring that your build only uses cached artifacts. -
Refreshing Dependencies: To force Gradle to update its dependencies, use the
--refresh-dependenciesflag. This option instructs Gradle to bypass the cache and check for updated artifacts in remote repositories. Gradle downloads them, but only if it detects a change, using hashes to avoid unnecessary downloads.
1. The dependency cache
The Gradle dependency cache consists of two storage types located under $GRADLE_USER_HOME/caches:
-
A file-based store of downloaded artifacts, including binaries like jars as well as raw downloaded meta-data like POM files and Ivy files. Artifacts are stored under a checksum, so name clashes will not cause issues.
-
A binary store of resolved module metadata, including the results of resolving dynamic versions, module descriptors, and artifacts.
Separate metadata cache
Gradle keeps a record of various aspects of dependency resolution in binary format in the metadata cache.
The information stored in the metadata cache includes:
-
The result of resolving a dynamic version (e.g.
1.+) to a concrete version (e.g.1.2). -
The resolved module metadata for a particular module, including module artifacts and module dependencies.
-
The resolved artifact metadata for a particular artifact, including a pointer to the downloaded artifact file.
-
The absence of a particular module or artifact in a particular repository, eliminating repeated attempts to access a resource that does not exist.
Every entry in the metadata cache includes a record of the repository that provided the information as well as a timestamp that can be used for cache expiry.
Repository caches are independent
As described above, for each repository there is a separate metadata cache. A repository is identified by its URL, type and layout.
If a module or artifact has not been previously resolved from this repository, Gradle will attempt to resolve the module against the repository. This will always involve a remote lookup on the repository, however in many cases no download will be required.
Dependency resolution will fail if required artifacts aren’t available in the repository from which they were originally resolved. Once resolved from a specific repository, artifacts become "sticky," meaning Gradle will avoid resolving them from other repositories to prevent unexpected or potentially unsafe changes in artifact sources. This ensures consistency across environments, but it may also lead to failures if repositories differ between machines.
Repository independence allows builds to be isolated from each other. This is a key feature to create builds that are reliable and reproducible in any environment.
Artifact reuse
Before downloading an artifact, Gradle attempts to retrieve the artifact’s checksum by downloading an associated .sha512, .sha256, .sha1, or .md5 file (attempting each in order).
If the checksum is available, Gradle skips the download if an artifact with the same ID and checksum already exists. However, if the checksum cannot be retrieved from the remote server, Gradle proceeds to download the artifact but will ignore it if it matches an existing one.
Gradle also tries to reuse artifacts from the local Maven repository. If an artifact previously downloaded by Maven is a match, Gradle will use it, provided it can be verified against the checksum from the remote server.
Checksum based storage
It is possible for different repositories to provide a different binary artifact in response to the same artifact identifier.
This is often the case with Maven SNAPSHOT artifacts, but can also be true for any artifact which is republished without changing its identifier. By caching artifacts based on their checksum, Gradle is able to maintain multiple versions of the same artifact. This means that when resolving against one repository Gradle will never overwrite the cached artifact file from a different repository. This is done without requiring a separate artifact file store per repository.
Cache locking
The Gradle dependency cache uses file-based locking to ensure that it can safely be used by multiple Gradle processes concurrently. The lock is held whenever the binary metadata store is being read or written, but is released for slow operations such as downloading remote artifacts.
This concurrent access is only supported if the different Gradle processes can communicate together. This is usually not the case for containerized builds.
Cache cleanup
Gradle tracks which artifacts in the dependency cache are accessed. Based on this information, the cache is periodically scanned (no more than once every 24 hours) to identify artifacts that haven’t been used in over 30 days. These obsolete artifacts are then deleted to prevent the cache from growing indefinitely.
You can learn more about cache cleanup in Gradle-managed Directories.
2. Changing dependencies
Gradle treats dependencies marked as "changing" (such as SNAPSHOT dependencies) differently from regular dependencies, refreshing them more frequently to ensure that you are always using the latest version.
To declare a dependency as changing, you can set the changing = true attribute in your dependency declaration.
This is useful for dependencies expected to change frequently without a new version number:
dependencies {
implementation("com.example:some-library:1.0-SNAPSHOT") // Automatically gets treated as changing
implementation("com.example:my-library:1.0") { // Must be explicitly set as changing
changing = true
}
}Caching changing dependencies
By default, Gradle caches these dependencies (including dynamic versions and changing modules) for 24 hours, meaning it does not contact remote repositories for new versions during this time.
To have Gradle check for newer versions more frequently or with every build, you can adjust the caching threshold or time-to-live (TTL) settings accordingly.
| Using a short TTL threshold for dynamic or changing versions may result in longer build times due to increased remote repository accesses. |
You can fine-tune certain aspects of caching programmatically using the ResolutionStrategy for a configuration. The programmatic approach is useful if you want to change the settings permanently.
To change how long Gradle will cache the resolved version for a dynamic version, use:
configurations.all {
resolutionStrategy.cacheDynamicVersionsFor(10, "minutes")
}configurations.all {
resolutionStrategy.cacheDynamicVersionsFor 10, 'minutes'
}To change how long Gradle will cache the metadata and artifacts for a changing module, use:
configurations.all {
resolutionStrategy.cacheChangingModulesFor(4, "hours")
}configurations.all {
resolutionStrategy.cacheChangingModulesFor 4, 'hours'
}3. Using offline mode
The --offline command-line switch instructs Gradle to use dependency modules from the cache, regardless of whether they are due to be checked again.
When running with offline, Gradle will not attempt to access the network for dependency resolution.
If the required modules are not in the dependency cache, the build will fail.
4. Force-refreshing dependencies
You can control the behavior of dependency caching for a distinct build invocation from the command line. Command line options help make a selective, ad-hoc choice for a single build execution.
At times, the Gradle Dependency Cache can become out of sync with the actual state of the configured repositories.
Perhaps a repository was initially misconfigured, or maybe a "non-changing" module was published incorrectly.
To refresh all dependencies in the dependency cache, use the --refresh-dependencies option on the command line.
The --refresh-dependencies option tells Gradle to ignore all cached entries for resolved modules and artifacts.
A fresh resolve will be performed against all configured repositories, with dynamic versions recalculated, modules refreshed, and artifacts downloaded.
However, where possible Gradle will check if the previously downloaded artifacts are valid before downloading again.
This is done by comparing published checksum values in the repository with the checksum values for existing downloaded artifacts.
Refreshing dependencies will cause Gradle to invalidate its listing caches. However:
-
it will perform HTTP HEAD requests on metadata files but will not re-download them if they are identical
-
it will perform HTTP HEAD requests on artifact files but will not re-download them if they are identical
In other words, refreshing dependencies only has an impact if you actually use dynamic dependencies or that you have changing dependencies that you were not aware of (in which case it is your responsibility to declare them correctly to Gradle as changing dependencies).
It’s a common misconception to think that using --refresh-dependencies will force the download of dependencies.
This is not the case: Gradle will only perform what is strictly required to refresh the dynamic dependencies.
This may involve downloading new listings, metadata files, or even artifacts, but the impact is minimal if nothing changed.
Dealing with ephemeral builds
It’s a common practice to run builds in ephemeral containers. A container is typically spawned to only execute a single build before it is destroyed. This can become a practical problem when a build depends on a lot of dependencies which each container has to re-download. To help with this scenario, Gradle provides a couple of options:
-
copying the dependency cache into each container
-
sharing a read-only dependency cache between multiple containers
Copying and reusing the cache
The dependency cache, both the file and metadata parts, are fully encoded using relative paths. This means that it is perfectly possible to copy a cache around and see Gradle benefit from it.
The path that can be copied is $GRADLE_USER_HOME/caches/modules-<version>.
The only constraint is placing it using the same structure at the destination, where the value of GRADLE_USER_HOME can be different.
Do not copy the *.lock or gc.properties files if they exist.
Note that creating the cache and consuming it should be done using compatible Gradle version, as shown in the table below. Otherwise, the build might still require some interactions with remote repositories to complete missing information, which might be available in a different version. If multiple incompatible Gradle versions are in play, all should be used when seeding the cache.
| Module cache version | File cache version | Metadata cache version | Gradle version(s) |
|---|---|---|---|
|
|
|
Gradle 6.1 to Gradle 6.3 |
|
|
|
Gradle 6.4 to Gradle 6.7 |
|
|
|
Gradle 6.8 to Gradle 7.4 |
|
|
|
Gradle 7.5 to Gradle 7.6.1 |
|
|
|
Gradle 7.6.2 |
|
|
|
Gradle 8.0 |
|
|
|
Gradle 8.1 |
|
|
|
Gradle 8.2 to Gradle 8.10.2 |
|
|
|
Gradle 8.11 and above |
Sharing the dependency cache with other Gradle instances
Instead of copying the dependency cache into each container, it’s possible to mount a shared, read-only directory that will act as a dependency cache for all containers. This cache, unlike the classical dependency cache, is accessed without locking, making it possible for multiple builds to read from the cache concurrently. It’s important that the read-only cache is not written to when other builds may be reading from it.
When using the shared read-only cache, Gradle looks for dependencies (artifacts or metadata) in both the writable cache in the local Gradle User Home directory and the shared read-only cache. If a dependency is present in the read-only cache, it will not be downloaded. If a dependency is missing from the read-only cache, it will be downloaded and added to the writable cache. In practice, this means that the writable cache will only contain dependencies that are unavailable in the read-only cache.
The read-only cache should be sourced from a Gradle dependency cache that already contains some of the required dependencies. The cache can be incomplete; however, an empty shared cache will only add overhead.
| The shared read-only dependency cache is an incubating feature. |
The first step in using a shared dependency cache is to create one by copying of an existing local cache. For this you need to follow the instructions above.
Then set the GRADLE_RO_DEP_CACHE environment variable to point to the directory containing the cache:
$GRADLE_RO_DEP_CACHE
|-- modules-2 : the read-only dependency cache, should be mounted with read-only privileges
$GRADLE_HOME
|-- caches
|-- modules-2 : the container specific dependency cache, should be writable
|-- ...
|-- ...
In a CI environment, it’s a good idea to have one build which "seeds" a Gradle dependency cache, which is then copied to a different directory or distributed, for example, as a Docker volume. This directory can then be used as the read-only cache for other builds. You shouldn’t use an existing Gradle installation cache as the read-only cache, because this directory may contain locks and may be modified by the seeding build.