Chapter 20. Working With Files

Table of Contents

20.1. Locating files
20.2. File collections
20.3. File trees
20.4. Using the contents of an archive as a file tree
20.5. Specifying a set of input files
20.6. Copying files
20.7. Using the Sync task
20.8. Creating archives
20.9. Properties files

Most builds work with files. Gradle adds some concepts and APIs to help you achieve this.

20.1. Locating files

You can locate a file relative to the project directory using the Project.file(java.lang.Object) method.

Example 20.1. Locating files

build.gradle

// Using a relative path
File configFile = file('src/config.xml')

// Using an absolute path
configFile = file(configFile.absolutePath)

// Using a File object with a relative path
configFile = file(new File('src/config.xml'))

You can pass any object to the file() method, and it will attempt to convert the value to an absolute File object. Usually, you would pass it a String or File instance. If this path is an absolute path, it is used to construct a File instance. Otherwise, a File instance is constructed by prepending the project directory path to the supplied path. The file() method also understands URLs, such as file:/some/path.xml.

Using this method is a useful way to convert some user provided value into an absolute File. It is preferable to using new File(somePath), as file() always evaluates the supplied path relative to the project directory, which is fixed, rather than the current working directory, which can change depending on how the user runs Gradle.

20.2. File collections

A file collection is simply a set of files. It is represented by the FileCollection interface. Many objects in the Gradle API implement this interface. For example, dependency configurations implement FileCollection.

One way to obtain a FileCollection instance is to use the Project.files(java.lang.Object[]) method. You can pass this method any number of objects, which are then converted into a set of File objects. The files() method accepts any type of object as its parameters. These are evaluated relative to the project directory, as per the file() method, described in Section 20.1, “Locating files”. You can also pass collections, iterables, maps and arrays to the files() method. These are flattened and the contents converted to File instances.

Example 20.2. Creating a file collection

build.gradle

FileCollection collection = files('src/file1.txt',
                                  new File('src/file2.txt'),
                                  ['src/file3.txt', 'src/file4.txt'])

A file collection is iterable, and can be converted to a number of other types using the as operator. You can also add 2 file collections together using the + operator, or subtract one file collection from another using the - operator. Here are some examples of what you can do with a file collection.

Example 20.3. Using a file collection

build.gradle

// Iterate over the files in the collection
collection.each { File file ->
    println file.name
}

// Convert the collection to various types
Set set = collection.files
Set set2 = collection as Set
List list = collection as List
String path = collection.asPath
File file = collection.singleFile
File file2 = collection as File

// Add and subtract collections
def union = collection + files('src/file3.txt')
def different = collection - files('src/file3.txt')


You can also pass the files() method a closure or a Callable instance. This is called when the contents of the collection are queried, and its return value is converted to a set of File instances. The return value can be an object of any of the types supported by the files() method. This is a simple way to 'implement' the FileCollection interface.

Example 20.4. Implementing a file collection

build.gradle

task list {
    doLast {
        File srcDir

        // Create a file collection using a closure
        collection = files { srcDir.listFiles() }

        srcDir = file('src')
        println "Contents of $srcDir.name"
        collection.collect { relativePath(it) }.sort().each { println it }

        srcDir = file('src2')
        println "Contents of $srcDir.name"
        collection.collect { relativePath(it) }.sort().each { println it }
    }
}

Output of gradle -q list

> gradle -q list
Contents of src
src/dir1
src/file1.txt
Contents of src2
src2/dir1
src2/dir2

Some other types of things you can pass to files():

FileCollection

These are flattened and the contents included in the file collection.

Task

The output files of the task are included in the file collection.

TaskOutputs

The output files of the TaskOutputs are included in the file collection.

It is important to note that the content of a file collection is evaluated lazily, when it is needed. This means you can, for example, create a FileCollection that represents files which will be created in the future by, say, some task.

20.3. File trees

A file tree is a collection of files arranged in a hierarchy. For example, a file tree might represent a directory tree or the contents of a ZIP file. It is represented by the FileTree interface. The FileTree interface extends FileCollection, so you can treat a file tree exactly the same way as you would a file collection. Several objects in Gradle implement the FileTree interface, such as source sets.

One way to obtain a FileTree instance is to use the Project.fileTree(java.util.Map) method. This creates a FileTree defined with a base directory, and optionally some Ant-style include and exclude patterns.

Example 20.5. Creating a file tree

build.gradle

// Create a file tree with a base directory
FileTree tree = fileTree(dir: 'src/main')

// Add include and exclude patterns to the tree
tree.include '**/*.java'
tree.exclude '**/Abstract*'

// Create a tree using path
tree = fileTree('src').include('**/*.java')

// Create a tree using closure
tree = fileTree('src') {
    include '**/*.java'
}

// Create a tree using a map
tree = fileTree(dir: 'src', include: '**/*.java')
tree = fileTree(dir: 'src', includes: ['**/*.java', '**/*.xml'])
tree = fileTree(dir: 'src', include: '**/*.java', exclude: '**/*test*/**')

You use a file tree in the same way you use a file collection. You can also visit the contents of the tree, and select a sub-tree using Ant-style patterns:

Example 20.6. Using a file tree

build.gradle

// Iterate over the contents of a tree
tree.each {File file ->
    println file
}

// Filter a tree
FileTree filtered = tree.matching {
    include 'org/gradle/api/**'
}

// Add trees together
FileTree sum = tree + fileTree(dir: 'src/test')

// Visit the elements of the tree
tree.visit {element ->
    println "$element.relativePath => $element.file"
}

20.4. Using the contents of an archive as a file tree

You can use the contents of an archive, such as a ZIP or TAR file, as a file tree. You do this using the Project.zipTree(java.lang.Object) and Project.tarTree(java.lang.Object) methods. These methods return a FileTree instance which you can use like any other file tree or file collection. For example, you can use it to expand the archive by copying the contents, or to merge some archives into another.

Example 20.7. Using an archive as a file tree

build.gradle

// Create a ZIP file tree using path
FileTree zip = zipTree('someFile.zip')

// Create a TAR file tree using path
FileTree tar = tarTree('someFile.tar')

//tar tree attempts to guess the compression based on the file extension
//however if you must specify the compression explicitly you can:
FileTree someTar = tarTree(resources.gzip('someTar.ext'))


20.5. Specifying a set of input files

Many objects in Gradle have properties which accept a set of input files. For example, the JavaCompile task has a source property, which defines the source files to compile. You can set the value of this property using any of the types supported by the files() method, which was shown above. This means you can set the property using, for example, a File, String, collection, FileCollection or even a closure. Here are some examples:

Usually, there is a method with the same name as the property, which appends to the set of files. Again, this method accepts any of the types supported by the files() method.

Example 20.8. Specifying a set of files

build.gradle

task compile(type: JavaCompile)

// Use a File object to specify the source directory
compile {
    source = file('src/main/java')
}

// Use a String path to specify the source directory
compile {
    source = 'src/main/java'
}

// Use a collection to specify multiple source directories
compile {
    source = ['src/main/java', '../shared/java']
}

// Use a FileCollection (or FileTree in this case) to specify the source files
compile {
    source = fileTree(dir: 'src/main/java').matching { include 'org/gradle/api/**' }
}

// Using a closure to specify the source files.
compile {
    source = {
        // Use the contents of each zip file in the src dir
        file('src').listFiles().findAll {it.name.endsWith('.zip')}.collect { zipTree(it) }
    }
}

build.gradle

compile {
    // Add some source directories use String paths
    source 'src/main/java', 'src/main/groovy'

    // Add a source directory using a File object
    source file('../shared/java')

    // Add some source directories using a closure
    source { file('src/test/').listFiles() }
}

20.6. Copying files

You can use the Copy task to copy files. The copy task is very flexible, and allows you to, for example, filter the contents of the files as they are copied, and map to the file names.

To use the Copy task, you must provide a set of source files to copy, and a destination directory to copy the files to. You may also specify how to transform the files as they are copied. You do all this using a copy spec. A copy spec is represented by the CopySpec interface. The Copy task implements this interface. You specify the source files using the CopySpec.from(java.lang.Object[]) method. To specify the destination directory, use the CopySpec.into(java.lang.Object) method.

Example 20.9. Copying files using the copy task

build.gradle

task copyTask(type: Copy) {
    from 'src/main/webapp'
    into 'build/explodedWar'
}

The from() method accepts any of the arguments that the files() method does. When an argument resolves to a directory, everything under that directory (but not the directory itself) is recursively copied into the destination directory. When an argument resolves to a file, that file is copied into the destination directory. When an argument resolves to a non-existing file, that argument is ignored. If the argument is a task, the output files (i.e. the files the task creates) of the task are copied and the task is automatically added as a dependency of the Copy task. The into() accepts any of the arguments that the file() method does. Here is another example:

Example 20.10. Specifying copy task source files and destination directory

build.gradle

task anotherCopyTask(type: Copy) {
    // Copy everything under src/main/webapp
    from 'src/main/webapp'
    // Copy a single file
    from 'src/staging/index.html'
    // Copy the output of a task
    from copyTask
    // Copy the output of a task using Task outputs explicitly.
    from copyTaskWithPatterns.outputs
    // Copy the contents of a Zip file
    from zipTree('src/main/assets.zip')
    // Determine the destination directory later
    into { getDestDir() }
}

You can select the files to copy using Ant-style include or exclude patterns, or using a closure:

Example 20.11. Selecting the files to copy

build.gradle

task copyTaskWithPatterns(type: Copy) {
    from 'src/main/webapp'
    into 'build/explodedWar'
    include '**/*.html'
    include '**/*.jsp'
    exclude { details -> details.file.name.endsWith('.html') &&
                         details.file.text.contains('staging') }
}

You can also use the Project.copy(org.gradle.api.Action) method to copy files. It works the same way as the task with some major limitations though. First, the copy() is not incremental (see Section 19.9, “Up-to-date checks (AKA Incremental Build)”).

Example 20.12. Copying files using the copy() method without up-to-date check

build.gradle

task copyMethod {
    doLast {
        copy {
            from 'src/main/webapp'
            into 'build/explodedWar'
            include '**/*.html'
            include '**/*.jsp'
        }
    }
}

Secondly, the copy() method can not honor task dependencies when a task is used as a copy source (i.e. as an argument to from()) because it's a method and not a task. As such, if you are using the copy() method as part of a task action, you must explicitly declare all inputs and outputs in order to get the correct behavior.

Example 20.13. Copying files using the copy() method with up-to-date check

build.gradle

task copyMethodWithExplicitDependencies{
    // up-to-date check for inputs, plus add copyTask as dependency
    inputs.file copyTask
    outputs.dir 'some-dir' // up-to-date check for outputs
    doLast{
        copy {
            // Copy the output of copyTask
            from copyTask
            into 'some-dir'
        }
    }
}

It is preferable to use the Copy task wherever possible, as it supports incremental building and task dependency inference without any extra effort on your part. The copy() method can be used to copy files as part of a task's implementation. That is, the copy method is intended to be used by custom tasks (see Chapter 40, Writing Custom Task Classes) that need to copy files as part of their function. In such a scenario, the custom task should sufficiently declare the inputs/outputs relevant to the copy action.

20.6.1. Renaming files

Example 20.14. Renaming files as they are copied

build.gradle

task rename(type: Copy) {
    from 'src/main/webapp'
    into 'build/explodedWar'
    // Use a closure to map the file name
    rename { String fileName ->
        fileName.replace('-staging-', '')
    }
    // Use a regular expression to map the file name
    rename '(.+)-staging-(.+)', '$1$2'
    rename(/(.+)-staging-(.+)/, '$1$2')
}

20.6.2. Filtering files

Example 20.15. Filtering files as they are copied

build.gradle

import org.apache.tools.ant.filters.FixCrLfFilter
import org.apache.tools.ant.filters.ReplaceTokens

task filter(type: Copy) {
    from 'src/main/webapp'
    into 'build/explodedWar'
    // Substitute property tokens in files
    expand(copyright: '2009', version: '2.3.1')
    expand(project.properties)
    // Use some of the filters provided by Ant
    filter(FixCrLfFilter)
    filter(ReplaceTokens, tokens: [copyright: '2009', version: '2.3.1'])
    // Use a closure to filter each line
    filter { String line ->
        "[$line]"
    }
    // Use a closure to remove lines
    filter { String line ->
        line.startsWith('-') ? null : line
    }
    filteringCharset = 'UTF-8'
}

When you use the ReplaceTokens class with the “filter” operation, the result is a template engine that replaces tokens of the form “@tokenName@” (the Apache Ant-style token) with a set of given values. The “expand” operation does the same thing except it treats the source files as Groovy templates in which tokens take the form “${tokenName}”. Be aware that you may need to escape parts of your source files when using this option, for example if it contains literal “$” or “<%” strings.

It's a good practice to specify the charset when reading and writing the file, using the filteringCharset property. If not specified, the JVM default charset is used, which might not match with the actual charset of the files to filter, and might be different from one machine to another.

20.6.3. Using the CopySpec class

Copy specs form a hierarchy. A copy spec inherits its destination path, include patterns, exclude patterns, copy actions, name mappings and filters.

Example 20.16. Nested copy specs

build.gradle

task nestedSpecs(type: Copy) {
    into 'build/explodedWar'
    exclude '**/*staging*'
    from('src/dist') {
        include '**/*.html'
    }
    into('libs') {
        from configurations.runtime
    }
}

20.7. Using the Sync task

The Sync task extends the Copy task. When it executes, it copies the source files into the destination directory, and then removes any files from the destination directory which it did not copy. This can be useful for doing things such as installing your application, creating an exploded copy of your archives, or maintaining a copy of the project's dependencies.

Here is an example which maintains a copy of the project's runtime dependencies in the build/libs directory.

Example 20.17. Using the Sync task to copy dependencies

build.gradle

task libs(type: Sync) {
    from configurations.runtime
    into "$buildDir/libs"
}

20.8. Creating archives

A project can have as many JAR archives as you want. You can also add WAR, ZIP and TAR archives to your project. Archives are created using the various archive tasks: Zip, Tar, Jar, War, and Ear. They all work the same way, so let's look at how you create a ZIP file.

Example 20.18. Creating a ZIP archive

build.gradle

apply plugin: 'java'

task zip(type: Zip) {
    from 'src/dist'
    into('libs') {
        from configurations.runtime
    }
}

Why are you using the Java plugin?

The Java plugin adds a number of default values for the archive tasks. You can use the archive tasks without using the Java plugin, if you like. You will need to provide values for some additional properties.

The archive tasks all work exactly the same way as the Copy task, and implement the same CopySpec interface. As with the Copy task, you specify the input files using the from() method, and can optionally specify where they end up in the archive using the into() method. You can filter the contents of file, rename files, and all the other things you can do with a copy spec.

20.8.1. Archive naming

The format of projectName-version.type is used for generated archive file names. For example:

Example 20.19. Creation of ZIP archive

build.gradle

apply plugin: 'java'

version = 1.0

task myZip(type: Zip) {
    from 'somedir'
}

println myZip.archiveName
println relativePath(myZip.destinationDir)
println relativePath(myZip.archivePath)

Output of gradle -q myZip

> gradle -q myZip
zipProject-1.0.zip
build/distributions
build/distributions/zipProject-1.0.zip

This adds a Zip archive task with the name myZip which produces ZIP file zipProject-1.0.zip. It is important to distinguish between the name of the archive task and the name of the archive generated by the archive task. The default name for archives can be changed with the archivesBaseName project property. The name of the archive can also be changed at any time later on.

There are a number of properties which you can set on an archive task. These are listed below in Table 20.1, “Archive tasks - naming properties”. You can, for example, change the name of the archive:

Example 20.20. Configuration of archive task - custom archive name

build.gradle

apply plugin: 'java'
version = 1.0

task myZip(type: Zip) {
    from 'somedir'
    baseName = 'customName'
}

println myZip.archiveName

Output of gradle -q myZip

> gradle -q myZip
customName-1.0.zip

You can further customize the archive names:

Example 20.21. Configuration of archive task - appendix & classifier

build.gradle

apply plugin: 'java'
archivesBaseName = 'gradle'
version = 1.0

task myZip(type: Zip) {
    appendix = 'wrapper'
    classifier = 'src'
    from 'somedir'
}

println myZip.archiveName

Output of gradle -q myZip

> gradle -q myZip
gradle-wrapper-1.0-src.zip

Table 20.1. Archive tasks - naming properties

Property name Type Default value Description
archiveName String baseName-appendix-version-classifier.extension

If any of these properties is empty the trailing - is not added to the name.

The base file name of the generated archive
archivePath File destinationDir/archiveName The absolute path of the generated archive.
destinationDir File Depends on the archive type. JARs and WARs go into project.buildDir/libraries. ZIPs and TARs go into project.buildDir/distributions. The directory to generate the archive into
baseName String project.name The base name portion of the archive file name.
appendix String null The appendix portion of the archive file name.
version String project.version The version portion of the archive file name.
classifier String null The classifier portion of the archive file name,
extension String Depends on the archive type, and for TAR files, the compression type as well: zip, jar, war, tar, tgz or tbz2. The extension of the archive file name.

20.8.2. Sharing content between multiple archives

You can use the Project.copySpec(org.gradle.api.Action) method to share content between archives.

Often you will want to publish an archive, so that it is usable from another project. This process is described in Chapter 32, Publishing artifacts

20.9. Properties files

Properties files are used in many places during Java development. Gradle makes it easy to create properties files as a normal part of the build. You can use the WriteProperties task to create properties files.

The WriteProperties task also fixes a well-known problem with Properties.store() that can reduce the usefulness of incremental builds (see Section 19.9, “Up-to-date checks (AKA Incremental Build)”). The standard Java way to write a properties file produces a unique file every time, even when the same properties and values are used, because it includes a timestamp in the comments. Gradle's WriteProperties task generates exactly the same output byte-for-byte if none of the properties have changed. This is achieved by a few tweaks to how a properties file is generated:

  • no timestamp comment is added to the output
  • the line separator is system independent, but can be configured explicitly (it defaults to '\n')
  • the properties are sorted alphabetically