sbt Study Note

This is a study note of the sbt build tool. It is based on the sbt official document Getting Started with sbt.

1. Getting Started

sbt stands for “simple build tool”. It provides a parallel execution engine and configuration system that run Scala build scripts. sbt is an interactive tools built upon a set of consistent concepts.

One should define the sbt version in project/build.properties file such as sbt.version=1.3.8. It is the only property for this file.

The build.sbt is used to define a build definition. A build definition consists of a set of projects (also called subproject). Each subproject is configured by a sequence of key-value pairs. Each key-value pair is called a setting expression.

In addition to build.sbt, project driectory can contain .scala files or .sbt files that defines helper objects and one-off plugins. Common file names are project/dependencies.scala and project/plugins.sbt.

Run sbt to enter the interactive mode. Prefix a command with ~ cause the command to be executed automatically re-executed whenever one of the soruce files of the current project is modified, press Enter to exit auto-run mode. Some commonly used commands:

• about: basic info about sbt and the build.
• help [command]: list commands or show description of a command.
• inspect <key>: inspect a setting expression. Options include tree, uses, definition, and actual.
• tasks: see the list of tasks of the current project
• settings: shows the current project’s settings.
• reload: reloads the current build.
• projects: list, remove/add projects for the current build.
• project: display the current project or change the current project to another project.

Common settings include scalaVersion, organization, name, version and libraryDependencies. Typing a setting name displays the current setting. For example, type scalaSource gives the source directory for Scala source code, usually it is the src/main/scala folder in the current project. The target setting shows the folder of generated files. Settings can depend on each others. Use inspect to see the default value, dependencies and settings that depend on it. For example, inspect target shows that target depends on baseDirctory setting. There are three settings testListner, crossTarget and history depending on the target setting.

sbt provides a set of predefined tasks including clean, compile, run, test etc. use inspect to see a task’s dependencies and those tasks the depend on it.

2. Build Definitions

2.1 Setting Expression

A setting expression consists of three parts: a key, a setting/task body, and an operator that associate the key and the body. A key is an instance of SettingKey[T', TaskKey[T], or InputKey[T], where T is the expected value type. A setting/task body is a Scala expression that can produce a value of the expected type.

• SettingKey[T]: a key for a value computed once when loading the subproject.
• TaskKey[T]: a key for a value computed each time it is used.
• InputKey[T]: a key for a task that has command line argument as input.

You first craete an instance of one of the three setting types, usually using a lazy value. The variable name is the key of the setting. Then change the value of the setting. There are three operators to change value of a key:

• :=: reset a new value
• +=: appends a value to the seqence in a key
• ++=: appends a seqence of values to the sequence in a key

Keys have different value types. For example, the name key has a type of SettingKey[String]. The key libraryDependencies has a type of SettingKey[ModuleID]. sbt provides%method to create aModuleIDinstance from strings like"groupId" % "artifactId" % "version".

A given key always refers to either a task or a plain setting. In sbt shell, typing a setting key shows its value, typing a task key executes the task but doesn’t display the resulting value. Use show taskKey to execute the task and show its result.

2.2 Tasks

A task runs whenever you request its value. To create a new task, first create a new key that store the value of an operation. Then create a setting that bind the operation to this task key. Here the := is constructing a function that will compute the value fo the task key. By separating the computation (operation) from the setting, sbt allows parallel execution.

Following are commonly used tasks:

2.3 Setting Graph

sbt use .value to express setting dependencies. .value is not a normal Scala method call. sbt uses a macro to lift these otuside of the task body. For example:

1
2
3
4
5
6
7
8

scalacOptions := {
  val out = streams.value // streams task happens-before scalacOptions
  val log = out.log
  log.info("123")
  val ur = update.value // update task happens-before scalacOptions
  log.info("456")
  ur.allConfigurations.map(_.toString)
}

The scalacOptions depends on streams and update. Use inspect scalacOptions to check the dependencies or inspect tree scalacOptions to see transitive dependencies.

A setting key can depend on other setting keys but cannot depend on a task key because a setting key is evaluated only once on project load. The build.sbt DSL is used to construct a DAG of settings and tasks.

2.4 Subprojects

A build definition can have multiple projects (subproejcts). A project is defined by declaring a lazy val of type Project. For example:

1
2
3
4
5
6
7

lazy val util = (project in file("util"))
lazy val core = (project in file("core"))

// the base directory may be omitted if it’s the same as the name of the val.
// the following are the same as the above
// lazy val util = project
// lazy val core = project

Build-wide settings are common settings across multiple projects, use the ThisBuild scope. Common settings can be defined outside the project settings. Aggregation means that running a task on the aggregate project will also run it on the aggregated projects. Aggregation will run the aggregated tasks in parallel and with no defined ordering between them. For example,

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

ThisBuild / organization := "com.example"
ThisBuild / version      := "0.1.0-SNAPSHOT"
ThisBuild / scalaVersion := "2.12.10"

lazy val commonSettings = Seq(
  target := { baseDirectory.value / "target2" }
)

lazy val core = project.settings(
    commonSettings,
    // other settings
)

lazy val util = project.settings(
  commonSettings,
    // other settings
)

lazy val root = (project in file("."))
  .aggregate(util, core)
  .settings(
    update / aggregate := false // control aggregation per-task
  )

A project may depend on code in another project. lazy val foo = project.dependsOn(bar) allows foo to use classes from bar that must be comppiled first. It is an abbrevation of foo.dependsOn(bar % "compile->compile". The -> is used for task dependencies where compile the the default. foo.dependsOn(bar % "test" means foo.dependsOn(bar % "test->compile", foo’s test depend on the compile of bar. Multiple depdendencies can be defiend as foo.dependsOn(bar % "test->test;compile->compile".

If a project is not defined for the root directory in the build, sbt creates a default one that aggregate all subprojects in the build. You can ruan a task in another project by specifying the project ID, such as bar/compile.

It is recommended to put all project declarations and settings in the root build.sbt file in order to keep all build definition under a single file.

3 Scope

3.1 Key Scope

A key scope is defined by thre axes: subproject, dependency configuration and task. Subprojects are namespaces for tasks and settings. By default, sbt runs unprefiex settings/tasks against all projects. The subproject axis can be ThisBuild, which means a build-level setting that applies to all subprojects. A task key can be a scope for anther key.

Zero is a universal fallback for all scope axes. Global is a scope for all-Zero axes: Zero / Zero / Zero. Global / someKey is a shorthand for Zero / Zero / Zero / someKey. A key placed in build.sbt is scoped to ${current subproject} / Zero / Zero by default.

The inThisBuild(...) function will scope both the key and the body of the setting expression to ThisBuild.

3.2 Dependency Configurations

A dependency configuration (or “configuration” for short) defines a graph of library dependencies, classpath, sources and generated packages, etc. Configurations are namespaces for keys and settings. sbt has the following default configurations:

• Compile: settings and values used to compile the main project and generate production artifacts.
• Test: used to compile and run unit testing.
• Runtime: defines the classpath for the run task.
• IntegrationTest: used to run tests against production artifacts.

For example, the task defined at sources in Compile collects the source files to be compiled for production artifacts. A configuration can extend one or more configurations. For example, the Test extends the Runtime that extends Compile.

3.2 Referring to Scope

Use / operator to refer a scope of a key. In sbt shell, the scope is shown as ref / Config / taskKey / key. The ref could be a project id ProjectRef(uri("file:..."), "id") or ThisBuild that denotes the entire build scope. Zeor can appear for each axis. If a scoped key is missing, the infer rules are

• the current project will be used if the project axis is omitted
• a key-dependent configuration will be auto-detected if configuration or task axis is omitted.

3.3 Defining Values

Setting values can be used to define a values for other keys using the value method. Use Def.task to define call a function to compute a value from other settings or tasks.

3.4 Scope Delegation Rules

• R1: scope axes precedence: the subproject, the configuration, and the task.
• R2: task delegation order: given, then Zero.
• R3: configuration delegation order: given, its parent and ancestors, then Zero.
• R4: subproject delegation order: given, ThisBuild, then Zero.
• R5: a delegated scoped key and its dependent settings/tasks are evaluated without carrying the original context.

Use inspect command to understand keys, their scopes and delegation rules.

4 Library Dependency

There are internal dependencies (among projects) and external dependencies. External dependencies include unmanged dependencies (in the lib/ directory) and managed dependencies. The update task is responsible to resovle extenal dependencies.

4.1 Unmanaged Dependencies

Unmanaged can be simple: adding jar files to lib and they will be on the project classpath. It’s done.

Dependencies in lib go on all the classpaths for compile, test, run and console. Use configuration / dependencyClasspath to change it.

The default lib directory can be chanaged by setting unmanagedBase. For example: unmanagedBase := baseDirectory.value / "custom_lib".

There is also an unmanagedJars task that list the jars from the unmanagedBase directory. Change it for more complex settings.

4.2 Managed Dependencies

sbt uses Coursier that resove and fetch metadata and artifacts from both Maven and Ivy repositories. The sbt comes with built-in repositories such as Maven Central, Typesafe release and sbt community release.

Use libraryDependencies to specify library dependencies. For a library, the ModuleID format is "organization" % "artifactId" % "version". For a library built for a specific scala version, the ModuleID use "organization" %% "artifactId" % "version" to automatically append Scala version.

By default, all dependencies are put onto the default configuration, used for both running an compiling all code. To add a configuration such as test, just add it to the ModuleID: "organization" %% "artifactId" % "version" % "test".

The version doesn’t has to be a single fixed version. It can use the Ivy version syntax such as “1.9.+” or “latest.integration”.

Addtional repository is specified using resolvers += name at location. For example, resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots".

resovlers is empty by default. It is used to combine build definition with external ersolvers. To override default resolvers, override externalResolvers.

5 Plugins

A plugin extends the build definition, most commonly by adding new settings and tasks. Plugins are external libraries. sbt reads the .sbt and .scala files in /project to build definitions used in the root .sbt files. A plugin is a jar that contains settings and tasks. Usually include a plugin in project/plugins.sbt as the following example:

1
2
3
4

addSbtPlugin("org.scalastyle" %% "scalastyle-sbt-plugin" % "0.5.0")

// add non-default repository if necessary
resolvers += Resolver.sonatypeRepo("public")

Unlike the normal dependencies, the %% in addSbtPlugin appends both a Scala version and a sbt version to the name. The Scala version is the version used by the current sbt, not the project being built. The above code also includes an extra resolver because the plugin isn’t published to the standard set of repositories that sbt knows about by default.

Some plugins have the auto plugins feature that enables plugins to automatically and safely. Some plugins require explicit enablement using syntax as enablePlugins(FooPlugin, BarPlugin). Older non-auto plguins often require settings to te added explicitly. Use disablePlugins to remove a plugin settings.

Use plugins command to list plugins and there enable status.

Plugins can be installed for all your projects at once by declaring them in $HOME/.sbt/1.0/plugins/. Roughly speaking, any .sbt or .scala files in $HOME/.sbt/1.0/plugins/ behave as if they were in the project/ directory for all projects.

Organizing the Build

The definition in a .sbt file are not visible in another .sbt file. To share code, define one or more .scala files in the project/ directory of the build root.

One can put all dependencies in project/Dependencies.scala and import it into build.sbt.

For advance users, another way of organizing build is to define one-off auto-plugins in project/*.scala. By defining triggered plugins, auto plugins can inject tasks and commands across all subprojects.

The user’s program build is called proper build. The build.sbt and files in project/ are meta-build. The files in /project/project/ is called meta-meta-build.