Macro is widely used in popular Scala libraries such as Mill, Slick etc. Key concepts include reify, Context methods and Scala’s abstract tree syntax type Expr[T]. This article is based on Macro doc and the docs of Quasiquotes.

1 Overview

Macros is an experimental feature since Scala 2.10, around the year of 2014. It lets your write functions using the reflection API that are executed during compilation. There are many possible ways to hook into the compiler: type macros, annotation macros, untyped macros, etc. Scala use def macros. Def macros replace welltyped terms with other welltyped terms. Macros implementation use reflection API to analyze and generate code. Generated code can contain arbitrary Scala constructs.

Macros are good for code generation (term generation, type generation, materialization in type classes), static checks and DSL (language virtualization). The def macro is easy to understand because it is similar to the concept of a typed method call.

Scala has two types of macros: blackbox macros are macros that faithfully follow their type signatrues. Macros that can’t have precise signatures are called whitebox macros.

To use macro, either use import scala.language.experimental.macros on perfile basis or use language:experimental.macros compiler switch. In build tool, the macro should be placed in a separate project that is a dependcy for its callers. To debug Macro AST, use def scalacOptions = Seq("Ymacrodebuglite") to inpsect generated code.

Macros has some features such as Quasiquotes, Macro Bundles, Implict Macros, Extractor Macros, Type Providers, Macro Annotations, and Macro Paradis.

2 Def Macro

A macro is defined by using macro before the static implementation method. For example, def assert(cond: Boolean, msg: Any) = macro Asserts.assertImpl. A call assert(x < 10, "limit exceeded") at compile time becomes assertImpl(c)(<[ x < 10 ]>, <[ “limit exceeded” ]>). The c is a context argument that collects information by the compiler at the call site. <[ expr ]> denotes an expression c.Expr.

For big macro implementation, it is a good idea to define a help class that takes a context parameter. For example, class Helper(val c: Context) {...}. However, the c is a pathdependent type at call site. Two common ways to pass it are:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
// Method 1
// define Helper class
class Helper(val c: Context) {...}
// use refinement types
def impl(c1: Context): c1.Expr[Unit] = {
  val helper = new { val c: c1.type = c1 } with Helper
  c1.Expr( //use helper)
}

// method 2
// use extra type parameter
class Helper[C <: Context](val c: C) {...}
// pass the type
def impl(c: Context): c.Expr[Unit] = {
  val helper = new Helper[c.type](c)
  c.Expr(// use helper)

2.1 Expr

A Scala expression Expr is built from Tree. Scala uses Tree type to represent the abstract syntax tree. It has information such as type, position, children etc. A Scala macro manipulates trees to implement the logic. The Macro implementation method should return an Expr.

use Expr#tree to get a tree. use Expr#splice to convert an Expr to code that can be used in regular Scala code. use reify(code) to convert the code to an Expr.

The Expr is a member of the compiler library:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
trait Expr[+T] {
  // The scala AST of the expression
  def tree: Tree

  /**
  * It should only be used within a `reify` call, which eliminates the `splice` call and embeds
  * the wrapped tree into the reified surrounding expression.
  * For an expr of type `Expr[T]`, where `T` has a method `foo`, the following code
  * {{{
  *   reify{ expr.splice.foo }
  * }}}
  * uses splice to turn an expr of type Expr[T] into a value of type T in the context of `reify`.
  *
  * It is equivalent to
  * {{{
  *   Select( expr.tree, TermName("foo") )
  * }}}
  *
  * The following example code however does not compile
  * {{{
  *   reify{ expr.foo }
  * }}}
  * because expr of type Expr[T] itself does not have a method foo.
  */
  def splice: T

  ...
}

2.2 Generic Macros

If a macro has type parameters, the type arguments must be given explicitly in the macro definition’s body. Type parameters may come with WeakTypeTag context bounds. The type tags are instantiated at the application site when the macro is expanded. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
class Queryable[T] {
 def map[U](p: T => U): Queryable[U] = macro QImpl.map[T, U]
}

object QImpl {
 def map[T: c.WeakTypeTag, U: c.WeakTypeTag]
        (c: Context)
        (p: c.Expr[T => U]): c.Expr[Queryable[U]] = ...
}

// a call of the following
q.map[Int](s => s.length)

// wil be expanded as
QImpl.map(c)(<[ s => s.length ]>)
   (implicitly[WeakTypeTag[String]], implicitly[WeakTypeTag[Int]])
```

### 2.3 Universe Types

The `scala.reflect.macros.Context.Universe` abstract class has the main types used in Scala reflection. It has symbols, types, flagsets, scopes, names, trees, constants, annotations, exprs etc. Some types are:

 `If`: for `if` statement
 `Select(a, TermName(b))`: an AST for code `a.b`
 `TermName`: an ast for code `TermName(s)`
 `Ident(name)`: an AST node for code `name`
 `Apply`: metho call with a list of parameters
 `Listeral`: a literal data type
 `Constant`: a constant value type
 `Block`: This AST node corresponds to the following Scala code: `{ stats; expr }`
 `ValDef`: The AST for value definitions such as `` mods `val` name: tpt = rhs `` or `` mods `var` name: tpt = rhs ``.

The `Universe` also defines a `reify` metehod as `def reify[T](expr: T): Expr[T] = macro ???`. It takes an`expr` (written in Scala code) of type `T` and returns an `Expr[T]`. For example:

```scala
val five = reify { 5 }  // Literal(Constant(5))
reify { 5.toString } // Apply(Select(Literal(Constant(5)), TermName("toString)), List())
reify { five.splice.toString } // Apply(Select(five, TermName("toString")), List())

The reify is timetravel: it regenerates an expression tree when it runs. Called from a macro implementation, it create the AST passed to it at macroexpansion time. Moreover, reify packages the result expression tree with the types and values of all free references that occur in it. All identifiers referred in the expression are checked and bound at the definition site. Scala macros are selfcleaning and the process is hygienic.

Inside the reify method, all experssions should evaluated using their splice methods.

2.4 Sample Code

The build.sc file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
import $ivy.`com.lihaoyi::millcontribbloop:$MILL_VERSION`

import mill._, scalalib._

object mt extends ScalaModule {

  def scalaVersion = "2.13.1"

  // to debug Macro AST
  // def scalacOptions = Seq("-Ymacro-debug-lite")

  def ivyDeps = Agg(
    ivy"org.scalalang:scalareflect:2.13.1"
  )
}

object foo extends ScalaModule {
  def scalaVersion = "2.13.1"

  def moduleDeps = Seq(mt)
}

The mt/src/Macros.scala file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
import scala.language.experimental.macros
import scala.reflect.macros.Context
import scala.collection.mutable.{ListBuffer, Stack}

object Macros {
  def printf(format: String, params: Any*): Unit = macro printf_impl

  def printf_impl(
      c: Context
  )(format: c.Expr[String], params: c.Expr[Any]*): c.Expr[Unit] = {
    import c.universe._

    val Literal(Constant(s_format: String)) = format.tree

    val evals = ListBuffer[ValDef]()
    def precompute(value: Tree, tpe: Type): Ident = {
      val freshName = TermName(c.freshName("eval$"))
      evals += ValDef(Modifiers(), freshName, TypeTree(tpe), value)
      Ident(freshName)
    }

    val paramsStack = Stack[Tree]((params map (_.tree)): _*)
    val refs = s_format.split("(?<=%[\\w%])|(?=%[\\w%])") map {
      case "%d" => precompute(paramsStack.pop, typeOf[Int])
      case "%s" => precompute(paramsStack.pop, typeOf[String])
      case "%%" => Literal(Constant("%"))
      case part => Literal(Constant(part))
    }

    val stats =
      evals ++ refs.map(ref => reify(print(c.Expr[Any](ref).splice)).tree)
    c.Expr[Unit](Block(stats.toList, Literal(Constant(()))))
  }

  def debug(param: Any): Unit = macro debug_impl

  def debug_impl(c: Context)(param: c.Expr[Any]): c.Expr[Unit] = {
    import c.universe._

    val paramStr = param.tree.toString
    val paramRep = Literal(Constant(paramStr))
    val paramEx = c.Expr[String](paramRep)

    reify { println(paramEx.splice + " = " + param.splice) }
  }
}

The foo/src/Example.scala file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
// Example.scala
import Macros._

object Example {
  def main(args: Array[String]): Unit = {

    printf("hello macro %s!\n", " world")

    val x = 10
    val y = 10
    debug(x)
    debug(x + y)
  }
}

3 Quasiquotes

Quasiquotes are used to manipulate Scala syntax with ease. `q”…” string interpolator becomes a syntactic tree. To use it in REPLE, import the following types:

1
2
3
4
5
6
7
val universe: scala.reflect.runtime.universe.type = scala.reflect.runtime.universe
// this brings in the quasiquotes
import universe._

// to use ToolBox API
import scala.reflect.runtime.currentMirror
import scala.tools.reflect.ToolBox

The ToolBox API can generate, compile and run Scala code at runtime.

3.1 Introduction

There are five interpolators:

q: expressions, definitions and imports tq: types pq: patterns cq: case clause fq: for loop enumerator

Use $name to unquote an expression tree inside a quasiquote. Use ..$name to flat a Iterable[Tree] and ...$name to flat Iterable[Iterable[Tree]].

Lifting is used to unquote custome data types in quasiquotes. Unlifting recovers a value from a quasiquote.

A code generator is called hygienic if it ensures the absence of name clashes between regular and generated code, preventing accidental capture of identifiers.

3.2 Expression Deatils

3.2.1 Empty, Literal and Identifier

q"" has a type of EmptyTree. It indicates that some part of the tree is not provided.

q"$value" has a type of Literal. For example: q"1", q"true", q""" "string" """.

You can also use lifting. val x = 1; val one = q"$x". Lifting doesn’t work for null value. Use q"null". For unlifting, use val q"${x: Int}" = q"1".

q"$tname" or q"name" has a type of Ident. Example:

1
2
3
4
5
6
7
8
val name = TermName("Foo")
val foo = q"$name"

val name2 = q"Foo"


// unlifting
val q"${name: TermName}" = q"Foo"

3.2.2 Application, Assign, Update and Return

q"$expr(..$exprss)" for single parameter list or q"$expr(...$exprss)" for multipel parameter lists, they have a type of Apply. q"$expr[..$tpts]" has a type of TypeApply.

1
2
3
4
5
val apps = List(q"f[Int](1, 2)", q"f('a, 'b)")
apps.foreach {
  case q"f[..$ts](..$args)" =>
    println(s"type arguments: $ts, value arguments: $args")
}

q"$expr = $expr" has a type of Assign or AssignOrNamedArg. For example: q"x = 2" or q"array(0) = 1".

q"$expr(..$exprs) = $expr" has a type of Tree.

q"return $expr" has a type of Return. For example: q"return 2 + 2".

3.2.3 Ascription and Annotated

q"$expr: $tpt" has a type of Typed. For example: q"(1 + 1): Int". q"$expr: @$annot" has a type of Annotated. For example: q"(1 + 1): @positive"

3.2.4 Others

Code Feature Quasiquote Type
Tuple q”(..\$exprs)” Tree
Block q”{ ..\$stats }” Block
If q”if ($expr) $expr else \$expr” If
Pattern Match q”$expr match { case ..$cases }” Match
Try q”try $expr catch { case ..$cases } finally \$expr” Try
Function q”(..$params) => $expr” Function
Partial Function q”{ case ..\$cases }” Match
While Loop q”while ($expr) $expr” LabelDef
DoWhile Loop q”do $expr while ($expr)” LabelDef
For Loop q”for (..$enums) $expr” Tree
ForYield Loop q”for (..$enums) yield $expr” Tree
New q”new { ..$earlydefns } with ..$parents { $self => ..\$stats }” Tree