11.5. Macros🔗

Macros are transformations from Syntax to Syntax that occur during elaboration and during tactic execution. Replacing syntax with the result of transforming it with a macro is called macro expansion. Multiple macros may be associated with a single syntax kind, and they are attempted in order of definition. Macros are run in a monad that has access to some compile-time metadata and has the ability to either emit an error message or to delegate to subsequent macros, but the macro monad is much less powerful than the elaboration monads.

Macros are associated with syntax kinds. An internal table maps syntax kinds to macros of type Syntax MacroM Syntax. Macros delegate to the next entry in the table by throwing the unsupportedSyntax exception. A given Syntax value is a macro when there is a macro associated with its syntax kind that does not throw unsupportedSyntax. If a macro throws any other exception, an error is reported to the user. Syntax categories are irrelevant to macro expansion; however, because each syntax kind is typically associated with a single syntax category, they do not interfere in practice.

Macro Error Reporting

The following macro reports an error when its parameter is the literal numeral five. It expands to its argument in all other cases.

syntax &"notFive" term:arg : term open Lean in macro_rules | `(term|notFive 5) => Macro.throwError "'5' is not allowed here" | `(term|notFive $e) => pure e

When applied to terms that are not syntactically the numeral five, elaboration succeeds:

5#eval notFive (2 + 3)
5

When the error case is triggered, the user receives an error message:

#eval '5' is not allowed herenotFive 5
'5' is not allowed here

Before elaborating a piece of syntax, the elaborator checks whether its syntax kind has macros associated with it. These are attempted in order. If a macro succeeds, potentially returning syntax with a different kind, the check is repeated and macros are expanded again until the outermost layer of syntax is no longer a macro. Elaboration or tactic execution can then proceed. Only the outermost layer of syntax (typically a node) is expanded, and the output of macro expansion may contain nested syntax that is a macro. These nested macros are expanded in turn when the elaborator reaches them.

In particular, macro expansion occurs in three situations in Lean:

  1. During term elaboration, macros in the outermost layer of the syntax to be elaborated are expanded prior to invoking the syntax's term elaborator.

  2. During command elaboration, macros in the outermost layer of the syntax to be elaborated are expanded prior to invoking the syntax's command elaborator.

  3. During tactic execution, macros in the outermost layer of the syntax to be elaborated are expanded prior to executing the syntax as a tactic.

11.5.1. Hygiene🔗

A macro is hygienic if its expansion cannot result in identifier capture. Identifier capture is when an identifier ends up referring to a binding site other than that which is in scope where the identifier occurs in the source code. There are two types of identifier capture:

  • If a macro's expansion introduces binders, then identifiers that are parameters to the macro may end up referring to the introduced binders if their names happen to match.

  • If a macro's expansion is intended to refer to a name, but the macro is used in a context that either locally binds this name or in which a new global name has been introduced, it may end up referring to the wrong name.

The first kind of variable capture can be avoided by ensuring that every binding introduced by a macro uses a freshly generated, globally-unique name, while the second can be avoided by always using fully-qualified names to refer to constants. The fresh names must be generated again at each invocation of the macro to avoid variable capture in recursive macros. These techniques are error-prone. Variable capture issues are difficult to test for because they rely on coincidences of name choices, and consistently applying these techniques results in noisy code.

Lean features automatic hygiene: in almost all cases, macros are automatically hygienic. Capture by introduced bindings is avoided by annotating identifiers introduced by a macro with macro scopes, which uniquely identify each invocation of macro expansion. If the binding and the use of the identifier have the same macro scopes, then they were introduced by the same step of macro expansion and should refer to one another. Similarly, uses of global names in code generated by a macro are not captured by local bindings in the context in which they are expanded because these use sites have macro scopes that are not present in the binding occurrence. Capture by newly-introduced global names is prevented by annotating potential global name references with the set of global names that match at quotation time in code produced in the macro's body. Identifiers annotated with potential referents are called pre-resolved identifiers, and the Syntax.Preresolved field on the Syntax.ident constructor is used to store the potential referents. During elaboration, if an identifier has pre-resolved global names associated with it, then other global names are not considered as valid reference targets.

The introduction of macro scopes and pre-resolved identifiers to generated syntax occurs during quotation. Macros that construct syntax by other means than quotation should also ensure hygiene by some other means. For more details on Lean's hygiene algorithm, please consult Ullrich and de Moura (2020)Sebastian Ullrich and Leonardo de Moura, 2020. “Beyond notations: Hygienic macro expansion for theorem proving languages”. In Proceedings of the International Joint Conference on Automated Reasoning. and Ullrich (2023)Sebastian Ullrich, 2023. An Extensible Theorem Proving Frontend. Dr. Ing. dissertation, Karlsruhe Institute of Technology.

11.5.2. The Macro Monad🔗

The macro monad MacroM is sufficiently powerful to implement hygiene and report errors. Macro expansion does not have the ability to modify the environment directly, to carry out unification, to examine the current local context, or to do anything else that only makes sense in one particular context. This allows the same macro mechanism to be used throughout Lean, and it makes macros much easier to write than elaborators.

🔗def
Lean.MacroM (α : Type) : Type

The MacroM monad is the main monad for macro expansion. It has the information needed to handle hygienic name generation, and is the monad that macro definitions live in.

Notably, this is a (relatively) pure monad: there is no IO and no access to the Environment. That means that things like declaration lookup are impossible here, as well as IO.Ref or other side-effecting operations. For more capabilities, macros can instead be written as elab using adaptExpander.

🔗def
Lean.Macro.expandMacro? (stx : Lean.Syntax)
    : Lean.MacroM (Option Lean.Syntax)

expandMacro? stx returns some stxNew if stx is a macro, and stxNew is its expansion.

🔗def
Lean.Macro.trace (clsName : Lean.Name)
    (msg : String) : Lean.MacroM Unit

Add a new trace message, with the given trace class and message.

11.5.2.1. Exceptions and Errors🔗

The unsupportedSyntax exception is used for control flow during macro expansion. It indicates that the current macro is incapable of expanding the received syntax, but that an error has not occurred. The exceptions thrown by throwError and throwErrorAt terminate macro expansion, reporting the error to the user.

🔗def
Lean.Macro.throwUnsupported {α : Type}
    : Lean.MacroM α

Throw an unsupportedSyntax exception.

🔗
Lean.Macro.Exception.unsupportedSyntax
    : Lean.Macro.Exception

An unsupported syntax exception. We keep this separate because it is used for control flow: if one macro does not support a syntax then we try the next one.

🔗def
Lean.Macro.throwError {α : Type} (msg : String)
    : Lean.MacroM α

Throw an error with the given message, using the ref for the location information.

🔗def
Lean.Macro.throwErrorAt {α : Type}
    (ref : Lean.Syntax) (msg : String)
    : Lean.MacroM α

Throw an error with the given message and location information.

11.5.2.2. Hygiene-Related Operations🔗

Hygiene is implemented by adding macro scopes to the identifiers that occur in syntax. Ordinarily, the process of quotation adds all necessary scopes, but macros that construct syntax directly must add macro scopes to the identifiers that they introduce.

🔗def
Lean.Macro.withFreshMacroScope {α : Type}
    (x : Lean.MacroM α) : Lean.MacroM α

Increments the macro scope counter so that inside the body of x the macro scope is fresh.

🔗def
Lean.Macro.addMacroScope (n : Lean.Name)
    : Lean.MacroM Lean.Name

Add a new macro scope to the name n.

11.5.2.3. Querying the Environment🔗

Macros have only limited support for querying the environment. They can check whether a constant exists and resolve names, but further introspection is unavailable.

🔗def
Lean.Macro.hasDecl (declName : Lean.Name)
    : Lean.MacroM Bool

Returns true if the environment contains a declaration with name declName

🔗def
Lean.Macro.getCurrNamespace
    : Lean.MacroM Lean.Name

Gets the current namespace given the position in the file.

🔗def
Lean.Macro.resolveNamespace (n : Lean.Name)
    : Lean.MacroM (List Lean.Name)

Resolves the given name to an overload list of namespaces.

🔗def
Lean.Macro.resolveGlobalName (n : Lean.Name)
    : Lean.MacroM
      (List (Lean.Name × List String))

Resolves the given name to an overload list of global definitions. The List String in each alternative is the deduced list of projections (which are ambiguous with name components).

Remark: it will not trigger actions associated with reserved names. Recall that Lean has reserved names. For example, a definition foo has a reserved name foo.def for theorem containing stating that foo is equal to its definition. The action associated with foo.def automatically proves the theorem. At the macro level, the name is resolved, but the action is not executed. The actions are executed by the elaborator when converting Syntax into Expr.

11.5.3. Quotation🔗

Quotation marks code for representation as data of type Syntax. Quoted code is parsed, but not elaborated—while it must be syntactically correct, it need not make sense. Quotation makes it much easier to programmatically generate code: rather than reverse-engineering the specific nesting of node values that Lean's parser would produce, the parser can be directly invoked to create them. This is also more robust in the face of refactoring of the grammar that may change the internals of the parse tree without affecting the user-visible concrete syntax. Quotation in Lean is surrounded by ​`( and ).

The syntactic category or parser being quoted may be indicated by placing its name after the opening backtick and parenthesis, followed by a vertical bar (|). As a special case, the name tactic may be used to parse either tactics or sequences of tactics. If no syntactic category or parser is provided, Lean attempts to parse the quotation both as a term and as a non-empty sequence of commands. Term quotations have higher priority than command quotations, so in cases of ambiguity, the interpretation as a term is chosen; this can be overridden by explicitly indicating that the quotation is of a command sequence.

Term vs Command Quotation Syntax

In the following example, the contents of the quotation could either be a function application or a sequence of commands. Both match the same region of the file, so the local longest-match rule is not relevant. Term quotation has a higher priority than command quotation, so the quotation is interpreted as a term. Terms expect their antiquotations to have type TSyntax `term rather than TSyntax `command.

example (cmd1 cmd2 : TSyntax `command) : MacroM (TSyntax `command) := `($application type mismatch cmd1.raw argument cmd1 has type TSyntax `command : Type but is expected to have type TSyntax `term : Typecmd1 $application type mismatch cmd2.raw argument cmd2 has type TSyntax `command : Type but is expected to have type TSyntax `term : Typecmd2)

The result is two type errors like the following:

application type mismatch
  cmd1.raw
argument
  cmd1
has type
  TSyntax `command : Type
but is expected to have type
  TSyntax `term : Type

The type of the quotation (MacroM (TSyntax `command)) is not used to select a result because syntax priorities are applied prior to elaboration. In this case, specifying that the antiquotations are commands resolves the ambiguity because function application would require terms in these positions:

example (cmd1 cmd2 : TSyntax `command) : MacroM (TSyntax `command) := `($cmd1:command $cmd2:command)

Similarly, inserting a command into the quotation eliminates the possibility that it could be a term:

example (cmd1 cmd2 : TSyntax `command) : MacroM (TSyntax `command) := `($cmd1 $cmd2 #eval "hello!")
syntax

Lean's syntax includes quotations for terms, commands, tactics, and sequences of tactics, as well as a general quotation syntax that allows any input that Lean can parse to be quoted. Term quotations have the highest priority, followed by tactic quotations, general quotations, and finally command quotations.

term ::=
      Syntax quotation for terms. `(term)
    | `(command+)
    | `(tactic|tactic)
    | `(tactic|tactic;*)
    | `(p : identp:ident|Parse a p : identp here )

Rather than having type Syntax, quotations are monadic actions with type m Syntax. Quotation is monadic because it implements hygiene by adding macro scopes and pre-resolving identifiers, as described in the section on hygiene. The specific monad to be used is an implicit parameter to the quotation, and any monad for which there is an instance of the MonadQuotation type class is suitable. MonadQuotation extends MonadRef, which gives the quotation access to the source location of the syntax that the macro expander or elaborator is currently processing. MonadQuotation additionally includes the ability to add macro scopes to identifiers and use a fresh macro scope for a sub-task. Monads that support quotation include MacroM, TermElabM, CommandElabM, and TacticM.

11.5.3.1. Quasiquotation🔗

Quasiquotation is a form of quotation that may contain antiquotations, which are regions of the quotation that are not quoted, but instead are expressions that are evaluated to yield syntax. A quasiquotation is essentially a template; the outer quoted region provides a fixed framework that always yields the same outer syntax, while the antiquotations yield the parts of the final syntax that vary. All quotations in Lean are quasiquotations, so no special syntax is needed to distinguish quasiquotations from other quotations. The quotation process does not add macro scopes to identifiers that are inserted via antiquotations, because these identifiers either come from another quotation (in which case they already have macro scopes) or from the macro's input (in which case they should not have macro scopes, because they are not introduced by the macro).

Basic antiquotations consist of a dollar sign ($) immediately followed by an identifier. This means that the value of the corresponding variable, which should be a syntax tree, is to be substituted into this position of the quoted syntax. Entire expressions may be used as antiquotations by wrapping them in parentheses.

Lean's parser assigns every antiquotation a syntax category based on what the parser expects at the given position. If the parser expects syntax category c, then the antiquotation's type is TSyntax c.

Some syntax categories can be matched by elements of other categories. For example, numeric and string literals are valid terms in addition to being their own syntax categories. Antiquotations may be annotated with the expected category by suffixing them with a colon and the category name, which causes the parser to validate that the annotated category is acceptable in the given position and construct any intermediate layers that are required in the parse tree.

syntaxAntiquotations
antiquot ::=
      $ident(:ident)?
    | $(term)(:ident)?

Whitespace is not permitted between the dollar sign ('$') that initiates an antiquotation and the identifier or parenthesized term that follows. Similarly, no whitespace is permitted around the colon that annotates the syntax category of the antiquotation.

Quasiquotation

Both forms of antiquotation are used in this example. Because natural numbers are not syntax, quote is used to transform a number into syntax that represents it.

open Lean in example [Monad m] [MonadQuotation m] (x : Term) (n : Nat) : m Syntax := `($x + $(quote (n + 2)))
Antiquotation Annotations

This example requires that m is a monad that can perform quotation.

variable {m : Type Type} [Monad m] [MonadQuotation m]

By default, the antiquotation $e is expected to be a term, because that's the syntactic category that's immediately expected as the second argument to addition.

def ex1 (e) := show m _ from `(2 + $e) ex1 {m : Type → Type} [Monad m] [MonadQuotation m] (e : TSyntax `term) : m (TSyntax `term)#check ex1
ex1 {m : Type → Type} [Monad m] [MonadQuotation m] (e : TSyntax `term) : m (TSyntax `term)

Annotating $e as a numeric literal succeeds, because numeric literals are also valid terms. The expected type of the parameter e changes to TSyntax `num.

def ex2 (e) := show m _ from `(2 + $e:num) ex2 {m : Type → Type} [Monad m] [MonadQuotation m] (e : TSyntax `num) : m (TSyntax `term)#check ex2
ex2 {m : Type → Type} [Monad m] [MonadQuotation m] (e : TSyntax `num) : m (TSyntax `term)

Spaces are not allowed between the dollar sign and the identifier.

def ex2 (e) := show m _ from `(2 + expected '`(tactic|' or no space before spliced term$ e:num)
<example>:1:35: expected '`(tactic|' or no space before spliced term

Spaces are also not allowed before the colon:

def ex2 (e) := show m _ from `(2 + $e expected ')':num)
<example>:1:38: expected ')'
Expanding Quasiquotation

Printing the definition of f demonstrates the expansion of a quasiquotation.

open Lean in def f [Monad m] [MonadQuotation m] (x : Term) (n : Nat) : m Syntax := `(fun k => $x + $(quote (n + 2)) + k) def f : {m : Type → Type} → [inst : Monad m] → [inst : Lean.MonadQuotation m] → Lean.Term → Nat → m Syntax := fun {m} [Monad m] [Lean.MonadQuotation m] x n => do let info ← Lean.MonadRef.mkInfoFromRefPos let scp ← Lean.getCurrMacroScope let mainModule ← Lean.getMainModule pure { raw := Syntax.node2 info `Lean.Parser.Term.fun (Syntax.atom info "fun") (Syntax.node4 info `Lean.Parser.Term.basicFun (Syntax.node1 info `null (Syntax.ident info "k".toSubstring' (Lean.addMacroScope mainModule `k scp) [])) (Syntax.node info `null #[]) (Syntax.atom info "=>") (Syntax.node3 info `«term_+_» (Syntax.node3 info `«term_+_» x.raw (Syntax.atom info "+") (Lean.quote `term (n + 2)).raw) (Syntax.atom info "+") (Syntax.ident info "k".toSubstring' (Lean.addMacroScope mainModule `k scp) []))) }.raw#print f
def f : {m : Type → Type} → [inst : Monad m] → [inst : Lean.MonadQuotation m] → Lean.Term → Nat → m Syntax :=
fun {m} [Monad m] [Lean.MonadQuotation m] x n => do
  let info ← Lean.MonadRef.mkInfoFromRefPos
  let scp ← Lean.getCurrMacroScope
  let mainModule ← Lean.getMainModule
  pure
      {
          raw :=
            Syntax.node2 info `Lean.Parser.Term.fun (Syntax.atom info "fun")
              (Syntax.node4 info `Lean.Parser.Term.basicFun
                (Syntax.node1 info `null (Syntax.ident info "k".toSubstring' (Lean.addMacroScope mainModule `k scp) []))
                (Syntax.node info `null #[]) (Syntax.atom info "=>")
                (Syntax.node3 info `«term_+_»
                  (Syntax.node3 info `«term_+_» x.raw (Syntax.atom info "+") (Lean.quote `term (n + 2)).raw)
                  (Syntax.atom info "+")
                  (Syntax.ident info "k".toSubstring' (Lean.addMacroScope mainModule `k scp) []))) }.raw

In this output, the quotation is a Lean.Parser.Term.do : termdo block. It begins by constructing the source information for the resulting syntax, obtained by querying the compiler about the current user syntax being processed. It then obtains the current macro scope and the name of the module being processed, because macro scopes are added with respect to a module to enable independent compilation and avoid the need for a global counter. It then constructs a node using helpers such as Syntax.node1 and Syntax.node2, which create a Syntax.node with the indicated number of children. The macro scope is added to each identifier, and TSyntax.raw is used to extract the contents of typed syntax wrappers. The antiquotations of x and quote (n + 2) occur directly in the expansion, as parameters to Syntax.node3.

11.5.3.2. Splices🔗

In addition to including other syntax via antiquotations, quasiquotations can include splices. Splices indicate that the elements of an array are to be inserted in order. The repeated elements may include separators, such as the commas between list or array elements. Splices may consist of an ordinary antiquotation with a splice suffix, or they may be extended splices that provide additional repeated structure.

Splice suffixes consist of either an asterisk or a valid atom followed by an asterisk (*). Suffixes may follow any identifier or term antiquotation. An antiquotation with the splice suffix * corresponds to a use of many or many1; both the * and + suffixes in syntax rules correspond to the * splice suffix. An antiquotation with a splice suffix that includes an atom prior to the asterisk corresponds to a use of sepBy or sepBy1. The splice suffix ? corresponds to a use of optional or the ? suffix in a syntax rule. Because ? is a valid identifier character, identifiers must be parenthesized to use it as a suffix.

While there is overlap between repetition specifiers for syntax and antiquotation suffixes, they have distinct syntaxes. When defining syntax, the suffixes *, +, ,*, ,+, ,*,?, and ,+,? are built in to Lean. There is no shorter way to specify separators other than ,. Antiquotation suffixes are either just * or whatever atom was provided to sepBy or sepBy1 followed by *. The syntax repetitions + and * correspond to the splice suffix *; the repetitions ,*, ,+, ,*,?, and ,+,? correspond to ,*. The optional suffix ? in syntax and splices correspond with each other.

Syntax Repetition

Splice Suffix

+*

*

,*,+,*,?,+,?

,*

sepBy(_, "S")sepBy1(_, "S")

S*

?

?

Suffixed Splices

This example requires that m is a monad that can perform quotation.

variable {m : Type Type} [Monad m] [MonadQuotation m]

By default, the antiquotation $e is expected to be an array of terms separated by commas, as is expected in the body of a list:

def ex1 (xs) := show m _ from `(#[$xs,*]) ex1 {m : Type → Type} [Monad m] [MonadQuotation m] (xs : Syntax.TSepArray `term ",") : m (TSyntax `term)#check ex1
ex1 {m : Type → Type} [Monad m] [MonadQuotation m] (xs : Syntax.TSepArray `term ",") : m (TSyntax `term)

However, Lean includes a collection of coercions between various representations of arrays that will automatically insert or remove separators, so an ordinary array of terms is also acceptable:

def ex2 (xs : Array (TSyntax `term)) := show m _ from `(#[$xs,*]) ex2 {m : Type → Type} [Monad m] [MonadQuotation m] (xs : Array (TSyntax `term)) : m (TSyntax `term)#check ex2
ex2 {m : Type → Type} [Monad m] [MonadQuotation m] (xs : Array (TSyntax `term)) : m (TSyntax `term)

Repetition annotations may also be used with term antiquotations and syntax category annotations. This example is in CommandElabM so the result can be conveniently logged.

def ex3 (size : Nat) := show CommandElabM _ from do let mut nums : Array Nat := #[] for i in [0:size] do nums := nums.push i let stx `(#[$(nums.map (Syntax.mkNumLit toString)):num,*]) -- Using logInfo here causes the syntax to be rendered via the pretty printer. logInfo stx #[0, 1, 2, 3]#eval ex3 4
#[0, 1, 2, 3]
Non-Comma Separators

The following unconventional syntax for lists separates numeric elements by either em dashes or double asterisks, rather than by commas.

syntax "⟦" sepBy1(num, " — ") "⟧": term syntax "⟦" sepBy1(num, " ** ") "⟧": term

This means that —* and *** are valid splice suffixes between the and atoms. In the case of ***, the first two asterisks are the atom in the syntax rule, while the third is the repetition suffix.

macro_rules | `($n:num—*) => `($n***) | `($n:num***) => `([$n,*]) [1, 2, 3]#eval 1 2 3
[1, 2, 3]
Optional Splices

The following syntax declaration optionally matches a term between two tokens. The parentheses around the nested term are needed because term? is a valid identifier.

syntax "⟨| " (term)? " |⟩": term

The ? splice suffix for a term expects an Option Term:

def mkStx [Monad m] [MonadQuotation m] (failed to infer binder type when the resulting type of a declaration is explicitly provided, all holes (e.g., `_`) in the header are resolved before the declaration body is processede) : m Term := `(⟨| $(e)? |⟩) mkStx {m : Type → Type} [Monad m] [MonadQuotation m] (e : Option (TSyntax `term)) : m Term#check mkStx
mkStx {m : Type → Type} [Monad m] [MonadQuotation m] (e : Option (TSyntax `term)) : m Term

Supplying some results in the optional term being present.

⟨| 5 |⟩#eval do logInfo ( mkStx (some (quote 5)))
⟨| 5 |⟩

Supplying none results in the optional term being absent.

⟨| |⟩#eval do logInfo ( mkStx none)
⟨| |⟩

11.5.3.3. Token Antiquotations🔗

In addition to antiquotations of complete syntax, Lean features token antiquotations which allow the source information of an atom to be replaced with the source information from some other syntax. This is primarily useful to control the placement of error messages or other information that Lean reports to users. A token antiquotation does not allow an arbitrary atom to be inserted via evaluation. A token antiquotation consists of an atom (that is, a keyword)

syntaxToken Antiquotations

Token antiquotations replace the source information (of type SourceInfo) on a token with the source information from some other syntax.

antiquot ::= ...
    | atom%$ident

More complex splices with brackets

11.5.4. Matching Syntax🔗

Quasiquotations can be used in pattern matching to recognize syntax that matches a template. Just as antiquotations in a quotation that's used as a term are regions that are treated as ordinary non-quoted expressions, antiquotations in patterns are regions that are treated as ordinary Lean patterns. Quote patterns are compiled differently from other patterns, so they can't be intermixed with non-quote patterns in a single Lean.Parser.Term.match : termPattern matching. `match e, ... with | p, ... => f | ...` matches each given term `e` against each pattern `p` of a match alternative. When all patterns of an alternative match, the `match` term evaluates to the value of the corresponding right-hand side `f` with the pattern variables bound to the respective matched values. If used as `match h : e, ... with | p, ... => f | ...`, `h : e = p` is available within `f`. When not constructing a proof, `match` does not automatically substitute variables matched on in dependent variables' types. Use `match (generalizing := true) ...` to enforce this. Syntax quotations can also be used in a pattern match. This matches a `Syntax` value against quotations, pattern variables, or `_`. Quoted identifiers only match identical identifiers - custom matching such as by the preresolved names only should be done explicitly. `Syntax.atom`s are ignored during matching by default except when part of a built-in literal. For users introducing new atoms, we recommend wrapping them in dedicated syntax kinds if they should participate in matching. For example, in ```lean syntax "c" ("foo" <|> "bar") ... ``` `foo` and `bar` are indistinguishable during matching, but in ```lean syntax foo := "foo" syntax "c" (foo <|> "bar") ... ``` they are not. match expression. Like ordinary quotations, quote patterns are first processed by Lean's parser. The parser's output is then compiled into code that determines whether there is a match. Syntax matching assumes that the syntax being matched was produced by Lean's parser, either via quotation or directly in user code, and uses this to omit some checks. For example, if nothing but a particular keyword can be present in a given position, the check may be omitted.

Syntax matches a quote pattern in the following cases:

Atoms

Keyword atoms (such as termIfThenElse : term`if c then t else e` is notation for `ite c t e`, "if-then-else", which decides to return `t` or `e` depending on whether `c` is true or false. The explicit argument `c : Prop` does not have any actual computational content, but there is an additional `[Decidable c]` argument synthesized by typeclass inference which actually determines how to evaluate `c` to true or false. Write `if h : c then t else e` instead for a "dependent if-then-else" `dite`, which allows `t`/`e` to use the fact that `c` is true/false. if or Lean.Parser.Term.match : termPattern matching. `match e, ... with | p, ... => f | ...` matches each given term `e` against each pattern `p` of a match alternative. When all patterns of an alternative match, the `match` term evaluates to the value of the corresponding right-hand side `f` with the pattern variables bound to the respective matched values. If used as `match h : e, ... with | p, ... => f | ...`, `h : e = p` is available within `f`. When not constructing a proof, `match` does not automatically substitute variables matched on in dependent variables' types. Use `match (generalizing := true) ...` to enforce this. Syntax quotations can also be used in a pattern match. This matches a `Syntax` value against quotations, pattern variables, or `_`. Quoted identifiers only match identical identifiers - custom matching such as by the preresolved names only should be done explicitly. `Syntax.atom`s are ignored during matching by default except when part of a built-in literal. For users introducing new atoms, we recommend wrapping them in dedicated syntax kinds if they should participate in matching. For example, in ```lean syntax "c" ("foo" <|> "bar") ... ``` `foo` and `bar` are indistinguishable during matching, but in ```lean syntax foo := "foo" syntax "c" (foo <|> "bar") ... ``` they are not. match) result in singleton nodes whose kind is token. followed by the atom. In many cases, it is not necessary to check for specific atom values because the grammar allows only a single keyword, and no checking will be performed. If the syntax of the term being matched requires the check, then the node kind is compared.

Literals, such as string or numeric literals, are compared via their underlying string representation. The pattern ​`(0x15) and the quotation ​`(21) do not match.

Nodes

If both the pattern and the value being matched represent Syntax.node, there is a match when both have the same syntax kind, the same number of children, and each child pattern matches the corresponding child value.

Identifiers

If both the pattern and the value being matched are identifiers, then their literal Name values are compared for equality modulo macro scopes. Identifiers that “look” the same match, and it does not matter if they refer to the same binding. This design choice allows quote pattern matching to be used in contexts that don't have access to a compile-time environment in which names can be compared by reference.

Because quotation pattern matching is based on the node kinds emitted by the parser, quotations that look identical may not match if they come from different syntax categories. If in doubt, including the syntax category in the quotation can help.

11.5.5. Defining Macros🔗

There are two primary ways to define macros: the Lean.Parser.Command.macro_rules : commandmacro_rules command and the Lean.Parser.Command.macro : commandmacro command. The Lean.Parser.Command.macro_rules : commandmacro_rules command associates a macro with existing syntax, while the Lean.Parser.Command.macro : commandmacro command simultaneously defines new syntax and a macro that translates it to existing syntax. The Lean.Parser.Command.macro : commandmacro command can be seen as a generalization of Lean.Parser.Command.notation : commandnotation that allows the expansion to be generated programmatically, rather than simply by substitution.

11.5.5.1. The macro_rules Command🔗

syntax

The Lean.Parser.Command.macro_rules : commandmacro_rules command takes a sequence of rewrite rules, specified as syntax pattern matches, and adds each as a macro. The rules are attempted in order, before previously-defined macros, and later macro definitions may add further macro rules.

command ::= ...
    | A `docComment` parses a "documentation comment" like `/-- foo -/`. This is not treated like
a regular comment (that is, as whitespace); it is parsed and forms part of the syntax tree structure.

A `docComment` node contains a `/--` atom and then the remainder of the comment, `foo -/` in this
example. Use `TSyntax.getDocString` to extract the body text from a doc string syntax node. docComment?
      (@[attrInstance,*])?
      `attrKind` matches `("scoped" <|> "local")?`, used before an attribute like `@[local simp]`. attrKind macro_rules ((kind := ident))?
        (| Syntax quotation for terms. `((p : identp:ident|)?Suitable syntax for p : identp ) => term)*

The patterns in the macros must be quotation patterns. They may match syntax from any syntax category, but a given pattern can only ever match a single syntax kind. If no category or parser is specified for the quotation, then it may match terms or (sequences of) commands, but never both. In case of ambiguity, the term parser is chosen.

Internally, macros are tracked in a table that maps each syntax kind to its macros. The Lean.Parser.Command.macro_rules : commandmacro_rules command may be explicitly annotated with a syntax kind.

If a syntax kind is explicitly provided, the macro definition checks that each quotation pattern has that kind. If the parse result for the quotation was a choice node (that is, if the parse was ambiguous), then the pattern is duplicated once for each alternative with the specified kind. It is an error if none of the alternatives have the specified kind.

If no kind is provided explicitly, then the kind determined by the parser is used for each pattern. The patterns are not required to all have the same syntax kind; macros are defined for each syntax kind used by at least one of the patterns. It is an error if the parse result for a quotation pattern was a choice node (that is, if the parse was ambiguous).

The documentation comment associated with Lean.Parser.Command.macro_rules : commandmacro_rules is displayed to users if the syntax itself has no documentation comment. Otherwise, the documentation comment for the syntax itself is shown.

As with notations and operators, macro rules may be declared scoped or local. Scoped macros are only active when the current namespace is open, and local macro rules are only active in the current section scope.

Idiom Brackets

Idiom brackets are an alternative syntax for working with applicative functors. If the idiom brackets contain a function application, then the function is wrapped in pure and applied to each argument using <*>. Operator hyperlinking to docs Lean does not support idiom brackets by default, but they can be defined using a macro.

syntax (name := idiom) "⟦" (term:arg)+ "⟧" : term macro_rules | `($f $args*) => do let mut out `(pure $f) for arg in args do out `($out <*> $arg) return out

This new syntax can be used immediately.

def addFirstThird [Add α] (xs : List α) : Option α := Add.add xs[0]? xs[2]? none#eval addFirstThird (α := Nat) []
none
none#eval addFirstThird [1]
none
some 4#eval addFirstThird [1,2,3,4]
some 4
Scoped Macros

Scoped macro rules are active only in their namespace. When the namespace ConfusingNumbers is open, numeric literals will be assigned an incorrect meaning.

namespace ConfusingNumbers

The following macro recognizes terms that are odd numeric literals, and replaces them with double their value. If it unconditionally replaced them with double their value, then macro expansion would become an infinite loop because the same rule would always match the output.

scoped macro_rules | `($n:num) => do if n.getNat % 2 = 0 then Lean.Macro.throwUnsupported let n' := (n.getNat * 2) `($(Syntax.mkNumLit (info := n.raw.getHeadInfo) (toString n')))

Once the namespace ends, the macro is no longer used.

end ConfusingNumbers

Without opening the namespace, numeric literals function in the usual way.

(3, 4)#eval (3, 4)
(3, 4)

When the namespace is open, the macro replaces 3 with 6.

open ConfusingNumbers (6, 4)#eval (3, 4)
(6, 4)

It is not typically useful to change the interpretation of numeric or other literals in macros. However, scoped macros can be very useful when adding new rules to extensible tactics such as trivial that work well with the contents of the namespaces but should not always be used.

Behind the scenes, a Lean.Parser.Command.macro_rules : commandmacro_rules command generates one macro function for each syntax kind that is matched in its quote patterns. This function has a default case that throws the unsupportedSyntax exception, so further macros may be attempted.

A single Lean.Parser.Command.macro_rules : commandmacro_rules command with two rules is not always equivalent to two separate single-match commands. First, the rules in a Lean.Parser.Command.macro_rules : commandmacro_rules are tried from top to bottom, but recently-declared macros are attempted first, so the order would need to be reversed. Additionally, if an earlier rule in the macro throws the unsupportedSyntax exception, then the later rules are not tried; if they were instead in separate Lean.Parser.Command.macro_rules : commandmacro_rules commands, then they would be attempted.

One vs. Two Sets of Macro Rules

The arbitrary! macro is intended to expand to some arbitrarily-determined value of a given type.

syntax (name := arbitrary!) "arbitrary!" term:arg : term macro_rules | `(arbitrary! ()) => `(()) | `(arbitrary! Nat) => `(42) | `(arbitrary! ($t1 × $t2)) => `((arbitrary! $t1, arbitrary! $t2)) | `(arbitrary! Nat) => `(0)

Users may extend it by defining further sets of macro rules, such as this rule for Empty that fails:

macro_rules | `(arbitrary! Empty) => throwUnsupported (42, 42)#eval arbitrary! (Nat × Nat)
(42, 42)

If all of the macro rules had been defined as individual cases, then the result would have instead used the later case for Nat. This is because the rules in a single Lean.Parser.Command.macro_rules : commandmacro_rules command are checked from top to bottom, but more recently-defined Lean.Parser.Command.macro_rules : commandmacro_rules commands take precedence over earlier ones.

macro_rules | `(arbitrary! ()) => `(()) macro_rules | `(arbitrary! Nat) => `(42) macro_rules | `(arbitrary! ($t1 × $t2)) => `((arbitrary! $t1, arbitrary! $t2)) macro_rules | `(arbitrary! Nat) => `(0) macro_rules | `(arbitrary! Empty) => throwUnsupported (0, 0)#eval arbitrary! (Nat × Nat)
(0, 0)

Additionally, if any rule throws the unsupportedSyntax exception, no further rules in that command are checked.

macro_rules | `(arbitrary! Nat) => throwUnsupported redundant alternative #2| `(arbitrary! Nat) => `(42) macro_rules | `(arbitrary! Int) => `(42) macro_rules | `(arbitrary! Int) => throwUnsupported

The case for Nat fails to elaborate, because macro expansion did not translate the arbitrary! : termarbitrary! syntax into something supported by the elaborator.

#eval elaboration function for 'arbitrary!' has not been implemented arbitrary! Natarbitrary! Nat
elaboration function for 'arbitrary!' has not been implemented
  arbitrary! Nat

The case for Int succeeds, because the first set of macro rules are attempted after the second throws the exception.

42#eval arbitrary! Int
42

11.5.5.2. The macro Command🔗

The Lean.Parser.Command.macro : commandmacro command simultaneously defines a new syntax rule and associates it with a macro. Unlike Lean.Parser.Command.notation : commandnotation, which can define only new term syntax and in which the expansion is a term into which the parameters are to be substituted, the Lean.Parser.Command.macro : commandmacro command may define syntax in any syntax category and it may use arbitrary code in the MacroM monad to generate the expansion. Because macros are so much more flexible than notations, Lean cannot automatically generate an unexpander; this means that new syntax implemented via the Lean.Parser.Command.macro : commandmacro command is available for use in input to Lean, but Lean's output does not use it without further work.

syntax
command ::= ...
    | A `docComment` parses a "documentation comment" like `/-- foo -/`. This is not treated like
a regular comment (that is, as whitespace); it is parsed and forms part of the syntax tree structure.

A `docComment` node contains a `/--` atom and then the remainder of the comment, `foo -/` in this
example. Use `TSyntax.getDocString` to extract the body text from a doc string syntax node. docComment?
      (@[attrInstance,*])?
      `attrKind` matches `("scoped" <|> "local")?`, used before an attribute like `@[local simp]`. attrKind macro(:prec)? ((name := ident))? ((priority := prio))? macroArg* : ident =>
        macroRhs
syntax

A macro's arguments are either syntax items (as used in the Lean.Parser.Command.syntax : commandsyntax command) or syntax items with attached names.

macroArg ::=
    stx
macroArg ::= ...
    | ident:stx

In the expansion, the names that are attached to syntax items are bound; they have type TSyntax for the appropriate syntax kinds. If the syntax matched by the parser does not have a defined kind (e.g. because the name is applied to a complex specification), then the type is TSyntax Name.anonymous.

The documentation comment is associated with the new syntax, and the attribute kind (none, local, or scoped) governs the visibility of the macro just as it does for notations: scoped macros are available in the namespace in which they are defined or in any section scope that opens that namespace, while local macros are available only in the local section scope.

Behind the scenes, the Lean.Parser.Command.macro : commandmacro command is itself implemented by a macro that expands it to a Lean.Parser.Command.syntax : commandsyntax command and a Lean.Parser.Command.macro_rules : commandmacro_rules command. Any attributes applied to the macro command are applied to the syntax definition, but not to the Lean.Parser.Command.macro_rules : commandmacro_rules command.

11.5.5.3. The Macro Attribute🔗

Macros can be manually added to a syntax kind using the Lean.Parser.Attr.macro : attrmacro attribute. This low-level means of specifying macros is typically not useful, except as a result of code generation by macros that themselves generate macro definitions.

attribute

The Lean.Parser.Attr.macro : attrmacro attribute specifies that a function is to be considered a macro for the specified syntax kind.

attr ::= ...
    | macro ident
The Macro Attribute/-- Generate a list based on N syntactic copies of a term -/ syntax (name := rep) "[" num " !!! " term "]" : term @[macro rep] def expandRep : Macro | `([ $n:num !!! $e:term]) => let e' := Array.mkArray n.getNat e `([$e',*]) | _ => throwUnsupported

Evaluating this new expression demonstrates that the macro is present.

["hello", "hello", "hello"]#eval [3 !!! "hello"]
["hello", "hello", "hello"]