Reasoning about the syntax and semantics of programming languages is a popular application of proof assistants. Before proving the first theorem of this kind, it is necessary to choose a formal encoding of the informal notions of syntax, dealing with such issues as variable binding conventions. I believe the pragmatic questions in this domain are far from settled and remain as important open research problems. However, in this chapter, I will demonstrate two underused encoding approaches. Note that I am not recommending either approach as a silver bullet! Mileage will vary across concrete problems, and I expect there to be significant future advances in our knowledge of encoding techniques. For a broader introduction to programming language formalization, using more elementary techniques, see Software Foundations by Pierce et al. This chapter is also meant as a case study, bringing together what we have learned in the previous chapters. We will see a concrete example of the importance of representation choices; translating mathematics from paper to Coq is not a deterministic process, and different creative choices can have big impacts. We will also see dependent types and scripted proof automation in action, applied to solve a particular problem as well as possible, rather than to demonstrate new Coq concepts. I apologize in advance to those readers not familiar with the theory of programming language semantics. I will make a few remarks intended to relate the material here with common ideas in semantics, but these remarks should be safe for others to skip. We will define a small programming language and reason about its semantics, expressed as an interpreter into Coq terms, much as we have done in examples throughout the book. It will be helpful to build a slight extension of crush that tries to apply functional extensionality, an axiom we met in Chapter 12, which says that two functions are equal if they map equal inputs to equal outputs. We also use f_equal to simplify goals of a particular form that will come up with the term denotation function that we define shortly.

At this point in the book source, some auxiliary proofs also appear.

Here is a definition of the type system we will use throughout the chapter. It is for simply typed lambda calculus with natural numbers as the base type.

Now we have some choices as to how we represent the syntax of programs. The two sections of the chapter explore two such choices, demonstrating the effect the choice has on proof complexity.

Dependent de Bruijn Indices

The first encoding is one we met first in Chapter 9, the dependent de Bruijn index encoding. We represent program syntax terms in a type family parameterized by a list of types, representing the typing context, or information on which free variables are in scope and what their types are. Variables are represented in a way isomorphic to the natural numbers, where number 0 represents the first element in the context, number 1 the second element, and so on. Actually, instead of numbers, we use the member dependent type family from Chapter 9.

Here is the definition of the term type, including variables, constants, addition, function abstraction and application, and let binding of local variables.

Here are two example term encodings, the first of addition packaged as a two-argument curried function, and the second of a sample application of addition to constants.

Since dependent typing ensures that any term is well-formed in its context and has a particular type, it is easy to translate syntactic terms into Coq values.

With this term representation, some program transformations are easy to implement and prove correct. Certainly we would be worried if this were not the the case for the identity transformation, which takes a term apart and reassembles it.

A slightly more ambitious transformation belongs to the family of constant folding optimizations we have used as examples in other chapters.

The correctness proof is more complex, but only slightly so.

The transformations we have tried so far have been straightforward because they do not have interesting effects on the variable binding structure of terms. The dependent de Bruijn representation is called first-order because it encodes variable identity explicitly; all such representations incur bookkeeping overheads in transformations that rearrange binding structure. As an example of a tricky transformation, consider one that removes all uses of "let x = e1 in e2" by substituting e1 for x in e2. We will implement the translation by pairing the "compile-time" typing environment with a "run-time" value environment or substitution, mapping each variable to a value to be substituted for it. Such a substitute term may be placed within a program in a position with a larger typing environment than applied at the point where the substitute term was chosen. To support such context transplantation, we need lifting, a standard de Bruijn indices operation. With dependent typing, lifting corresponds to weakening for typing judgments. The fundamental goal of lifting is to add a new variable to a typing context, maintaining the validity of a term in the expanded context. To express the operation of adding a type to a context, we use a helper function insertAt.

Another function lifts bound variable instances, which we represent with member values.

The final helper function for lifting allows us to insert a new variable anywhere in a typing context.

In the Let removal transformation, we only need to apply lifting to add a new variable at the beginning of a typing context, so we package lifting into this final, simplified form.

Finally, we can implement Let removal. The argument of type hlist (term G') G represents a substitution mapping each variable from context G into a term that is valid in context G'. Note how the Abs case (1) extends via lifting the substitution s to hold in the broader context of the abstraction body e1 and (2) maps the new first variable to itself. It is only the Let case that maps a variable to any substitute beside itself.

We have finished defining the transformation, but the parade of helper functions is not over. To prove correctness, we will use one more helper function and a few lemmas. First, we need an operation to insert a new value into a substitution at a particular position.

Next we prove that liftVar is correct. That is, a lifted variable retains its value with respect to a substitution when we perform an analogue to lifting by inserting a new mapping into the substitution.

An analogous lemma establishes correctness of lift'.

Correctness of lift itself is an easy corollary.

Finally, we can prove correctness of unletSound for terms in arbitrary typing environments.

The lemma statement is a mouthful, with all its details of typing contexts and substitutions. It is usually prudent to state a final theorem in as simple a way as possible, to help your readers believe that you have proved what they expect. We follow that advice here for the simple case of terms with empty typing contexts.

The Let removal optimization is a good case study of a simple transformation that may turn out to be much more work than expected, based on representation choices. In the second part of this chapter, we consider an alternate choice that produces a more pleasant experience.

Parametric Higher-Order Abstract Syntax

In contrast to first-order encodings, higher-order encodings avoid explicit modeling of variable identity. Instead, the binding constructs of an object language (the language being formalized) can be represented using the binding constructs of the meta language (the language in which the formalization is done). The best known higher-order encoding is called higher-order abstract syntax (HOAS) , and we can start by attempting to apply it directly in Coq.

With HOAS, each object language binding construct is represented with a function of the meta language. Here is what we get if we apply that idea within an inductive definition of term syntax.
  Inductive term : type -> Type :=
  | Const : nat -> term Nat
  | Plus : term Nat -> term Nat -> term Nat

  | Abs : forall dom ran, (term dom -> term ran) -> term (Func dom ran)
  | App : forall dom ran, term (Func dom ran) -> term dom -> term ran

  | Let : forall t1 t2, term t1 -> (term t1 -> term t2) -> term t2.

However, Coq rejects this definition for failing to meet the strict positivity restriction. For instance, the constructor Abs takes an argument that is a function over the same type family term that we are defining. Inductive definitions of this kind can be used to write non-terminating Gallina programs, which breaks the consistency of Coq's logic. An alternate higher-order encoding is parametric HOAS, as introduced by Washburn and Weirich for Haskell and tweaked by me for use in Coq. Here the idea is to parameterize the syntax type by a type family standing for a representation of variables.

Coq accepts this definition because our embedded functions now merely take variables as arguments, instead of arbitrary terms. One might wonder whether there is an easy loophole to exploit here, instantiating the parameter var as term itself. However, to do that, we would need to choose a variable representation for this nested mention of term, and so on through an infinite descent into term arguments. We write the final type of a closed term using polymorphic quantification over all possible choices of var type family.

Here are the new representations of the example terms from the last section. Note how each is written as a function over a var choice, such that the specific choice has no impact on the structure of the term.

The argument var does not even appear in the function body for add. How can that be? By giving our terms expressive types, we allow Coq to infer many arguments for us. In fact, we do not even need to name the var argument!

Even though the var formal parameters appear as underscores, they are mentioned in the function bodies that type inference calculates.

Functional Programming with PHOAS

It may not be at all obvious that the PHOAS representation admits the crucial computable operations. The key to effective deconstruction of PHOAS terms is one principle: treat the var parameter as an unconstrained choice of which data should be annotated on each variable. We will begin with a simple example, that of counting how many variable nodes appear in a PHOAS term. This operation requires no data annotated on variables, so we simply annotate variables with unit values. Note that, when we go under binders in the cases for Abs and Let, we must provide the data value to annotate on the new variable we pass beneath. For our current choice of unit data, we always pass tt.

The above definition may seem a bit peculiar. What gave us the right to represent variables as unit values? Recall that our final representation of closed terms is as polymorphic functions. We merely specialize a closed term to exactly the right variable representation for the transformation we wish to perform.

It is easy to test that CountVars operates properly.

     = 2

In fact, PHOAS can be used anywhere that first-order representations can. We will not go into all the details here, but the intuition is that it is possible to interconvert between PHOAS and any reasonable first-order representation. Here is a suggestive example, translating PHOAS terms into strings giving a first-order rendering. To implement this translation, the key insight is to tag variables with strings, giving their names. The function takes as an additional input a string giving the name to be assigned to the next variable introduced. We evolve this name by adding a prime to its end. To avoid getting bogged down in orthogonal details, we render all constants as the string "N".

     = "(((fun x => (fun x' => (x + x'))) N) N)"

However, it is not necessary to convert to first-order form to support many common operations on terms. For instance, we can implement substitution of terms for variables. The key insight here is to tag variables with terms, so that, on encountering a variable, we can simply replace it by the term in its tag. We will call this function initially on a term with exactly one free variable, tagged with the appropriate substitute. During recursion, new variables are added, but they are only tagged with their own term equivalents. Note that this function squash is parameterized over a specific var choice.

To define the final substitution function over terms with single free variables, we define Term1, an analogue to Term that we defined before for closed terms.

Substitution is defined by (1) instantiating a Term1 to tag variables with terms and (2) applying the result to a specific term to be substituted. Note how the parameter var of squash is instantiated: the body of Subst is itself a polymorphic quantification over var, standing for a variable tag choice in the output term; and we use that input to compute a tag choice for the input term.

     = fun var : type -> Type =>
       Plus
         (App
            (App
               (Abs
                  (fun x : var Nat =>
                   Abs (fun y : var Nat => Plus (Var x) (Var y))))
               (Const 1)) (Const 2)) (Const 3)

One further development, which may seem surprising at first, is that we can also implement a usual term denotation function, when we tag variables with their denotations.

     = 3

To summarize, the PHOAS representation has all the expressive power of more standard first-order encodings, and a variety of translations are actually much more pleasant to implement than usual, thanks to the novel ability to tag variables with data.

Verifying Program Transformations

Let us now revisit the three example program transformations from the last section. Each is easy to implement with PHOAS, and the last is substantially easier than with first-order representations. First, we have the recursive identity function, following the same pattern as in the previous subsection, with a helper function, polymorphic in a tag choice; and a final function that instantiates the choice appropriately.

Proving correctness is both easier and harder than in the last section, easier because we do not need to manipulate substitutions, and harder because we do the induction in an extra lemma about ident, to establish the correctness theorem for Ident.

The translation of the constant-folding function and its proof work more or less the same way.

Things get more interesting in the Let-removal optimization. Our recursive helper function adapts the key idea from our earlier definitions of squash and Subst: tag variables with terms. We have a straightforward generalization of squash, where only the Let case has changed, to tag the new variable with the term it is bound to, rather than just tagging the variable with itself as a term.

We can test Unlet first on an uninteresting example, three_the_hard_way, which does not use Let.

     = fun var : type -> Type =>
       App
         (App
            (Abs
               (fun x : var Nat =>
                Abs (fun x0 : var Nat => Plus (Var x) (Var x0))))
            (Const 1)) (Const 2)

Next, we try a more interesting example, with some extra Lets introduced in three_the_hard_way.

     = fun var : type -> Type =>
       App
         (App
            (Abs
               (fun x : var Nat =>
                Abs (fun x0 : var Nat => Plus (Var x) (Var x0))))
            (Const 1)) (Const 2)

The output is the same as in the previous test, confirming that Unlet operates properly here. Now we need to state a correctness theorem for Unlet, based on an inductively proved lemma about unlet. It is not at all obvious how to arrive at a proper induction principle for the lemma. The problem is that we want to relate two instantiations of the same Term, in a way where we know they share the same structure. Note that, while Unlet is defined to consider all possible var choices in the output term, the correctness proof conveniently only depends on the case of var := typeDenote. Thus, one parallel instantiation will set var := typeDenote, to take the denotation of the original term. The other parallel instantiation will set var := term typeDenote, to perform the unlet transformation in the original term. Here is a relation formalizing the idea that two terms are structurally the same, differing only by replacing the variable data of one with another isomorphic set of variable data in some possibly different type family.

To formalize the tag isomorphism, we will use lists of values with the following record type. Each entry has an object language type and an appropriate tag for that type, in each of the two tag families var1 and var2.

Here is the inductive relation definition. An instance wf G e1 e2 asserts that terms e1 and e2 are equivalent up to the variable tag isomorphism G. Note how the Var rule looks up an entry in G, and the Abs and Let rules include recursive wf invocations inside the scopes of quantifiers to introduce parallel tag values to be considered as isomorphic.

We can state a well-formedness condition for closed terms: for any two choices of tag type families, the parallel instantiations belong to the wf relation, starting from an empty variable isomorphism.

After digesting the syntactic details of Wf, it is probably not hard to see that reasonable term encodings will satisfy it. For example:

Now we are ready to give a nice simple proof of correctness for unlet. First, we add one hint to apply a small variant of a standard library theorem connecting Forall, a higher-order predicate asserting that every element of a list satisfies some property; and In, the list membership predicate.

The rest of the proof is about as automated as we could hope for.

With this example, it is not obvious that the PHOAS encoding is more tractable than dependent de Bruijn. Where the de Bruijn version had lift and its helper functions, here we have Wf and its auxiliary definitions. In practice, Wf is defined once per object language, while such operations as lift often need to operate differently for different examples, forcing new implementations for new transformations. The reader may also have come up with another objection: via Curry-Howard, wf proofs may be thought of as first-order encodings of term syntax! For instance, the In hypothesis of rule WfVar is equivalent to a member value. There is some merit to this objection. However, as the proofs above show, we are able to reason about transformations using first-order representation only for their inputs, not their outputs. Furthermore, explicit numbering of variables remains absent from the proofs. Have we really avoided first-order reasoning about the output terms of translations? The answer depends on some subtle issues, which deserve a subsection of their own.

Establishing Term Well-Formedness

Can there be values of type Term t that are not well-formed according to Wf? We expect that Gallina satisfies key parametricity properties, which indicate how polymorphic types may only be inhabited by specific values. We omit details of parametricity theorems here, but forall t (E : Term t), Wf E follows the flavor of such theorems. One option would be to assert that fact as an axiom, "proving" that any output of any of our translations is well-formed. We could even prove the soundness of the theorem on paper meta-theoretically, say by considering some particular model of CIC. To be more cautious, we could prove Wf for every term that interests us, threading such proofs through all transformations. Here is an example exercise of that kind, for Unlet. First, we prove that wf is monotone, in that a given instance continues to hold as we add new variable pairs to the variable isomorphism.

Now we are ready to prove that unlet preserves any wf instance. The key invariant has to do with the parallel execution of unlet on two different var instantiations of a particular term. Since unlet uses term as the type of variable data, our variable isomorphism context G contains pairs of terms, which, conveniently enough, allows us to state the invariant that any pair of terms in the context is also related by wf.

Repackaging unletWf into a theorem about Wf and Unlet is straightforward.

This example demonstrates how we may need to use reasoning reminiscent of that associated with first-order representations, though the bookkeeping details are generally easier to manage, and bookkeeping theorems may generally be proved separately from the independently interesting theorems about program transformations.

A Few More Remarks

Higher-order encodings derive their strength from reuse of the meta language's binding constructs. As a result, we can write encoded terms so that they look very similar to their informal counterparts, without variable numbering schemes like for de Bruijn indices. The example encodings above have demonstrated this fact, but modulo the clunkiness of explicit use of the constructors of term. After defining a few new Coq syntax notations, we can work with terms in an even more standard form.

     = 3

The PHOAS approach shines here because we are working with an object language that has an easy embedding into Coq. That is, there is a straightforward recursive function translating object terms into terms of Gallina. All Gallina programs terminate, so clearly we cannot hope to find such embeddings for Turing-complete languages; and non-Turing-complete languages may still require much more involved translations. I have some work on modeling semantics of Turing-complete languages with PHOAS, but my impression is that there are many more advances left to be made in this field, possibly with completely new term representations that we have not yet been clever enough to think up.