### annotate src/Large.v @ 509:21079e42b773

Pass through Chapter 16
author Adam Chlipala Tue, 12 Feb 2013 11:14:52 -0500 da576746c3ba ed829eaa91b2
rev   line source
adamc@235 10 (* begin hide *)
adamc@235 16 (* end hide *)
adam@381 19 (** %\part{The Big Picture}
adam@381 21 \chapter{Proving in the Large}% *)
adam@433 23 (** It is somewhat unfortunate that the term "theorem proving" looks so much like the word "theory." Most researchers and practitioners in software assume that mechanized theorem proving is profoundly impractical. Indeed, until recently, most advances in theorem proving for higher-order logics have been largely theoretical. However, starting around the beginning of the 21st century, there was a surge in the use of proof assistants in serious verification efforts. That line of work is still quite new, but I believe it is not too soon to distill some lessons on how to work effectively with large formal proofs.
adamc@236 25 Thus, this chapter gives some tips for structuring and maintaining large Coq developments. *)
adamc@236 28 (** * Ltac Anti-Patterns *)
adam@433 30 (** In this book, I have been following an unusual style, where proofs are not considered finished until they are %\index{fully automated proofs}%"fully automated," in a certain sense. Each such theorem is proved by a single tactic. Since Ltac is a Turing-complete programming language, it is not hard to squeeze arbitrary heuristics into single tactics, using operators like the semicolon to combine steps. In contrast, most Ltac proofs "in the wild" consist of many steps, performed by individual tactics followed by periods. Is it really worth drawing a distinction between proof steps terminated by semicolons and steps terminated by periods?
adamc@237 32 I argue that this is, in fact, a very important distinction, with serious consequences for a majority of important verification domains. The more uninteresting drudge work a proof domain involves, the more important it is to work to prove theorems with single tactics. From an automation standpoint, single-tactic proofs can be extremely effective, and automation becomes more and more critical as proofs are populated by more uninteresting detail. In this section, I will give some examples of the consequences of more common proof styles.
adamc@236 34 As a running example, consider a basic language of arithmetic expressions, an interpreter for it, and a transformation that scales up every constant in an expression. *)
adamc@236 36 Inductive exp : Set :=
adamc@236 37 | Const : nat -> exp
adamc@236 38 | Plus : exp -> exp -> exp.
adamc@236 40 Fixpoint eval (e : exp) : nat :=
adamc@236 42 | Const n => n
adamc@236 43 | Plus e1 e2 => eval e1 + eval e2
adamc@236 46 Fixpoint times (k : nat) (e : exp) : exp :=
adamc@236 48 | Const n => Const (k * n)
adamc@236 49 | Plus e1 e2 => Plus (times k e1) (times k e2)
adam@452 52 (** We can write a very manual proof that [times] really implements multiplication. *)
adamc@236 54 Theorem eval_times : forall k e,
adamc@236 55 eval (times k e) = k * eval e.
adam@368 67 (* begin thide *)
adam@367 68 (** We use spaces to separate the two inductive cases, but note that these spaces have no real semantic content; Coq does not enforce that our spacing matches the real case structure of a proof. The second case mentions automatically generated hypothesis names explicitly. As a result, innocuous changes to the theorem statement can invalidate the proof. *)
adam@368 72 Theorem eval_times : forall k x,
adamc@236 73 eval (times k x) = k * eval x.
adamc@236 87 The inductive hypotheses are named [IHx1] and [IHx2] now, not [IHe1] and [IHe2]. *)
adamc@236 91 (** We might decide to use a more explicit invocation of [induction] to give explicit binders for all of the names that we will reference later in the proof. *)
adamc@236 93 Theorem eval_times : forall k e,
adamc@236 94 eval (times k e) = k * eval e.
adamc@236 95 induction e as [ | ? IHe1 ? IHe2 ].
adam@509 106 (** We pass %\index{tactics!induction}%[induction] an%\index{intro pattern}% _intro pattern_, using a [|] character to separate instructions for the different inductive cases. Within a case, we write [?] to ask Coq to generate a name automatically, and we write an explicit name to assign that name to the corresponding new variable. It is apparent that, to use intro patterns to avoid proof brittleness, one needs to keep track of the seemingly unimportant facts of the orders in which variables are introduced. Thus, the script keeps working if we replace [e] by [x], but it has become more cluttered. Arguably, neither proof is particularly easy to follow.
adamc@237 108 That category of complaint has to do with understanding proofs as static artifacts. As with programming in general, with serious projects, it tends to be much more important to be able to support evolution of proofs as specifications change. Unstructured proofs like the above examples can be very hard to update in concert with theorem statements. For instance, consider how the last proof script plays out when we modify [times] to introduce a bug. *)
adamc@236 112 Fixpoint times (k : nat) (e : exp) : exp :=
adamc@236 114 | Const n => Const (1 + k * n)
adamc@236 115 | Plus e1 e2 => Plus (times k e1) (times k e2)
adamc@236 118 Theorem eval_times : forall k e,
adamc@236 119 eval (times k e) = k * eval e.
adamc@236 120 induction e as [ | ? IHe1 ? IHe2 ].
adam@398 136 (** Can you spot what went wrong, without stepping through the script step-by-step? The problem is that [trivial] never fails. Originally, [trivial] had been succeeding in proving an equality that follows by reflexivity. Our change to [times] leads to a case where that equality is no longer true. The invocation [trivial] happily leaves the false equality in place, and we continue on to the span of tactics intended for the second inductive case. Unfortunately, those tactics end up being applied to the _first_ case instead.
adam@433 138 The problem with [trivial] could be "solved" by writing, e.g., [solve [ trivial ]] instead, so that an error is signaled early on if something unexpected happens. However, the root problem is that the syntax of a tactic invocation does not imply how many subgoals it produces. Much more confusing instances of this problem are possible. For example, if a lemma [L] is modified to take an extra hypothesis, then uses of [apply L] will generate more subgoals than before. Old unstructured proof scripts will become hopelessly jumbled, with tactics applied to inappropriate subgoals. Because of the lack of structure, there is usually relatively little to be gleaned from knowledge of the precise point in a proof script where an error is raised. *)
adamc@236 142 Fixpoint times (k : nat) (e : exp) : exp :=
adamc@236 144 | Const n => Const (k * n)
adamc@236 145 | Plus e1 e2 => Plus (times k e1) (times k e2)
adam@387 148 (** Many real developments try to make essentially unstructured proofs look structured by applying careful indentation conventions, idempotent case-marker tactics included solely to serve as documentation, and so on. All of these strategies suffer from the same kind of failure of abstraction that was just demonstrated. I like to say that if you find yourself caring about indentation in a proof script, it is a sign that the script is structured poorly.
adamc@236 150 We can rewrite the current proof with a single tactic. *)
adamc@236 152 Theorem eval_times : forall k e,
adamc@236 153 eval (times k e) = k * eval e.
adamc@236 154 induction e as [ | ? IHe1 ? IHe2 ]; [
adamc@236 156 | simpl; rewrite IHe1; rewrite IHe2; rewrite mult_plus_distr_l; trivial ].
adam@509 159 (** We use the form of the semicolon operator that allows a different tactic to be specified for each generated subgoal. This change improves the robustness of the script: we no longer need to worry about tactics from one case being applied to a different case. Still, the proof script is not especially readable. Probably most readers would not find it helpful in explaining why the theorem is true. The same could be said for scripts using the%\index{bullets}% _bullets_ or curly braces provided by Coq 8.4, which allow code like the above to be stepped through interactively, with periods in place of the semicolons, while representing proof structure in a way that is enforced by Coq. Interactive replay of scripts becomes easier, but readability is not really helped.
adamc@236 161 The situation gets worse in considering extensions to the theorem we want to prove. Let us add multiplication nodes to our [exp] type and see how the proof fares. *)
adamc@236 165 Inductive exp : Set :=
adamc@236 166 | Const : nat -> exp
adamc@236 167 | Plus : exp -> exp -> exp
adamc@236 168 | Mult : exp -> exp -> exp.
adamc@236 170 Fixpoint eval (e : exp) : nat :=
adamc@236 172 | Const n => n
adamc@236 173 | Plus e1 e2 => eval e1 + eval e2
adamc@236 174 | Mult e1 e2 => eval e1 * eval e2
adamc@236 177 Fixpoint times (k : nat) (e : exp) : exp :=
adamc@236 179 | Const n => Const (k * n)
adamc@236 180 | Plus e1 e2 => Plus (times k e1) (times k e2)
adamc@236 181 | Mult e1 e2 => Mult (times k e1) e2
adamc@236 184 Theorem eval_times : forall k e,
adamc@236 185 eval (times k e) = k * eval e.
adamc@236 187 induction e as [ | ? IHe1 ? IHe2 ]; [
adamc@236 189 | simpl; rewrite IHe1; rewrite IHe2; rewrite mult_plus_distr_l; trivial ].
adamc@236 193 Error: Expects a disjunctive pattern with 3 branches.
adamc@236 198 (** Unsurprisingly, the old proof fails, because it explicitly says that there are two inductive cases. To update the script, we must, at a minimum, remember the order in which the inductive cases are generated, so that we can insert the new case in the appropriate place. Even then, it will be painful to add the case, because we cannot walk through proof steps interactively when they occur inside an explicit set of cases. *)
adamc@236 200 Theorem eval_times : forall k e,
adamc@236 201 eval (times k e) = k * eval e.
adamc@236 202 induction e as [ | ? IHe1 ? IHe2 | ? IHe1 ? IHe2 ]; [
adamc@236 204 | simpl; rewrite IHe1; rewrite IHe2; rewrite mult_plus_distr_l; trivial
adamc@236 205 | simpl; rewrite IHe1; rewrite mult_assoc; trivial ].
adamc@236 208 (** Now we are in a position to see how much nicer is the style of proof that we have followed in most of this book. *)
adamc@236 214 Theorem eval_times : forall k e,
adamc@236 215 eval (times k e) = k * eval e.
adam@368 218 (* end thide *)
adam@491 220 (** This style is motivated by a hard truth: one person's manual proof script is almost always mostly inscrutable to most everyone else. I claim that step-by-step formal proofs are a poor way of conveying information. Thus, we might as well cut out the steps and automate as much as possible.
adam@398 222 What about the illustrative value of proofs? Most informal proofs are read to convey the big ideas of proofs. How can reading [induction e; crush] convey any big ideas? My position is that any ideas that standard automation can find are not very big after all, and the _real_ big ideas should be expressed through lemmas that are added as hints.
adamc@237 224 An example should help illustrate what I mean. Consider this function, which rewrites an expression using associativity of addition and multiplication. *)
adamc@237 226 Fixpoint reassoc (e : exp) : exp :=
adamc@237 228 | Const _ => e
adamc@237 229 | Plus e1 e2 =>
adamc@237 230 let e1' := reassoc e1 in
adamc@237 231 let e2' := reassoc e2 in
adamc@237 233 | Plus e21 e22 => Plus (Plus e1' e21) e22
adamc@237 234 | _ => Plus e1' e2'
adamc@237 236 | Mult e1 e2 =>
adamc@237 237 let e1' := reassoc e1 in
adamc@237 238 let e2' := reassoc e2 in
adamc@237 240 | Mult e21 e22 => Mult (Mult e1' e21) e22
adamc@237 241 | _ => Mult e1' e2'
adamc@237 245 Theorem reassoc_correct : forall e, eval (reassoc e) = eval e.
adam@368 246 (* begin thide *)
adam@413 249 | [ |- context[match ?E with Const _ => _ | _ => _ end] ] =>
adamc@237 253 (** One subgoal remains:
adamc@237 255 IHe2 : eval e3 * eval e4 = eval e2
adamc@237 257 eval e1 * eval e3 * eval e4 = eval e1 * eval e2
adam@433 260 The [crush] tactic does not know how to finish this goal. We could finish the proof manually. *)
adamc@237 262 rewrite <- IHe2; crush.
adamc@237 264 (** However, the proof would be easier to understand and maintain if we separated this insight into a separate lemma. *)
adamc@237 268 Lemma rewr : forall a b c d, b * c = d -> a * b * c = a * d.
adamc@237 274 Theorem reassoc_correct : forall e, eval (reassoc e) = eval e.
adam@413 277 | [ |- context[match ?E with Const _ => _ | _ => _ end] ] =>
adam@368 281 (* end thide *)
adamc@237 283 (** In the limit, a complicated inductive proof might rely on one hint for each inductive case. The lemma for each hint could restate the associated case. Compared to manual proof scripts, we arrive at more readable results. Scripts no longer need to depend on the order in which cases are generated. The lemmas are easier to digest separately than are fragments of tactic code, since lemma statements include complete proof contexts. Such contexts can only be extracted from monolithic manual proofs by stepping through scripts interactively.
adam@387 285 The more common situation is that a large induction has several easy cases that automation makes short work of. In the remaining cases, automation performs some standard simplification. Among these cases, some may require quite involved proofs; such a case may deserve a hint lemma of its own, where the lemma statement may copy the simplified version of the case. Alternatively, the proof script for the main theorem may be extended with some automation code targeted at the specific case. Even such targeted scripting is more desirable than manual proving, because it may be read and understood without knowledge of a proof's hierarchical structure, case ordering, or name binding structure.
adam@413 287 A competing alternative to the common style of Coq tactics is the%\index{declarative proof scripts}% _declarative_ style, most frequently associated today with the %\index{Isar}%Isar%~\cite{Isar}% language. A declarative proof script is very explicit about subgoal structure and introduction of local names, aiming for human readability. The coding of proof automation is taken to be outside the scope of the proof language, an assumption related to the idea that it is not worth building new automation for each serious theorem. I have shown in this book many examples of theorem-specific automation, which I believe is crucial for scaling to significant results. Declarative proof scripts make it easier to read scripts to modify them for theorem statement changes, but the alternate%\index{adaptive proof scripts}% _adaptive_ style from this book allows use of the _same_ scripts for many versions of a theorem.
adam@387 289 Perhaps I am a pessimist for thinking that fully formal proofs will inevitably consist of details that are uninteresting to people, but it is my preference to focus on conveying proof-specific details through choice of lemmas. Additionally, adaptive Ltac scripts contain bits of automation that can be understood in isolation. For instance, in a big [repeat match] loop, each case can generally be digested separately, which is a big contrast from trying to understand the hierarchical structure of a script in a more common style. Adaptive scripts rely on variable binding, but generally only over very small scopes, whereas understanding a traditional script requires tracking the identities of local variables potentially across pages of code.
adam@509 291 One might also wonder why it makes sense to prove all theorems automatically (in the sense of adaptive proof scripts) but not construct all programs automatically. My view there is that _program synthesis_ is a very useful idea that deserves broader application! In practice, there are difficult obstacles in the way of finding a program automatically from its specification. A typical specification is not exhaustive in its description of program properties. For instance, details of performance on particular machine architectures are often omitted. As a result, a synthesized program may be correct in some sense while suffering from deficiencies in other senses. Program synthesis research will continue to come up with ways of dealing with this problem, but the situation for theorem proving is fundamentally different. Following mathematical practice, the only property of a formal proof that we care about is which theorem it proves, and it is trivial to check this property automatically. In other words, with a simple criterion for what makes a proof acceptable, automatic search is straightforward. Of course, in practice we also care about understandability of proofs to facilitate long-term maintenance, which is just what motivates the techniques outlined above, and the next section gives some related advice. *)
adamc@238 294 (** * Debugging and Maintaining Automation *)
adam@367 296 (** Fully automated proofs are desirable because they open up possibilities for automatic adaptation to changes of specification. A well-engineered script within a narrow domain can survive many changes to the formulation of the problem it solves. Still, as we are working with higher-order logic, most theorems fall within no obvious decidable theories. It is inevitable that most long-lived automated proofs will need updating.
adam@367 298 Before we are ready to update our proofs, we need to write them in the first place. While fully automated scripts are most robust to changes of specification, it is hard to write every new proof directly in that form. Instead, it is useful to begin a theorem with exploratory proving and then gradually refine it into a suitable automated form.
adam@387 300 Consider this theorem from Chapter 8, which we begin by proving in a mostly manual way, invoking [crush] after each step to discharge any low-hanging fruit. Our manual effort involves choosing which expressions to case-analyze on. *)
adamc@238 302 (* begin hide *)
adamc@238 304 (* end hide *)
adamc@238 306 Theorem cfold_correct : forall t (e : exp t), expDenote e = expDenote (cfold e).
adam@368 307 (* begin thide *)
adamc@238 310 dep_destruct (cfold e1); crush.
adamc@238 311 dep_destruct (cfold e2); crush.
adamc@238 313 dep_destruct (cfold e1); crush.
adamc@238 314 dep_destruct (cfold e2); crush.
adamc@238 316 dep_destruct (cfold e1); crush.
adamc@238 317 dep_destruct (cfold e2); crush.
adamc@238 319 dep_destruct (cfold e1); crush.
adamc@238 320 dep_destruct (expDenote e1); crush.
adamc@238 322 dep_destruct (cfold e); crush.
adamc@238 324 dep_destruct (cfold e); crush.
adamc@238 327 (** In this complete proof, it is hard to avoid noticing a pattern. We rework the proof, abstracting over the patterns we find. *)
adamc@238 331 Theorem cfold_correct : forall t (e : exp t), expDenote e = expDenote (cfold e).
adamc@238 334 (** The expression we want to destruct here turns out to be the discriminee of a [match], and we can easily enough write a tactic that destructs all such expressions. *)
adamc@238 337 repeat (match goal with
adam@413 338 | [ |- context[match ?E with NConst _ => _ | _ => _ end] ] =>
adamc@238 344 (** This tactic invocation discharges the whole case. It does the same on the next two cases, but it gets stuck on the fourth case. *)
adamc@238 352 (** The subgoal's conclusion is:
adamc@238 355 (if expDenote e1 then expDenote (cfold e2) else expDenote (cfold e3)) =
adamc@238 356 expDenote (if expDenote e1 then cfold e2 else cfold e3)
adamc@238 359 We need to expand our [t] tactic to handle this case. *)
adamc@238 362 repeat (match goal with
adam@413 363 | [ |- context[match ?E with NConst _ => _ | _ => _ end] ] =>
adamc@238 365 | [ |- (if ?E then _ else _) = _ ] => destruct E
adamc@238 370 (** Now the goal is discharged, but [t'] has no effect on the next subgoal. *)
adamc@238 374 (** A final revision of [t] finishes the proof. *)
adamc@238 377 repeat (match goal with
adam@413 378 | [ |- context[match ?E with NConst _ => _ | _ => _ end] ] =>
adamc@238 380 | [ |- (if ?E then _ else _) = _ ] => destruct E
adamc@238 381 | [ |- context[match pairOut ?E with Some _ => _
adamc@238 382 | None => _ end] ] =>
adam@367 391 (** We can take the final tactic and move it into the initial part of the proof script, arriving at a nicely automated proof. *)
adamc@238 395 Theorem cfold_correct : forall t (e : exp t), expDenote e = expDenote (cfold e).
adamc@238 397 repeat (match goal with
adam@413 398 | [ |- context[match ?E with NConst _ => _ | _ => _ end] ] =>
adamc@238 400 | [ |- (if ?E then _ else _) = _ ] => destruct E
adamc@238 401 | [ |- context[match pairOut ?E with Some _ => _
adamc@238 402 | None => _ end] ] =>
adam@368 406 (* end thide *)
adam@367 408 (** Even after we put together nice automated proofs, we must deal with specification changes that can invalidate them. It is not generally possible to step through single-tactic proofs interactively. There is a command %\index{Vernacular commands!Debug On}%[Debug On] that lets us step through points in tactic execution, but the debugger tends to make counterintuitive choices of which points we would like to stop at, and per-point output is quite verbose, so most Coq users do not find this debugging mode very helpful. How are we to understand what has broken in a script that used to work?
adamc@240 410 An example helps demonstrate a useful approach. Consider what would have happened in our proof of [reassoc_correct] if we had first added an unfortunate rewriting hint. *)
adamc@240 414 Theorem confounder : forall e1 e2 e3,
adamc@240 415 eval e1 * eval e2 * eval e3 = eval e1 * (eval e2 + 1 - 1) * eval e3.
adamc@240 421 Theorem reassoc_correct : forall e, eval (reassoc e) = eval e.
adam@368 422 (* begin thide *)
adam@413 425 | [ |- context[match ?E with Const _ => _ | _ => _ end] ] =>
adamc@240 429 (** One subgoal remains:
adamc@240 433 eval e1 * (eval e3 + 1 - 1) * eval e4 = eval e1 * eval e2
adam@367 436 The poorly chosen rewrite rule fired, changing the goal to a form where another hint no longer applies. Imagine that we are in the middle of a large development with many hints. How would we diagnose the problem? First, we might not be sure which case of the inductive proof has gone wrong. It is useful to separate out our automation procedure and apply it manually. *)
adamc@240 440 Ltac t := crush; match goal with
adam@413 441 | [ |- context[match ?E with Const _ => _ | _ => _ end] ] =>
adam@509 447 (** Since we see the subgoals before any simplification occurs, it is clear that we are looking at the case for constants. Our [t] makes short work of it. *)
adamc@240 451 (** The next subgoal, for addition, is also discharged without trouble. *)
adamc@240 455 (** The final subgoal is for multiplication, and it is here that we get stuck in the proof state summarized above. *)
adam@433 459 (** What is [t] doing to get us to this point? The %\index{tactics!info}%[info] command can help us answer this kind of question. (As of this writing, [info] is no longer functioning in the most recent Coq release, but I hope it returns.) *)
adam@433 464 (* begin hide *)
adam@437 465 (* begin thide *)
adam@433 466 Definition eir := eq_ind_r.
adam@437 467 (* end thide *)
adam@433 468 (* end hide *)
adam@375 471 == simpl in *; intuition; subst; autorewrite with core in *;
adam@375 472 simpl in *; intuition; subst; autorewrite with core in *;
adamc@240 473 simpl in *; intuition; subst; destruct (reassoc e2).
adamc@240 474 simpl in *; intuition.
adamc@240 476 simpl in *; intuition.
adam@375 478 simpl in *; intuition; subst; autorewrite with core in *;
adamc@240 480 (fun n : nat =>
adamc@240 481 n * (eval e3 + 1 - 1) * eval e4 = eval e1 * eval e2) _ IHe1);
adam@375 482 autorewrite with core in *; simpl in *; intuition;
adam@375 483 subst; autorewrite with core in *; simpl in *;
adamc@240 488 A detailed trace of [t]'s execution appears. Since we are using the very general [crush] tactic, many of these steps have no effect and only occur as instances of a more general strategy. We can copy-and-paste the details to see where things go wrong. *)
adamc@240 492 (** We arbitrarily split the script into chunks. The first few seem not to do any harm. *)
adam@375 494 simpl in *; intuition; subst; autorewrite with core in *.
adam@375 495 simpl in *; intuition; subst; autorewrite with core in *.
adamc@240 496 simpl in *; intuition; subst; destruct (reassoc e2).
adamc@240 497 simpl in *; intuition.
adamc@240 498 simpl in *; intuition.
adamc@240 500 (** The next step is revealed as the culprit, bringing us to the final unproved subgoal. *)
adam@375 502 simpl in *; intuition; subst; autorewrite with core in *.
adamc@240 504 (** We can split the steps further to assign blame. *)
adam@375 511 autorewrite with core in *.
adamc@240 513 (** It was the final of these four tactics that made the rewrite. We can find out exactly what happened. The [info] command presents hierarchical views of proof steps, and we can zoom down to a lower level of detail by applying [info] to one of the steps that appeared in the original trace. *)
adam@375 517 info autorewrite with core in *.
adamc@240 519 == refine (eq_ind_r (fun n : nat => n = eval e1 * eval e2) _
adamc@240 520 (confounder (reassoc e1) e3 e4)).
adamc@240 523 The way a rewrite is displayed is somewhat baroque, but we can see that theorem [confounder] is the final culprit. At this point, we could remove that hint, prove an alternate version of the key lemma [rewr], or come up with some other remedy. Fixing this kind of problem tends to be relatively easy once the problem is revealed. *)
adam@368 526 (* end thide *)
adamc@241 528 (** Sometimes a change to a development has undesirable performance consequences, even if it does not prevent any old proof scripts from completing. If the performance consequences are severe enough, the proof scripts can be considered broken for practical purposes.
adamc@241 530 Here is one example of a performance surprise. *)
adamc@241 535 (** The central element of the problem is the addition of transitivity as a hint. With transitivity available, it is easy for proof search to wind up exploring exponential search spaces. We also add a few other arbitrary variables and hypotheses, designed to lead to trouble later. *)
adamc@239 537 Variable A : Set.
adamc@239 538 Variables P Q R S : A -> A -> Prop.
adamc@239 539 Variable f : A -> A.
adamc@239 541 Hypothesis H1 : forall x y, P x y -> Q x y -> R x y -> f x = f y.
adamc@239 542 Hypothesis H2 : forall x y, S x y -> R x y.
adam@367 544 (** We prove a simple lemma very quickly, using the %\index{Vernacular commands!Time}%[Time] command to measure exactly how quickly. *)
adamc@239 546 Lemma slow : forall x y, P x y -> Q x y -> S x y -> f x = f y.
adamc@241 549 Finished transaction in 0. secs (0.068004u,0.s)
adamc@241 555 (** Now we add a different hypothesis, which is innocent enough; in fact, it is even provable as a theorem. *)
adamc@239 557 Hypothesis H3 : forall x y, x = y -> f x = f y.
adamc@239 559 Lemma slow' : forall x y, P x y -> Q x y -> S x y -> f x = f y.
adamc@241 562 Finished transaction in 2. secs (1.264079u,0.s)
adam@445 564 %\vspace{-.15in}%Why has the search time gone up so much? The [info] command is not much help, since it only shows the result of search, not all of the paths that turned out to be worthless. *)
adam@368 566 (* begin thide *)
adamc@241 570 == intro x; intro y; intro H; intro H0; intro H4;
adamc@241 585 simple apply H2; eexact H4.
adam@367 588 This output does not tell us why proof search takes so long, but it does provide a clue that would be useful if we had forgotten that we added transitivity as a hint. The [eauto] tactic is applying depth-first search, and the proof script where the real action is ends up buried inside a chain of pointless invocations of transitivity, where each invocation uses reflexivity to discharge one subgoal. Each increment to the depth argument to [eauto] adds another silly use of transitivity. This wasted proof effort only adds linear time overhead, as long as proof search never makes false steps. No false steps were made before we added the new hypothesis, but somehow the addition made possible a new faulty path. To understand which paths we enabled, we can use the %\index{tactics!debug}%[debug] command. *)
adam@433 593 (* begin hide *)
adam@437 594 (* begin thide *)
adam@433 595 Definition deeeebug := (@eq_refl, @sym_eq).
adam@437 596 (* end thide *)
adam@433 597 (* end hide *)
adamc@241 599 (** The output is a large proof tree. The beginning of the tree is enough to reveal what is happening:
adamc@241 607 1.1.1.1.1.1.1 depth=5 apply H3
adamc@241 608 1.1.1.1.1.1.1.1 depth=4 eapply trans_eq
adam@426 609 1.1.1.1.1.1.1.1.1 depth=4 apply eq_refl
adamc@241 610 1.1.1.1.1.1.1.1.1.1 depth=3 eapply trans_eq
adam@426 611 1.1.1.1.1.1.1.1.1.1.1 depth=3 apply eq_refl
adamc@241 612 1.1.1.1.1.1.1.1.1.1.1.1 depth=2 eapply trans_eq
adam@426 613 1.1.1.1.1.1.1.1.1.1.1.1.1 depth=2 apply eq_refl
adamc@241 614 1.1.1.1.1.1.1.1.1.1.1.1.1.1 depth=1 eapply trans_eq
adam@426 615 1.1.1.1.1.1.1.1.1.1.1.1.1.1.1 depth=1 apply eq_refl
adamc@241 616 1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1 depth=0 eapply trans_eq
adamc@241 617 1.1.1.1.1.1.1.1.1.1.1.1.1.1.2 depth=1 apply sym_eq ; trivial
adamc@241 618 1.1.1.1.1.1.1.1.1.1.1.1.1.1.2.1 depth=0 eapply trans_eq
adamc@241 619 1.1.1.1.1.1.1.1.1.1.1.1.1.1.3 depth=0 eapply trans_eq
adamc@241 620 1.1.1.1.1.1.1.1.1.1.1.1.2 depth=2 apply sym_eq ; trivial
adamc@241 621 1.1.1.1.1.1.1.1.1.1.1.1.2.1 depth=1 eapply trans_eq
adam@426 622 1.1.1.1.1.1.1.1.1.1.1.1.2.1.1 depth=1 apply eq_refl
adamc@241 623 1.1.1.1.1.1.1.1.1.1.1.1.2.1.1.1 depth=0 eapply trans_eq
adamc@241 624 1.1.1.1.1.1.1.1.1.1.1.1.2.1.2 depth=1 apply sym_eq ; trivial
adamc@241 625 1.1.1.1.1.1.1.1.1.1.1.1.2.1.2.1 depth=0 eapply trans_eq
adamc@241 626 1.1.1.1.1.1.1.1.1.1.1.1.2.1.3 depth=0 eapply trans_eq
adam@367 629 The first choice [eauto] makes is to apply [H3], since [H3] has the fewest hypotheses of all of the hypotheses and hints that match. However, it turns out that the single hypothesis generated is unprovable. That does not stop [eauto] from trying to prove it with an exponentially sized tree of applications of transitivity, reflexivity, and symmetry of equality. It is the children of the initial [apply H3] that account for all of the noticeable time in proof execution. In a more realistic development, we might use this output of [debug] to realize that adding transitivity as a hint was a bad idea. *)
adam@368 632 (* end thide *)
adam@387 635 (** As aggravating as the above situation may be, there is greater aggravation to be had from importing library modules with commands like %\index{Vernacular commands!Require Import}%[Require Import]. Such a command imports not just the Gallina terms from a module, but also all the hints for [auto], [eauto], and [autorewrite]. Some very recent versions of Coq include mechanisms for removing hints from databases, but the proper solution is to be very conservative in exporting hints from modules. Consider putting hints in named databases, so that they may be used only when called upon explicitly, as demonstrated in Chapter 13.
adam@413 637 It is also easy to end up with a proof script that uses too much memory. As tactics run, they avoid generating proof terms, since serious proof search will consider many possible avenues, and we do not want to build proof terms for subproofs that end up unused. Instead, tactic execution maintains%\index{thunks}% _thunks_ (suspended computations, represented with closures), such that a tactic's proof-producing thunk is only executed when we run %\index{Vernacular commands!Qed}%[Qed]. These thunks can use up large amounts of space, such that a proof script exhausts available memory, even when we know that we could have used much less memory by forcing some thunks earlier.
adam@433 639 The %\index{tactics!abstract}%[abstract] tactical helps us force thunks by proving some subgoals as their own lemmas. For instance, a proof [induction x; crush] can in many cases be made to use significantly less peak memory by changing it to [induction x; abstract crush]. The main limitation of [abstract] is that it can only be applied to subgoals that are proved completely, with no undetermined unification variables in their initial states. Still, many large automated proofs can realize vast memory savings via [abstract]. *)
adamc@235 642 (** * Modules *)
adam@462 644 (** Last chapter's examples of proof by reflection demonstrate opportunities for implementing abstract proof strategies with stronger formal guarantees than can be had with Ltac scripting. Coq's _module system_ provides another tool for more rigorous development of generic theorems. This feature is inspired by the module systems found in Standard ML%~\cite{modules}% and OCaml, and the discussion that follows assumes familiarity with the basics of one of those systems.
adam@413 646 ML modules facilitate the grouping of %\index{abstract type}%abstract types with operations over those types. Moreover, there is support for%\index{functor}% _functors_, which are functions from modules to modules. A canonical example of a functor is one that builds a data structure implementation from a module that describes a domain of keys and its associated comparison operations.
adam@462 648 When we add modules to a base language with dependent types, it becomes possible to use modules and functors to formalize kinds of reasoning that are common in algebra. For instance, the following module signature captures the essence of the algebraic structure known as a group. A group consists of a carrier set [G], an associative binary operation [f], a left identity element [id] for [f], and an operation [i] that is a left inverse for [f].%\index{Vernacular commands!Module Type}% *)
adamc@235 651 Parameter G : Set.
adamc@235 652 Parameter f : G -> G -> G.
adam@462 653 Parameter id : G.
adamc@235 654 Parameter i : G -> G.
adamc@235 656 Axiom assoc : forall a b c, f (f a b) c = f a (f b c).
adam@462 657 Axiom ident : forall a, f id a = a.
adam@462 658 Axiom inverse : forall a, f (i a) a = id.
adam@367 661 (** Many useful theorems hold of arbitrary groups. We capture some such theorem statements in another module signature.%\index{Vernacular commands!Declare Module}% *)
adamc@235 664 Declare Module M : GROUP.
adam@462 666 Axiom ident' : forall a, M.f a M.id = a.
adam@462 668 Axiom inverse' : forall a, M.f a (M.i a) = M.id.
adam@462 670 Axiom unique_ident : forall id', (forall a, M.f id' a = a) -> id' = M.id.
adam@387 673 (** We implement generic proofs of these theorems with a functor, whose input is an arbitrary group [M]. %\index{Vernacular commands!Module}% *)
adam@387 675 Module GroupProofs (M : GROUP) : GROUP_THEOREMS with Module M := M.
adam@413 677 (** As in ML, Coq provides multiple options for ascribing signatures to modules. Here we use just the colon operator, which implements%\index{opaque ascription}% _opaque ascription_, hiding all details of the module not exposed by the signature. Another option is%\index{transparent ascription}% _transparent ascription_ via the [<:] operator, which checks for signature compatibility without hiding implementation details. Here we stick with opaque ascription but employ the [with] operation to add more detail to a signature, exposing just those implementation details that we need to. For instance, here we expose the underlying group representation set and operator definitions. Without such a refinement, we would get an output module proving theorems about some unknown group, which is not very useful. Also note that opaque ascription can in Coq have some undesirable consequences without analogues in ML, since not just the types but also the _definitions_ of identifiers have significance in type checking and theorem proving. *)
adamc@235 679 Module M := M.
adam@387 680 (** To ensure that the module we are building meets the [GROUP_THEOREMS] signature, we add an extra local name for [M], the functor argument. *)
adam@387 683 (** It would be inconvenient to repeat the prefix [M.] everywhere in our theorem statements and proofs, so we bring all the identifiers of [M] into the local scope unqualified.
adam@387 685 Now we are ready to prove the three theorems. The proofs are completely manual, which may seem ironic given the content of the previous sections! This illustrates another lesson, which is that short proof scripts that change infrequently may be worth leaving unautomated. It would take some effort to build suitable generic automation for these theorems about groups, so I stick with manual proof scripts to avoid distracting us from the main message of the section. We take the proofs from the Wikipedia page on elementary group theory. *)
adam@462 687 Theorem inverse' : forall a, f a (i a) = id.
adamc@235 689 rewrite <- (ident (f a (i a))).
adamc@235 690 rewrite <- (inverse (f a (i a))) at 1.
adamc@235 693 rewrite <- (assoc (i a) a (i a)).
adam@462 699 Theorem ident' : forall a, f a id = a.
adamc@235 701 rewrite <- (inverse a).
adam@462 707 Theorem unique_ident : forall id', (forall a, M.f id' a = a) -> id' = M.id.
adam@462 709 rewrite <- (H id).
adamc@242 715 (** We can show that the integers with [+] form a group. *)
adamc@239 721 Definition G := Z.
adamc@239 722 Definition f x y := x + y.
adam@462 723 Definition id := 0.
adamc@239 724 Definition i x := -x.
adamc@239 726 Theorem assoc : forall a b c, f (f a b) c = f a (f b c).
adam@462 729 Theorem ident : forall a, f id a = a.
adam@462 730 unfold f, id; crush.
adam@462 732 Theorem inverse : forall a, f (i a) a = id.
adam@462 733 unfold f, i, id; crush.
adamc@242 737 (** Next, we can produce integer-specific versions of the generic group theorems. *)
adam@387 739 Module IntProofs := GroupProofs(Int).
adamc@242 744 : forall e' : Int.G, (forall a : Int.G, Int.f e' a = a) -> e' = Int.e
adam@367 747 Projections like [Int.G] are known to be definitionally equal to the concrete values we have assigned to them, so the above theorem yields as a trivial corollary the following more natural restatement: *)
adam@462 749 Theorem unique_ident : forall id', (forall a, id' + a = a) -> id' = 0.
adam@368 750 (* begin thide *)
adam@368 753 (* end thide *)
adam@475 755 (** As in ML, the module system provides an effective way to structure large developments. Unlike in ML, Coq modules add no expressiveness; we can implement any module as an inhabitant of a dependent record type. It is the second-class nature of modules that makes them easier to use than dependent records in many cases. Because modules may only be used in quite restricted ways, it is easier to support convenient module coding through special commands and editing modes, as the above example demonstrates. An isomorphic implementation with records would have suffered from lack of such conveniences as module subtyping and importation of the fields of a module. On the other hand, all module values must be determined statically, so modules may not be computed, e.g., within the definitions of normal functions, based on particular function parameters. *)
adamc@243 758 (** * Build Processes *)
adam@433 760 (* begin hide *)
adam@437 761 (* begin thide *)
adam@437 776 (* end thide *)
adam@433 777 (* end hide *)
adamc@243 779 (** As in software development, large Coq projects are much more manageable when split across multiple files and when decomposed into libraries. Coq and Proof General provide very good support for these activities.
adam@435 781 Consider a library that we will name [Lib], housed in directory <<LIB>> and split between files <<A.v>>, <<B.v>>, and <<C.v>>. A simple %\index{Makefile}%Makefile will compile the library, relying on the standard Coq tool %\index{coq\_makefile}%<<coq_makefile>> to do the hard work.
adamc@243 784 MODULES := A B C
adamc@243 785 VS := $(MODULES:%=%.v) adamc@243 786 adamc@243 787 .PHONY: coq clean adamc@243 788 adamc@243 789 coq: Makefile.coq adam@369 790$(MAKE) -f Makefile.coq
adamc@243 792 Makefile.coq: Makefile $(VS) adamc@243 793 coq_makefile -R . Lib$(VS) -o Makefile.coq
adam@369 796 $(MAKE) -f Makefile.coq clean adamc@243 797 rm -f Makefile.coq adamc@243 798 >> adamc@243 799 adam@435 800 The Makefile begins by defining a variable <<VS>> holding the list of filenames to be included in the project. The primary target is <<coq>>, which depends on the construction of an auxiliary Makefile called <<Makefile.coq>>. Another rule explains how to build that file. We call <<coq_makefile>>, using the <<-R>> flag to specify that files in the current directory should be considered to belong to the library [Lib]. This Makefile will build a compiled version of each module, such that <<X.v>> is compiled into <<X.vo>>. adamc@243 801 adam@433 802 Now code in <<B.v>> may refer to definitions in <<A.v>> after running adamc@243 803 [[ adamc@243 804 Require Import Lib.A. adam@367 805 ]] adam@433 806 %\vspace{-.15in}%Library [Lib] is presented as a module, containing a submodule [A], which contains the definitions from <<A.v>>. These are genuine modules in the sense of Coq's module system, and they may be passed to functors and so on. adamc@243 807 adam@433 808 The command [Require Import] is a convenient combination of two more primitive commands. The %\index{Vernacular commands!Require}%[Require] command finds the <<.vo>> file containing the named module, ensuring that the module is loaded into memory. The %\index{Vernacular commands!Import}%[Import] command loads all top-level definitions of the named module into the current namespace, and it may be used with local modules that do not have corresponding <<.vo>> files. Another command, %\index{Vernacular commands!Load}%[Load], is for inserting the contents of a named file verbatim. It is generally better to use the module-based commands, since they avoid rerunning proof scripts, and they facilitate reorganization of directory structure without the need to change code. adamc@243 809 adam@433 810 Now we would like to use our library from a different development, called [Client] and found in directory <<CLIENT>>, which has its own Makefile. adamc@243 811 adamc@243 812 << adamc@243 813 MODULES := D E adamc@243 814 VS :=$(MODULES:%=%.v)
adam@369 819 $(MAKE) -f Makefile.coq adamc@243 820 adamc@243 821 Makefile.coq: Makefile$(VS)
adamc@243 822 coq_makefile -R LIB Lib -R . Client $(VS) -o Makefile.coq adamc@243 823 adamc@243 824 clean:: Makefile.coq adam@369 825$(MAKE) -f Makefile.coq clean
adam@435 829 We change the <<coq_makefile>> call to indicate where the library [Lib] is found. Now <<D.v>> and <<E.v>> can refer to definitions from [Lib] module [A] after running
adam@433 833 %\vspace{-.15in}\noindent{}%and <<E.v>> can refer to definitions from <<D.v>> by running
adam@433 837 %\vspace{-.15in}%It can be useful to split a library into several files, but it is also inconvenient for client code to import library modules individually. We can get the best of both worlds by, for example, adding an extra source file <<Lib.v>> to [Lib]'s directory and Makefile, where that file contains just this line:%\index{Vernacular commands!Require Export}%
adamc@243 839 Require Export Lib.A Lib.B Lib.C.
adam@367 841 %\vspace{-.15in}%Now client code can import all definitions from all of [Lib]'s modules simply by running
adam@367 845 %\vspace{-.15in}%The two Makefiles above share a lot of code, so, in practice, it is useful to define a common Makefile that is included by multiple library-specific Makefiles.
adam@433 849 The remaining ingredient is the proper way of editing library code files in Proof General. Recall this snippet of <<.emacs>> code from Chapter 2, which tells Proof General where to find the library associated with this book.
adamc@243 859 To do interactive editing of our current example, we just need to change the flags to point to the right places.
adamc@243 864 ; '(coq-prog-args '("-I" "/path/to/cpdt/src"))
adamc@243 865 '(coq-prog-args '("-R" "LIB" "Lib" "-R" "CLIENT" "Client"))