Lecture 0: Introduction

COMS E6998 Formal Verification of System Software
Fall 2018
Ronghui Gu

1. This course is about …

• Techniques for improving reliability of computer systems
  • Applicable to both software and hardware
  • Focus on software
• Theories about formal verification (especially about the theorem proving)
• How to use Coq to realize these theories?

2. Why should we care?

• Software systems run everywhere and even control our lives
• We have to trust them but they are not trustworthy due to program errors

  • Software Errors Cost Economy $312 Billion Annually. – Cambridge[2013]

  • Toyota’s killer firmware (stack overflow in the assembly module)
  • TheDao (decentralized autonomous organization) attack (double-spend attack), $50M lost

  /*
    @pre forall i, self.balance[i] < fund /\ self.balance[i] >= 0
    @post fund - fund_p <= self.balance[account] /\ self.balance_p[account] = 0
  */
  void withdraw(account) {
      uint balance = self.balance[account];
      if (balance <= 0) {
        return;
      }
      self.fund -= balance;
      receiverHandlePayment (account, balance);
      self.balance[account] = 0;
  }

3. Goal of Formal Verification

• Prove (or verify) the absence of program errors
• Easy? NO!
  • Finite code often has an infinite set of behaviors:
    sort (int* a)
    Infinite number of a-inputs and outputs
  • What does it even mean for sort to be correct?
• A number of ways to approach the problem

4. Methods to Achieve the Goal

4.1 Testing?

• Can we use testing to prove correctness/incorrectness?
• No! Need an infinite testsuit

Program testing can be used to show the presence of bugs, but never to show their absence. – Edsger Dijkstra

4.2 Algorithmic Verification?

• Can we come up with an algorithm to prove correctness/incorrectness for any program automatically?
• Create a program av that inputs another program p and after finite time outputs false if p has a bug for some input, or true otherwise.

  bool av (program p) {
    if (p has bugs) {
      return false;
    }
    return true;
  }

• NO! Because of the halting problem [Alan Turing 1936]: if av exists then there is a paradox; thus av can’t exist.
• Proof.
  Let “bug”=infinite loop=infloop(). If p is a program then let 'p(p) be the program that when run, it will execute p on input p. Assume av exists. Create:
  paradox(p:Prog) = if av(’p(p)) then infloop() else true
  What does paradox(paradox) do?
  1. If paradox(paradox) loops forever then av(’paradox(paradox))=false, thus paradox(paradox) returns true. Contradiction
  2. If paradox(paradox) returns true, i.e., av(’paradox(paradox))=true, thus paradox(paradox) loops forever. Contradiction

4.3 Deductive Verification?

• Can we have a mathematical proof system to prove correctness/incorrectness for all programs?
• Create a system L of logical axioms and rules, such that for any program p we can prove either
  • p has a bug for some input
  • p has no bug for any input
• If such a system L exists then we can create a fully automatic verification algorithm (simply systematically explore all logical derivations and eventually, in finite time, derive “p has a bug” or “p has no bug”.)
• NO! Kurt Gödel proved in 1931 （incompleteness theorems）that no such logical system exists.
• Hilbert’s program
  • A formulation of all mathematics; in other words all mathematical statements should be written in a precise formal language, and manipulated according to well defined rules.
  • Completeness: a proof that all true mathematical statements can be proved in the formalism.
  • Consistency: a proof that no contradiction can be obtained in the formalism of mathematics.
  • Conservation: a proof that any result about “real objects” obtained using reasoning about “ideal objects” (such as uncountable sets) can be proved without using ideal objects.
  • Decidability: there should be an algorithm for deciding the truth or falsity of any mathematical statement.

5. Holy Grail of Formal Verification

Soundness: If the verification method reports no failure, then the program under examination has no bug Completeness: If the verification method reports a failure, then the program under examination has a bug Termination: The verification method will terminate, giving back an answer.

• Pick TWO. Having all three is theoretically impossible.
• Usually verification systems pick soundness and termination.

6. Methods for Formal Verification

However not all is lost! Sound and terminating systems can prove the correctness of virtually every program we would care about.

The scientific community continuously pushes the limits of these systems to extreme levels!

• Algorithmic verification: model checking, abstract interpretation, static analysis.

  • create a model of the program in a decidable framework (finite state system, pushdown system)
  • usually: semi-automatic (semi-manual) model creation
  • automated model verification

• Deductive verification: Hoare Logic, Separation Logic, Rely/Guarantee Reasoning

  • create a correctness proof of the program in a logic (with axioms and logical rules)
  • usually: semi-automatic (semi-manual) specification construction and proof construction
  • automated proof checking

• This course will focus on the second approach

  • We will manually prove programs correct using pen and paper proofs.
  • We will use software that provide some automation to make this easier and more rigorous.

• Success stories from the mid-1990s

  • Paris metro line 14 (1998, combination)
  • Flight control software of A380 (2005, abstract interpretation)
  • seL4. verified sequential micro-kernel (2009, separation logic)
  • CompCert, a verified C compiler (2005 ~ present, simulation, Coq)
  • MS windows drivers (2010, model checking)
  • CertiKOS, a verified concurrent OS kernel (2016 ~ present, simulation, Coq)