COMS W3261
Computer Science Theory
Lecture 14: November 7, 2012
Algorithms and the Church-Turing Thesis

Outline

• Definition of algorithm
• Programming techniques for Turing machines
• Variants of Turing machines
• The Church-Turing thesis
• The Diagonalization Language L_d is not RE

1. Definition of Algorithm

• Surprisingly, there is no universally agreed-upon definition for the term "algorithm". Informally, we can think of an algorithm as a collection of well-defined instructions for carrying out some task.
• In The Art of Computer Programming, Donald Knuth states that an algorithm should have five properties:
1. Finiteness: An algorithm must always terminate after a finite number of steps.
2. Definiteness: Each step of an algorithm must be precisely defined.
3. Input: An algorithm has zero or more inputs.
4. Output: An algorithm has one or more outputs, quantities which have a specified relation to the inputs.
5. Effectiveness: All of the operations to be performed in an algorithm can be done exactly and in a finite length of time.
• In this course we will use a Turing machine that halts on all inputs as the definition of an algorithm. The term decider is sometimes used for such a Turing machine.
• A language L that can be recognized by an algorithm is said to be recursive.
• If a language L is recursive, we say L is decidable.
• If a language L is not recursive, we say L is undecidable.
• In general, a Turing machine need not halt all inputs. An input on which a Turing machine never halts is not in the language defined by the Turing machine.
• A language L that can be recognized by a Turing machine is said to be recursively enumerable.
• The term Turing-recognizable language is sometimes used for a recursively enumerable language.
• Note that a language may be undecidable because it is not recursive but is recursively enumerable or because it is not recursively enumerable.

2. Programming techniques for Turing machines

• Turing machines are exactly as powerful as conventional computers.
• To make the behavior of a Turing machine clearer, we can use the finite-state control of a Turing machine to hold a finite amount of data. One way to do this is to use states with multiple fields, where one field represents a position in the Turing machine program, and the other fields hold data elements. The number of fields in a state is always finite.
• Another way to make the behavior of a Turing machine clearer, is to think of the tape as having several tracks.
• We can also group states into "subroutines". A subroutine has its own start state, and another state which can serve as a "return" state.

3. Variants of Turing machines

• There is no standard definition of a Turing machine. In this course we will use the definition in HMU, which assumes that a TM always halts when it enters an accepting state.
• Some books, such as Sipser, define a TM as having a semi-infinite input tape with one accept state and one reject state. The TM halts if it enters either the accept or reject state. A Sipser-style TM is equivalent to an HMU-style TM in computational capability.
• We can generalize a Turing machine to have multiple tapes. This extension does not add any new computational power to a Turing machine.
• We can allow a Turing machine to be nondeterministic by allowing it to make a choice of moves in a given state on a given tape symbol. This extension does not add any new computational power to a Turing machine.
• A Turing machine with a semi-infinite tape (one that is infinite only to the right) is just as powerful as a regular Turing machine.
• A pushdown automaton with two stacks is just as powerful as a regular Turing machine.
• A two-counter machine is just as powerful as a regular Turing machine.

4. The Church-Turing Thesis

• A Turing machine can compute a function from an input to an output by reading the input, making a sequence of moves, and then halting, leaving only the output of the function on the tape.
• A recursive function is one that can be computed by a Turing machine that halts on all inputs.
• A partial-recursive function is on that can be computed by a Turing machine that need not halt on all inputs. The output of the function on an input for which the Turing machine does not halt is said to be undefined.
• The Church-Turing thesis says that any general way to compute will allow us to compute only the partial-recursive functions. The Church-Turing thesis is unprovable because there is no precise definition for "any general way to compute."
• An informal way of expressing the Church-Turing thesis is that any function that can be effectively computed can be computed by a Turing machine.

5. The Diagonalization Language L_d is not Recursively Enumerable

• We can enumerate all binary strings.
• We can enumerate all Turing machines.
• We define L_d, the diagonalization language, as follows:
1. Let w₁, w₂, w₃,... be an enumeration of all binary strings.
2. Let M₁, M₂, M₃,... be an enumeration of all Turing machines.
3. Let L_d = { w_i | w_i is not in L(M_i) }.
• Theorem: L_d is not a recursively enumerable language.
• Proof:
• Suppose L_d = L(M_i) for some TM M_i.
• This gives rise to a contradiction. Consider what M_i will do on the input w_i.
• If M_i accepts w_i, then by definition w_i cannot be in L_d.
• If M_i does not accepts w_i, then by definition w_i is in L_d.
• Since w_i can neither be in L_d nor not be in L_d, we must conclude there is no Turing machine that can define L_d.

6. Reading Assignment

• HMU: Ch. 8, Sect. 9.1

7. Practice Problems

1. Informally describe how a single-tape Turing machine can simulate a two-tape Turing machine.
2. Informally describe how a deterministic Turing machine can simulate a nondeterministic Turing machine.
3. Informally describe how a pushdown automaton with two stacks can simulate a Turing machine.
4. Given a suitable encoding of input strings and Turing machines, show that L = { w_i | w_i is not accepted by M_2i } is not recursively enumerable.