2011年1月17日星期一

A Simple Lambda Calculus Evaluator - III

In the previous post, we translated the lambda calculus expressions into a syntax tree using some compiler generate tools. Now, we are about to evaluate it by traversing the syntax tree. First, I will introduce the evaluation algorithm in this post. Then, the implementation is presented along with some code snippet. Finally, an interactive interface which combines all the parts together is implemented. The full source code can be checkout here.

Evaluation Algorithm
For the evaluation algorithm, we have a syntax tree as the input, and as a result we are also going to output a syntax tree which is evaluated from the original one.

Let's consider the simple case first, an identifier node which doesn't have any child node, the node should evaluate to itself because no reduction can be applied to it. For the abstraction node, which has an identifier child node and another expression child node, which it is a syntax tree itself. The expression child node will be evaluated first and the resulting node will be set as the expression child node. The other part will stay the same. For an application node, which has two expression child nodes, both of them will be evaluated first and replaced by the resulting nodes. Finally according to the semantic of lambda application, a beta-reduction should be performed on the application node because the beta-reduction captures the idea of function application.

From the analysis above, we can work out a recursive algorithm which does a deep-first traversal on the syntax tree and evaluates the syntax tree in a bottom-up manner. Here is the psuedocode:
evaluate(tree):
    if tree is identifier node:
        return tree;
    else if tree is abstraction node:
        tree->right = evaluate(tree->right);
        return tree;
    else if tree is application node:
        tree->left = evaluate(tree->left);
        tree->right = evaluate(tree->right);
        tree = betaReduction(tree);
        return tree;

Now, the problem is reduced to the beta-reduction of a lambda calculus expression. In the first post, we learned that the beta-reduction can be simple defined in terms of substitution, so again our problem is reduced to implement the substitution algorithm of expressions. We can simply derive a recursive algorithm from the substitution rules defined in the first post:
substitute(tree, var, sub):
    if tree is identifier node:
        if tree.name equals var.name:
            return sub;
        else:
            return tree;
    else if tree is application node:
        tree->left = substitute(tree->left, var, sub);
        tree->right = substitute(tree->right, var, sub);
        return tree;
    else if tree is abstraction node:
        if tree.name not equals var.name:
            if tree->left is a free variable in sub:
                tree = alpahReduction(tree);
            tree->right = substitute(tree->right, var, sub);
        return tree;

Pay attention to the abstraction node case, we check the conditions in the last rule. If the condition is not met, an alpha-reduction is applied to the syntax tree before doing substitution.

There are two remaining problems: alpha-conversion and free variables. The definition of free variables in the first post is very straightforward, so I won't duplicate the algorithm here. With regard to alpha-conversion, the process consists of the following steps:
  1. Find the set of free variables of the expression child node;
  2. Pick a new identifier name which is different from the old one and not in the free variable set in step 1;
  3. Substitute all free occurrences of the identifier in expression child node with the new identifier;
  4. Replace the old identifier node with the new one.
In step 3, the substitute procedure above will be used. All the algorithms used for this evaluator has been presented, now we are going to write the codes.

Implementation
It's time to implement the evaluator. The code snippet of different algorithms will be shown in this section. To emphasize the most important part, the error handling and memory release stuff is ignored.

First and foremost, let's see the function that finds the free variable set of an expression.
static VarSet * FV(TreeNode *expr) {
    VarSet* set = NULL;
    VarSet* set1 = NULL;
    VarSet* set2 = NULL;
    switch(expr->kind) {
        case IdK:
            set = newVarSet();
            addVar(set,expr->name);
            break;
        case AbsK:
            set = FV(expr->children[1]);
            deleteVar(set,expr->children[0]->name);
            break;
        case AppK:
            set = newVarSet();
            set1 = FV(expr->children[0]);
            set2 = FV(expr->children[1]);
            unionVarSet(set,set1,set2);
            break;
        default:
            fprintf(errOut,"Unknown expression type.\n");
    }
    return set;
}
The VarSet is a data structure that represents a set of variables. It uses an inner hashtable to store the variables. The functions, newVarSet, addVar, deleteVar, unionVarSet, are very intuitive by their names. Refer to file varset.c for information about the VarSet implementation.

The code snippet for alpha-conversion is like:
TreeNode * alphaConversion(TreeNode *expr) {
    VarSet* set = FV(expr->children[1]);
    char * name = strdup(expr->children[0]->name);
    // pick a new name
    while(strcmp(name,expr->children[0]->name)==0 ||  contains(set,name)==1) {
        char lastchar = name[strlen(name)-1];
        name[strlen(name)-1] = 'a' + (lastchar+1-'a')%('z'-'a'+1);
    }
    TreeNode *var = newTreeNode(IdK);
    var->name = name;
    TreeNode *result = substitute(expr->children[1], expr->children[0], var);
    
    expr->children[1] = result;
    expr->children[0] = var;
    return expr;
}
The method for picking a new variable name is: replace the last character of the variable by a letter comes after it in the alphabet. This works for most cases though it may failed if all the attempted names are used up.

Here comes the most important function, substitute, which is used by both alpha-conversion and beta-reduction:
static TreeNode *substitute(TreeNode *expr, TreeNode *var, TreeNode *sub) {
    const char * parname = NULL;
    TreeNode * result = NULL;
    switch(expr->kind) {
        case IdK:
            if(strcmp(expr->name,var->name)==0) {
                return sub;
            }else {
                return expr;
            }
        case AbsK:
            parname = expr->children[0]->name;
            if(strcmp(parname,var->name)!=0) {
                VarSet* set = FV(sub);
                while(contains(set,parname)) {  // do alpha conversion
                    expr = alphaConversion(expr);
                    parname = expr->children[0]->name;
                }
                result = substitute(expr->children[1],var,sub);
                expr->children[1] = result;
            }
            return expr;
        case AppK:
            result = substitute(expr->children[0],var,sub);
            expr->children[0] = result;
            result = substitute(expr->children[1],var,sub);
            expr->children[1] = result;
            return expr;
        default:
            fprintf(errOut,"Unknown expression type.\n");
    }
    return expr;
}
It recursively applies itself to the child nodes of the expression.

Now, we can deal with the beta-reduction:
TreeNode * betaReduction(TreeNode *expr) {
    TreeNode* left = expr->children[0];
    if(left->kind==IdK || left->kind==AppK) {
        return expr;
    }else if(left->kind==AbsK) {
        TreeNode* result = substitute(left->children[1],left->children[0],expr->children[1]);
        return result;
    }
    return expr;
}

Finally, all the build blocks are ready. We can implement the main evaluation function:
TreeNode * evaluate(TreeNode *expr) {
    if(expr!=NULL) {
        switch(expr->kind) {
            case IdK:
                return expr;
            case AbsK:
                expr->children[1] = evaluate(expr->children[1]);
                return expr;
            case AppK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                return betaReduction(expr);
            default:
                fprintf(errOut,"Unkown expression kind.\n");
        }
    }
    return expr;
}

Evaluator Driver
Different parts, including scanner, parser and evaluator, has been implemented separately. We need a driver method to combine them all, and it should provides an interactive interface which reads a lambda calculus expression from user input and output the evaluated expression in a human-readable format.

Here is the code:
FILE* in;
FILE* out;
FILE* errOut;

TreeNode * tree = NULL;    // used in the parser

#define BUFF_SIZE 255

int main(int argc, char* argv[]) {
    in = stdin;
    out = stdout;
    errOut = stderr;

    char buff[BUFF_SIZE];

    fprintf(out,"Welcome to Lambda Calculus Evaluator.\n");
    fprintf(out,"Press Ctrl+C to quit.\n\n");
    while(1) {
        fprintf(out,"> ");
        fgets(buff,BUFF_SIZE-1,in);
        yy_scan_string(buff);
        yyparse();

        tree = evaluate(tree);
        fprintf(out,"-> ");
        printExpression(tree);
        deleteTreeNode(tree);
        tree=NULL;
        yy_delete_buffer();
        buff[0] = EOF;
        fprintf(out,"\n\n");
    }
    return 0;
}
We set the standard input as the input and standard output as the output. The library function fgets() is used to read the user input. We use yy_scan_string() to feed the user input to the parser, so the parser will read input from this buffer rather than the standard input. The yyparse() function is the interface exposed by Yacc to run the parser. The printExpression() function is a utility that prints the expression in a human-readable format.

Let's try some expressions:
$ ./main
Welcome to Lambda Calculus Evaluator.
Press Ctrl+C to quit.

> a
-> a

> (lambda x (lambda y y) a)
-> (lambda x a)

> (lambda x x) a
-> a

Conclusion
This series of post introduces how to write a very simple evaluator for lambda calculus. It covers the topics about scanner, parser and evaluator. In order to keep it really pure, there are some limitations of the syntax, such as not constant support, cannot place parentheses around expressions to change the evaluation order from left to right. However, it is rather easy to support those features by extending the syntax. We can do it in the future.

References
1. Lambda Calculus on Wikipedia.
2. The LEX & YACC Page.
3. A lecture notes about lambda calculus, including a lambda calculus evaluator.

A Simple Lambda Calculus Evaluator - II

This is the second part of a series of posts. In this post, I am going to define the concrete syntax for lambda calculus and generate the scanner and parser using Lex and Yacc(actually, they are Flex and Bison on my machine). Source code is available here.

Concrete Syntax
I am trying to keep this lambda calculus evaluator really simple and pure. So before bringing you the concrete syntax, I'd like to set some restrictions on the implementation:
  • Variable can only be a single lower case character from the alphabet, a...z.
  • Since natural numbers and booleans can be modeled in lambda calculus, no constant definition is provided in the syntax.
  • Every function is anonymous and only one argument is allowed.
  • All application expressions are left associative.
However, we can get rid of these restrictions very easily by extending the syntax. Derived from the formal definition of lambda calculus, we have following concrete syntax:
     Expression := identifier |
                            (lambda identifier Expression) |
                            Expression Expression

The first branch defines the variables. The second one defines the abstraction, and the third defines the application. The applications are not enclosed in parentheses, and they will be applied from left to right, so this syntax doesn't support the definitions of natural numbers. Just leave as it is, and we can extend it later.

Flex & Bison
The scanner and parser can be written by hand, or generated by compiler generating tools. The later method is used in this evaluator. Lex is a program that generates scanner, and it is commonly used with Yacc, which is parser generator. In this lambda calculus evaluator, the open source versions, Flex and Bison, are used. For more information, please refer to this page.

Scanner
The syntax is so simple that only two tokens are defined. The token LAMBDA is for keyword "lambda", and the ID is for variables. Here is a code snippet of the Flex definition:
// file: scanner.l

lambda        "lambda"
identifier    [a-z]
whitespace    [ \t]+
newline       [\r\n]+

%%

{lambda}        {return LAMBDA;}
{identifier}    {return ID;}

{whitespace}    ;
{newline}       ;

.               {return yytext[0];}

%% 
Flex will work with Bison, so the token values of LAMBDA and ID are defined in the Bison definition file. For unmatched character, the ASCII code value will be returned as its token value, so '(' and ')' can be matched in the bison rules. We will see that in the next section.

Parser
Grammar rules
The BNF grammar is used to describe the syntax in Bison. The above concrete syntax can be simple written as:
expression    : ID
              | '(' LAMBDA ID expression ')'
              | expression expression
              ;
However, this may cause a reduce/shift conflict for Bison. The parser may be confused by whether do a reduce action or shift, when dealing with the third branch "expression : expression expression". The default action taken by Bison is shift. So the applications will be right associative rather than left associative. We need to revise the syntax to make it left associative:
// file: parser.y

expression_list    : expression_list expression
                   | expression
                   ;

expression         : ID
                   | '(' LAMBDA ID expression_list ')'
                   ;

Tree structure
Before adding the semantic actions for each grammar rule, I'd like to introduce the tree structure constructed after parsing. Our goal is to translate the lambda calculus expressions into a single root tree, which will be passed as an input to the evaluation process. According to the concrete syntax, we have three kinds of expressions: identifier, abstraction and application. Here is the definition:
// file: globals.h

/* expression types */
typedef enum { IdK, AbsK, AppK } ExprKind;
For the identifier node, only the name need to be saved in the node. For abstraction node, we need to save two child nodes, one for the identifier and the other for the expression. For application node, we also need to save two child nodes, one for the first expression and the other for the second. So we have the tree node definition as follows:
// file: globals.h

#define MAXCHILDREN 2
/* tree nodes */
typedef struct treeNode {
    ExprKind kind;
    char * name;    // only for IdK
    struct treeNode * children[MAXCHILDREN];
} TreeNode;
Every lambda calculus expression can be represented in such a tree structure. Take (lambda x x) a as an example, the syntax tree would look like:
Syntax tree:
               AppK
                /\
               /  \
            AbsK  IdK("a")
             /\
            /  \
      IdK("x") IdK("x")
In Bison, each semantic action returns a value and the default value is int. In order to construct the syntax tree, we need to set the type to TreeNode. This is a code snippet of the declaration section, with token definitions for the scanner:
// file: parser.y

%{
...

#define YYSTYPE TreeNode *

extern YYSTYPE tree;
%}

%token  LAMBDA
%token  ID

%start expression_list

%%

The tree variable will point to the root node of the resulting syntax tree.

Constructing syntax tree
Now, we are going to add some C statements in the semantic actions to construct the syntax tree. It is really simple and intuitive. Here is the full list of code:
// file: parser.y

%%

expression_list : expression_list expression
                    {
                        $$ = newTreeNode(AppK);
                        $$->children[0] = $1;
                        $$->children[1] = $2;
                        tree = $$;
                    }
                |
                 expression
                    {
                        tree = $1;
                    }
                ;

expression      : ID    
                    {
                        $$ = newTreeNode(IdK);
                        $$->name = stringCopy(yytext);
                    }
                | '(' LAMBDA ID 
                    {
                        $$ = newTreeNode(IdK);
                        $$->name = stringCopy(yytext);
                    } 
                    expression_list ')'
                    {
                        $$ = newTreeNode(AbsK);
                        $$->children[0] = $4;
                        $$->children[1] = $5;
                    }
                ;

%%


For that last rule, we add a semantic action inside the rule, as a result a temporary node which represents the ID expression will be created by Bison. So the index number should be 4 and 5 instead of 3 and 4 in the last semantic action, which may confuse beginners of Bison. The newTreeNode(ExprKind kind) function is a utility to create a tree node of type kind in the heap.

After this scan and parse step, a syntax tree represents the lambda calculus expressions is ready for evaluation. In the next post, I will show you the evaluation part of this simple lambda calculus evaluator.

A Simple Lambda Calculus Evaluator - I

Recently, I wrote a very simple lambda calculus evaluator in C, with scanner and parser generated by Lex and Yacc. This series of posts introduces how to implement such a lambda calculus evaluator. This first post gives a brief introduction of the lambda calculus, including its definition and reductions, which are the essentials to the evaluation algorithms. In the second post, the concrete syntax for the lambda calculus expression and its scanner and parser are presented. The evaluation process is described in the third post. You can checkout the source code from here.

This post gives a brief introduction of lambda calculus. Most part is based on the lambda calculus page on Wikipedia. I am not going to write a formal article about lambda calculus, so only the concepts related to this evaluator are mentioned here. For more detailed information, please refer to it.

Lambda Calculus
Lambda calculus, also written as λ-calculus, is accepted as the foundation for functional programming languages. It is introduced by Alonzo Church in 1930s to model the simple semantics for computation with functions in mathematics.

Function can be considered as a black box, which accepts some inputs and sends back some output. Consider the following examples. Function increase(x) = x + 1 takes a single input, x, and returns the result by adding 1 to x. Function sum(x,y) = x + y takes two inputs, x and y, and returns the sum of them, x+y. Some ideas in lambda calculus can be introduced by make some observations of the functions.

Firstly, the function can be anonymous. That means we do not need to name the function. The function increase(x) = x + 1 can be rewritten as (x) -> x + 1.

Secondly, the name for function's argument is irrelevant. That is, choosing different names for a function makes no difference. So function (x,y) -> x + y is equivalent to (a,b) -> a + b.

Finally, any function that requires two inputs can be redefined into an equivalent function that accepts a single input, and returns another function, that in turn accepts a single input. This can be extended to three or more inputs. So function (x,y) -> x + y can be redefined as (x) -> (y) -> x + y.

The lambda calculus provides a way to model functions. Since functions can be anonymous, lambda calculus doesn't provide a means to name a function. Since the names of arguments are irrelevant, the functions with only different argument names are equivalent. Since all functions expecting more than one arguments can be transformed to equivalent functions expecting one argument, lambda calculus creates functions that accepts one single argument.

Here comes the formal definition of the lambda calculus from Wikipedia:
Lambda expressions are composed of:
    variables v1, v2, ..., vn
    the abstraction symbols λ and .
    parentheses ()
The set of lambda expressions, Λ, can be defined recursively:
    1. If x is a variable, then x ∈ Λ
    2. If x is a variable and M ∈ Λ, then (λx. M) ∈ Λ
    3. If M, N ∈ Λ, then (M N) ∈ Λ
Instances of rule 2 are known as abstractions and rule 3 are known as applications.

With this definition, the before mentioned functions can be defined as (λx. x+1) and (λx. (λy. x+y)).

Reductions
Suppose we have two lambda expressions: (λx. x+1) and a. If we apply the first to the second, we will have (λx. x+1) a. According to the semantic, the result should be a+1. This is called a reduction. Actually, it is a β-reduction. There are three kinds of reduction:
  • α-conversion: changing bound variables;
  • β-reduction: applying functions to their arguments;
  • η-conversion: which captures a notion of extensionality.
Only α-conversion and β-reduction will be covered in this post, with some other necessary terms.

Free and bound variables
Variables that only exist in the scope of a lambda abstraction are said to be bound. All others are called free. For example, y is a bould variable and x is free in expression: (λy. x + y).

The set of free variables of expression M is denoted as FV(M). It is defined by recursion as follows:
  1. FV(x)    = {x}, where x is a variable
  2. FV(λx.M) = FV(M) - {x}
  3. FV(M N)  = FV(M) ∪ FV(N)
α-conversion
Alpha-conversion allows bound variable names to be changed. For example, alpha-conversion of (λx. x) could be (λy. y).

You need to pay attention to the "bound" word, which has extended meanings. First, when alpha-converting an abstraction, only the variables bound to the same abstraction could be changed. For example, alpha-conversion of (λx. (λx. x)) could result in (λy. (λx. x)), but not (λy. (λx. y)). Second, after alpha-conversion, the new argument variable should not be captured by a different abstraction. For example, alpha-conversion of (λx. (λy. x)) could not result in (λy. (λy. y)).

Substitution
Substitution, written as E[V := E'], is the process of replacing all free occurrences of the variable V by expression E'. It is defined by recursion as follows:
    x[x := N]        ≡ N
    y[x := N]        ≡ y, if x ≠ y
    (M1 M2)[x := N]  ≡ (M1[x := N]) (M2[x := N])
    (λy. M)[x := N]  ≡ λy. (M[x := N]), if x ≠ y and y ∉ FV(N)
The y ∉ FV(N) condition in the last rule is important. Without it, the variable y, which is free in expression N, will be bound in the result abstraction after substitution. For example, it is not correct for (λx. y)[y := x] to result in (λx. x). For this case, an alpha-conversion should be applied to the expression (λx. y) before substitution, which results in (λz. y).

β-reduction
Beta-reduction captures the idea of function application. It is defined in terms of substitution: the beta-reduction of ((λV. E) E') is E[V := E'].

All the rules defined in this section is essential to lambda calculus. And they are also the basic elements for evaluation algorithm which will be presented in part III.

Encoding Datatypes
From our mathematics knowledge of middle school, we know that functions are usually applied to numbers. However, lambda calculus doesn't have any numbers? That's true! The pure lambda calculus doesn't define any data types. But we can model some data types using this basic lambda calculus. Two data types, boolean and natural number, will be introduced in this section.

Natural Number
The most common way to define natural numbers is as follows:
    0 := (λf. (λx. x))
    1 := (λf. (λx. f x))
    2 := (λf. (λx. f (f x)))
    3 := (λf. (λx. f (f (f x))))
The natural number n is a function, which takes a function f as an argument, and apply it n times to its argument.

From the definition, we can define a successor function, which takes a number n and apply one more time of function f to it:
    SUCC := (λn. (λf. (λx. f (n f x))))

And the addition of two numbers, m and n, can be defined by apply m+n times of function f:
    PLUS := (λm. (λn. (λf. (λx. m f (n f x)))))

Boolean
The boolean values TRUE and FALSE can be defined as follows:
    TRUE   := (λx. (λy. x))
    FALSE  := (λx. (λy. y))
And some logical operators:
    AND := (λp. (λq. p q p))
    OR  := (λp. (λq. p p q))
    NOT := (λp. (λa. (λb. p b a)))

The Wikipedia page has more details about the data types, so I won't duplicated it here.

Till now, the concepts about lambda calculus that are necessary for the evaluator have been covered. In the next post, I am going to define the concrete syntax of lambda calculus and implement the scanner and parser.