显示标签为“plt”的博文。显示所有博文
显示标签为“plt”的博文。显示所有博文

2011年7月28日星期四

Lazy CEK Machine

In the CEK Machine, though the substitution for an identifier is delayed, it doesn't mean it is a lazy machine. As described in Eager and Lazy Evaluation, a lazy evaluation strategy doesn't evaluate an argument before applying it to a function. However, in the CEK machine, it always evaluates the argument to a value, so it is still a call-by-value machine. If you are careful enough, you may already notice whenever an environment is created, the expression in its closure is always a value. The real power of environment is working with lazy evaluation. So in this post, I am going to implement the call-by-name and call-by-need strategies for the CEK machine.

Call by Name
It is quite easy to implement the call-by-name strategy based on the current call-by-value one. Since the machine no longer evaluates the argument before applying it to a function, the reduction rules related to applications should be modified(For a primitive operator, the operands are needed at the moment it is evaluated, so it stays unchanged.). In the call-by-value machine, when the control string is an application, first an ArgKK continuation is handled, then a FunKK continuation is handled. Now, the two steps, 2.1 and 2.2, should be combined into one, and the FunKK continuation type is removed.

2) If the control string is a value(abstraction or constant), check the type of continuation:
    2.1) If it is ArgKK, pop up the continuation, create a new environment that maps the function parameter to current closure, and set the function body and the new environment as the current expression and environment respectively;

However, the machine may need to evaluate the same argument many times now. Take expression (lambda x  + x x) ((lambda y y) 1) as an example, x is mapped to ((lambda y y) 1) after reducing the outer application. Then, when evaluating + x x, ((lambda y y) 1) is evaluated twice. The reason is that a parameter may be used several times in a function body, and whenever it is needed, the original argument it maps to should be evaluated. What if remember the result of the argument for the first time it is evaluated, so it doesn't bother to re-compute it later? I will talk this in next section.

Call by Need
The idea is quite straightforward. Remember the value for a closure, so next time the closure is needed, the value is retrieved directly. Considering the same example (lambda x + x x) ((lambda y y) 1), when the first x in + x x is needed, ((lambda y y) 1) is evaluated and the result is 1. After this, the environment which maps x to ((lambda y y) 1) is updated so x is mapped to 1. When the second x is needed, 1 is retrieved from the environment and no extra evaluation is performed.

However, the implementation is not that simple. When evaluating ((lambda y y) 1), I can't simply put it as the control string of current machine, because this may destroy the machine state and the machine can't go back to previous state after it is evaluated. Instead, a separate evaluation process should be performed on it. This is kind of a sub-machine, whose evaluation logic is the same, but it relies on some states from the parent machine, such as environments because the expression itself may contain free variables. In current evaluate() function, it initializes the machine state at the beginning and clean it up at the end. It was designed like this because no sub-machine was ever needed before. I am going to refactor it so it is fed with a machine state and focuses on the reduction logic, instead of managing the lifecycle of the machine state. One advantage is an expression can be put into a particular context for evaluation. It is more flexible. The interface is changed to:

// file: eval.c
static int _evaluate(State *state);

TreeNode * evaluate(TreeNode *expr) {
    Environment *globalEnv = buildGlobalEnvironment();
    State *state = cek_newState();
    state->closure = cek_newClosure(expr,globalEnv);

    TreeNode* result = NULL;
    if(_evaluate(state)) {
        // The result control string may contain free variables, need to 
        // substitute them using the environment for it.
        TreeNode *tmp = resolveFreeVariables(state->closure->expr,state->closure->env);
        if(tmp!=NULL) {
            // actually, tmp and state->closure->expr are the same expression
            result = tmp;
            state->closure->expr = NULL;
        }
    }
    cek_cleanup(state);
    return result;
}

And the code for looking up an identifier is:

// file: eval.c
static Closure* lookupVariable(const char *name, Environment *env) {
    if(env==NULL) return NULL;
    if(strcmp(name,env->name)==0) {
        Closure *closure = env->closure;
        if(closure->env!=NULL) {
            // evaluates the expression in its environment
            State *state = cek_newState();
            state->closure = cek_newClosure(duplicateTree(closure->expr),cek_newEnvironment("",cek_newClosure(NULL,NULL),closure->env));

            if(_evaluate(state)) {
                deleteTree(closure->expr);
                cek_deleteClosure(closure);
                // save the evaluated result
                env->closure = cek_newClosure(state->closure->expr,state->closure->env);
                state->closure->expr = NULL;
            }
            cek_cleanup(state);
        }
        return env->closure;
    }
    return lookupVariable(name,env->parent);
}

More Words about Closure and Environment
Another design improvement I want to mention is the global environment. Currently, if the control string is an identifier, the machine first tries to look it up in the environments and if not found it tries to resolve it from the builtin and standard libraries. Since a library maps identifiers to expressions, it is just like an environment. So the identifier lookup can be treated universally by introducing a global environment, which contains all the identifiers and functions provided by the evaluator. It is the root environment for all environments that are created during the evaluation and it is initialized when the machine is started. Thus, to handle a control string which is an identifier, the machine only needs to look it up in the environments. If it is a builtin or standard function, the machine can finally find it in the global environment.

A closure is an expression associated with an environment. The environment is where the expression is defined rather than it is evaluated. Let's consider the reduction steps of expression (lambda x  (lambda y (lambda x + x y) 2) x) 1.

(lambda x (lambda y (lambda x + x y) 2) x) 1
  => (lambda y (lambda x + x y) 2) x)                    A = {x=>(1,G), G}
  => (lambda x + x y) 2                                          B = {y=>(x,A), A}
  => + x y                                                               C = {x=>(2,B), B}
  => + 2 y                                                               C = {x=>(2,B), B}
  => + 2 1                                                               C = {x=>(2,B), B}  *
  => 3

The result is 3 rather than 4, because at step (*), y is mapped to x, and x is looked up in environment A, where it is defined, but not the latest environment C where is evaluated.

You can get the source code in branches call-by-name and call-by-need via github.

2011年7月23日星期六

CEK Machine

In the textual machines I mentioned in previous posts, when reducing an application, without exception, everyone requires the substitution of an argument in place of all the occurences of an identifier in the abstraction body. The machine traverses the expression tree during substitution, and if the result is not a value, it needs to re-traverse it again to evaluate it. Take (lambda x + x x) 1 as an example, + x x is traversed to replace x with 1. After that, we get result + 1 1. To evaluate it, the expression + 1 1 is traversed again. It is inefficient to traverse an expression twice. We cannot get rid of this if we stick on the substitution when reducing an application. We can, however, remember the identifier and the argument at the moment and delay the substitution until it is actually needed.

To represent this delay substitution, a component that maps an identifier to an expression is needed. It is called an environment. Every expression in the machine is replaced by a pair, whose first part is an expression probably with free variables in it and the other part is an environment. It is called a closure. Of course, the environment could not map identifiers to expressions, because the machine never works on a single expression. It should map identifiers to closures. And the continuation also contains a closure. Now, the machine state changes to a control string, an environment and a continuation, so it is called a CEK machine. Here is the data structures introduced for environment and closure:

// file: cek_machine.h
struct envStruct {
    char *name;
    struct closureStruct *closure;
    struct envStruct *parent;
};

struct closureStruct {
    TreeNode *expr;
    struct envStruct *env;
};

typedef struct continuationStruct {
    ContinuationKind tag;
    Closure * closure;
    struct continuationStruct * next;
} Continuation;

Conceptually, an environment can contain several identifier-closure mappings. However, in the definition above an environment only has one such mapping. Since mappings are created when reducing applications and an abstraction only has one parameter, I keep one mapping in a standalone environment and arrange them hierarchically. When looking up for the mapped closure of an identifier, first try it in its associated environment, and if failed, try it in the parent one.

CEK Machine
There is no significant change to continuation actions. Some new actions, like creating a closure, deleting a closure, creating an environment, looking up in an environment, are very common in the reduction steps. By the way, an identifier is no long a value because it may be mapped to an expression, which can be further reduced. Here are the reduction rules:
1) If the control string is an identifier, look up the mapped closure in the environment, and set the expression and environment of the closure as current expression and environment respectively;
2) If the control string is a value(abstraction or constant), check the type of continuation:
    2.1) If it is FunKK, pop up the continuation, create a new environment that maps the function parameter to current closure, and set the function body and the new environment as the current expression and environment respectively;
    2.2) If it is ArgKK, set the continuation as FunKK, and switch the current expression and environment with these in the closure of continuation respectively;
    2.3) If it is OprKK, pop up the continuation, evaluate the primitive expression, and set the result as the current expression and set the current environment to null;
    2.4) If it is OpdKK, set next argument expression and its environment as current expression and environment respectively;
3) If the control string is an application, create and push a continuation of type ArgKK, whose closure consists of the argument expression and current environment, and set the abstractioin expression as current expression, with current environment unchanged;
4) If the control string is a primitive, create and push a continuation of type OpdKK, whose closure consists of the primitive expression and current environment, and set the first argument expression as current expression, with current environment unchanged.

Though implementation is straightforward according to the reduction rules, the code is getting complicated, so I am not duplicate it here. You can still access it on github.

A drawback of the CEK machine is no intermediate expression can be obtained after each step. What a pity! The machine always delays the substitution and expression is not actually reduced after each step. It does less work at the expense of losing immediate effects.

2011年6月10日星期五

CK Machine

In CC and SCC machines, context expands and shrinks when traversing the expression tree in postorder. The nodes in the bottom of the tree are always evaluated before the upper ones. From another perspective, treating the expression as a text string, it is evaluated from innermost to outermost. In the case that the control string is an application, a new innermost application is pushed into the context. In the case that the control string is a value, the innermost application in the context is popped up and the machine decides what to do next by inspecting its shape. I used a first-in last-out list to save the contexts, because the reduction steps only depend on the innermost application, that is the latest context, but not on the rest of the data structure. Compared to the recursive implementation, this iterative evaluator doesn't accumulate runtime stack frames, but this data structure works as an explicit stack in the single runtime stack frame. If the control string is an application, a new "stack frame" is pushed. If it is a value, the top "stack frame" is popped up. Using the control string and information in the popped up "stack frame", the machine decides subsequent actions, performing a reduction or pushing the "stack frame" back. Here is the code snippet making the decision in SCC machine:
// File: eval.c line 92-106 

// control string is the right child
    if(state->controlStr==state->context->expr->children[1]) {
        // pop an expression from the context
        state->controlStr = state->context->expr;
        ctx = state->context;
        state->context = state->context->next;
        cc_deleteContext(ctx);
        ctx = NULL;
        if(!performReduction(state)) {
            return NULL;
        }
    } else {
        state->controlStr = state->context->expr->children[1];
    }
If the control string is the left child of the innermost application, which means it is the function part, the next action is to evaluate the argument. If it is the right child, which means it is the argument, the next action is to perform reduction on the innermost application. I have to inspect such relationship between the control string and innermost application because I didn't store it in the context, although it is available when the context was created. Look at the following code snippet:
// File: eval.c line 115-119

if(!isValue(state->controlStr->children[0])) {
    state->controlStr = state->controlStr->children[0];
}else {
    state->controlStr = state->controlStr->children[1];
}
The shape of the application was clearly known when setting the control string. At this phase, the machine was able to determine the actions once control string is evaluated. If the actions are saved in the newly created context, when popped up later the saved actions can be performed directly without inspecting the shape of the application. For ease of implementation, I will tag each context indicating what to do next when it is popped up. This tagged context is called continuation. The definition is:
// File: ck_machine.h
/* Kind of each continuation. */
typedef enum {
    FunKK, ArgKK, OprKK, OpdKK
} ContinuationKind;

/* Use a LIFO list to represent the continuation. */
typedef struct continuationStruct {
    ContinuationKind tag;
    TreeNode * expr;
    struct continuationStruct * next;
} Continuation;

The innermost application is kept in a continuation as expr. Four actions are defined, with the first two for applications and the other two for primitive applications. If its is FunKK, the next action is to perform reduction on the application; If it is ArgKK, the next action is to evaluate the argument expression; If it is OprKK, the next action is to evaluate the primitive application; If it is OpdKK, the next action is to evaluate the next argument of the primitive application.

CK Machine
Now, the state of the textual machine is changed to a pair of control string and continuation, and this machine is called a CK machine. The reduction rules are similar to that of CC/SCC machine, except for manipulations on continuations:
    1) If the control string is a value(identifier, constant or abstraction), check the type of continuation:
        1.1) If it is FunKK or OprKK, pop up the continuation, set the expression in it as control string, and perform reductions;
        1.2) If it is ArgKK, set the continuation to FunKK, and set the argument expression as control string;
        1.3) If it is OpdKK, set the continuation to OprKK, and set the next argument expression as control string;
    2) If the control string is an application, create and push a continuation of type ArgKK, and set the abstraction expression as control string;
    3) If the control string is an primitive, create and push a continuation of type OpdKK, and set the first argument as control string.

The primitive application only supports two arguments till now, so the next argument just means the second argument. Here is the code implementing the reduction rules:
// File: eval.c
TreeNode * evaluate(TreeNode *expr) {
    State * state = ck_newState();
    state->controlStr = expr;

    Continuation * ctn = NULL;
    while(!ck_canTerminate(state)) {
        if(isValue(state->controlStr)) {
            switch(state->continuation->tag) {
                case FunKK: // pop up the continuation
                case OprKK:
                    state->controlStr = state->continuation->expr;
                    ctn = state->continuation;
                    state->continuation = ctn->next;
                    ck_deleteContinuation(ctn);
                    ctn = NULL;
                    if(!performReduction(state)) {
                        return NULL;
                    }
                    break;
                case ArgKK: // change continuation to FunKK
                    state->continuation->tag = FunKK;
                    state->controlStr = state->continuation->expr->children[1];
                    break;
                case OpdKK: // change to OprKK
                    state->continuation->tag = OprKK;
                    state->controlStr = state->continuation->expr->children[1];
                    break;
                default:
                    fprintf(errOut,"Error: Unknown continuation tag.\n");
                    ck_cleanup(state);
                    return NULL;
                    
            }
        }else {
            if(state->controlStr->kind==AppK) { // push an ArgKK continuation
                ctn = ck_newContinuation(ArgKK);
            }else if(state->controlStr->kind==PrimiK) { // push OpdKK continuation
                ctn = ck_newContinuation(OpdKK);
            }
            ctn->expr = state->controlStr;
            ctn->next = state->continuation;
            state->continuation = ctn;
            state->controlStr = state->controlStr->children[0];
            ctn = NULL;
        }
        #ifdef DEBUG
            // print intermediate steps
            if(!ck_canTerminate(state)) {
                fprintf(out,"-> ");
                printExpression(ck_getProgram(state),out);
                fprintf(out,"\n");
            }
        #endif
    }

    TreeNode* result = state->controlStr;
    ck_deleteState(state);
    return result;
}

The implementation is quite simple but I want to add some words about the CC/SCC and CK machine. From the code and reduction rules, the CK machine looks quite similar to that of CC/SCC machine, and there is only a tiny difference, a tag, in the data structure for context and continuation. However, the ideas behind them are really distinct. For a context, it stores the innermost application expression, which can be thought as data. While for a continuation, besides such data, it also keeps the information for actions, which can be thought as code. In the above implementation, the continuation is attached with a tag, but for other implementations, functions or programs may be attached to make it more usable and flexible. Furthermore, the programmer can even provide a program as actions to continuations explicitly. Another significant difference is: with context, the applications can only be evaluated serially from innermost to outermost, because the relationship of the innermost application and control string is necessary to decide what the next action is. With continuation, this is not required because action is saved in the continuation, and each continuation itself can decide the next action. So in some circumstances, several continuations can be skipped and another continuation can be picked to continue the evaluation. This is essential to control flow and error handling. For example, if an error happens in an application, the machine should go to the function that handles the error, instead of evaluating the next expression in current application. I will go back to this topic in later posts.

2011年5月28日星期六

CC and SCC Machine

An obvious inefficiency of our first textual machine is the repeated traversal of the expression tree to find the redex. After each reduction step, it starts over again from the root node. We really don't have to do this, because the next redex is often the reduction result of current redex, or at least, closes to the current one. Let's consider the evaluation of the following expression:
(+ ((lambda x (lambda y y) x) 1) 1)
    => (+ ((lambda y y) 1) 1)
    => (+ 1 1)
    => 2

The redex for each step is underlined. In the first two steps, the redex for next step is just the evaluation result of the current redex. And in the third step, the next redex is the parent expression of the result of previous step. Actually, we can identify the next redex by checking the result of current one or its parent for every expression. Suppose the evaluation result of a redex is an application, if either of its children is not a value, the non-value child expression will be the next redex. Otherwise, the result itself will be the next one. Suppose the result is a value, its parent or the other child will probably be the next redex. We didn't do this kind of checking in the textual machine, because we only had the complete expression as our machine state, and the machine were not aware of the result and its parent. Though this could be obtained in the implementation, from the model view, the machine does only know the complete expression. So in this post, two new machines using the strategy will be defined to improve the efficiency.

CC Machine
According to the discussion above, if the textual machine knows the redex and its parent, it can easily decide what the next redex is, rather than traversing the entire expression tree again. To keep it running until getting the final result, not just the parent of the redex, but also the parent's parent should be recorded. The redex is the machine currently interested in, and it is called control string. Except the redex, the other part of the expression is called evaluation context, from which all the parents can be obtained. The two elements are paired together to form the machine's state. So this machine is called CC machine. Combining the control string and context yields the complete expression, which is the state of the previous machine. I'd like to call the complete expression as program. Initially, the program is the control string, and the context is empty. The termination condition of this machine is when the control string is a value and context is empty. Here is the reduction rules for this machine:
    1) If the control string is a value(identifier, constant or abstraction), remove its parent from the context, and set the parent as control string;
    2) If the control string is an application or primitive, and at least one of its children is not a value, put itself into the context and set the non-value child as control string;
    3) If the control string is an application or primitive, and both of its children are values, perform reductions on it and set the result as control string.

To find the first redex, we still need to traverse the expression tree, but the traversed nodes should be remembered in the context. Since only the parent of control string is interested for each step, and from each traversed parent, we can navigate to any part of the program, I use a FILO(First-In-Last-Out) list of the traversed nodes to represent the context(Actually, we don't need such kind of data structure if a pointer to parent node is added to each node, but this requires changes to the parser, so I don't choose this option). From the implementation view, the machines perform evaluations by popping and pushing control string into or from the context to find the redex. Here is the definition for machine state and the context:
// file: cc_machine.h
/* Use a LIFO list to represent the context. */
typedef struct contextStruct {
    TreeNode * expr;
    struct contextStruct * next;
} Context;

/* Machine state is a pair of control string and the context. */
typedef struct stateStruct {
    TreeNode * controlStr;
    Context * context;
} State;


From the reductions rules, we can easily derive the actions on context:
    1) If the control string is a value, pop the head node from the list and set the node as control string;
    2) If the control string is an application or primitive, and at least one of its children is not a value, push it into the head of the list and set the non-value child as control string;
    3) If the control string is an application or primitive, and both of its children are values, perform reductions on it and set the result as control string and no action for context.

Here is the listing of code:
// file: eval.c
TreeNode * evaluate(TreeNode *expr) {
    State * state = cc_newState();
    state->controlStr = expr;

    Context * ctx = NULL;
    while(!cc_canTerminate(state)) {
        if(isValue(state->controlStr)) {
            // pop an expression from the context
            state->controlStr = state->context->expr;
            ctx = state->context;
            state->context = state->context->next;
            cc_deleteContext(ctx);
            ctx = NULL;
        }else {
            if(!isValue(state->controlStr->children[0])
                || !isValue(state->controlStr->children[1])) {
                // push the current expression into context
                ctx = cc_newContext();
                ctx->expr = state->controlStr;
                ctx->next = state->context;
                state->context = ctx;
                if(!isValue(state->controlStr->children[0])) {
                    state->controlStr = state->controlStr->children[0];
                }else {
                    state->controlStr = state->controlStr->children[1];
                }
            } else { // evaluate control string
                if(state->controlStr->kind==AppK) {
                    if(state->controlStr->children[0]->kind==ConstK) {
                        fprintf(errOut, "Error: cannot apply a constant to any argument.\n");
                        fprintf(errOut, "\t\t");
                        printExpression(state->controlStr,errOut);
                        cc_cleanup(state);
                        return NULL;
                    }else if(state->controlStr->children[0]->kind==IdK) {
                        // find function from builtin and standard library
                        TreeNode* fun = resolveFunction(state->controlStr->children[0]->name);
                        if(fun==NULL) {
                            fprintf(errOut, "Error: %s is not a predefined function.\n", state->controlStr->children[0]->name);
                            cc_cleanup(state);
                            return NULL;
                        }
                        deleteTree(state->controlStr->children[0]);
                        state->controlStr->children[0] = fun;
                    } else {
                        TreeNode *tmp = betaReduction(state->controlStr);
                        if(state->context!=NULL) {
                            if(state->context->expr->children[0]==state->controlStr) {
                                state->context->expr->children[0] = tmp;
                            }else {
                                state->context->expr->children[1] = tmp;
                            }
                        }
                        state->controlStr = tmp;
                    }
                }else if(state->controlStr->kind==PrimiK) {
                    // only perform primitive operation if operands are constants
                    if(state->controlStr->children[0]->kind==ConstK
                        && state->controlStr->children[1]->kind==ConstK) {
                        TreeNode* tmp  = evalPrimitive(state->controlStr);
                        if(state->context!=NULL) {
                            if(state->context->expr->children[0]==state->controlStr) {
                                state->context->expr->children[0] = tmp;
                            } else {
                                state->context->expr->children[1] = tmp;
                            }
                        }
                        deleteTree(state->controlStr);
                        state->controlStr = tmp;
                    } else {
                        fprintf(errOut, "Error: %s can only be applied on constants.\n", state->controlStr->name);
                        cc_cleanup(state);
                        return NULL;
                    }
                }else {
                    fprintf(errOut,"Error: Cannot evaluate unkown expression kind.\n");
                    cc_cleanup(state);
                    return NULL;

                }
            }
        }
        #ifdef DEBUG
            // print intermediate steps
            if(!cc_canTerminate(state)) {
                fprintf(out,"-> ");
                printExpression(cc_getProgram(state),out);
                fprintf(out,"\n");
            }
        #endif
    }

    TreeNode* result = state->controlStr;
    cc_deleteState(state);
    return result;
}
Let's try it:
$ ./main
Welcome to Lambda Calculus Evaluator.
Press Ctrl+C to quit.

> (lambda x x) (lambda y y) 1
-> (lambda x x) (lambda y y) 1
-> (lambda y y) 1
-> (lambda y y) 1
-> 1

SCC Machine
Examining the reduction steps at the end of the last section, step 2 and 3 are the same. That's because after step 2, the control string is (lambda y y) and the context is the parent of it. Based on the rules, the machine pops the parent and set it as control string. The machine state is changed but not actual reduction is performed on the program, so the program for both steps are the same. After that, beta-reduction is performed on the control string. Actually, we only have two possible actions after step 2: performing beta-reduction or primitive evaluation on the parent node or setting the other child as control string. We can simplify the process by checking the parent node so those two steps can be combined into one. Rule 1) of the CC machine is now revised to:
    1') If the control string is a value and the other child of its parent is a value too, remove its parent from the context, perform reductions to it and set the result as control string. Otherwise, set the other child as control string.

The code needs only simple changes, so I won't duplicate it here. Here is the evaluation steps for the same expression, and the duplicated step is gone:
$ ./main
Welcome to Lambda Calculus Evaluator.
Press Ctrl+C to quit.

> (lambda x x) (lambda y y) 1
-> (lambda x x) (lambda y y) 1
-> (lambda y y) 1
-> 1

Compared to CC machine, SCC machile always has less steps. All the code is available at here.

2011年5月22日星期日

Textual Machine

For a complex expression, such as a recursive function using Y, we may go through dozens of reduction steps until getting to the final result. Currently, our evaluator only gives the final result. However, these intermediate steps are really useful for analysis and troubleshooting sometimes. Just like the manual reductions we did in last post, we can check whether the evaluator follows the reduction rules by examining the intermediate steps. So in this post, I am going to refactor the implementation of the evaluator so that it is able to print the intermediate reduction steps.

Recursive Way
An essential task of the evaluator is to find the redex to be reduced next. In our current implementation, we find it in a recursive way. That is, to find a redex in an expression, we try to find it in its child expressions first. If not found, we check if the expression itself is a redex. In this process, the evaluate function may call itself for some subexpressions. This keeps the code simple and clean. However, it also puts us in a dilemma that we cannot get the whole picture of the evaluation process. In each function invocation, we are only aware of the expression or subexpression which is under evaluation, but we don't know its role(as a function or an argument) in the original expression, what its parent expression is and so on. All the information is called evaluation context. Actually, we do have it in the sequence of C function calls. In the runtime environment of C, the function invocations are stored in a stack, and for each one, there is a stack frame for it, which keeps data for arguments, results and temporary values. So the expression we are currently evaluating is stored in the top stack frame, and its parent is stored in a previous one, and so on. The evaluation context is hidden in the list of stack frames, but we don't have an easy way to get them in C.

A potential solution is to use a global variable pointing to the root of the expression tree and print it after each evaluation step. However, this method may not work for all cases. Take (lambda x + x x) 1 as an example, after a beta-reduction, the original tree is deleted with a new tree returned as the result, so the global variable becomes a dangling pointer. Even if this could meet our requirement for printing intermediate evaluation steps, we still cannot get more evaluation context information. So I am going to implement the evaluation algorithm in a different way to make the information easy accessible.

Iterative Way
The problem for recursive way is the evaluation context is saved across many stack frames and only that in the top one is accessible. So a solution we can easily thought of is keeping all the evaluation context information in the same stack frame. We need to get rid of the recursive calls and use an iterative way. To find the redex, we need to traverse the expression tree, and all the work is done in a single evaluate function call. Since everything is in the same stack frame now, we are allowed to get anything about the evaluation context.

Based on the call-by-value evaluation strategy, we can derive the following rules to find a redex:
    1) An identifier, constant or abstraction expression is not a redex;
    2) If the expression is an application or primitive application, check its child expressions from left to right, if anyone is not a value, find the redex in it. Otherwise, pick the expression as a redex.

Since the leftmost expression and the argument expression is evaluated first, we need a post-order traversal of the expression tree to find the redex and evaluate it. After each step, a new expression is got and it is closer to the final result. The same post-order traversal will be performed on the expression again and again, until a value is returned. For the case that an expression can never reduce to a value, the evaluator will run forever. For another case that something is wrong in the expression, an error message will be printed and the evaluator will returns NULL. The error handling mechanism will be added in the future so values will be returned for erroneous expressions as well. Here is the code for this iterative way:
// file: eval.c
TreeNode * evaluate(TreeNode *expr) {
    TreeNode* state = expr;
    TreeNode** previous = NULL;
    TreeNode* current = NULL;
    while(state!=NULL && !isValue(state)) {
        previous = &state;
        current = state;
        while(current!=NULL) {
            if(!isValue(current->children[0])) {
                previous = ¤t->children[0];
                current = current->children[0];
            }else if(!isValue(current->children[1])) {
                previous = ¤t->children[1];
                current = current->children[1];
            }else { // reduce the current node
                if(current->kind==AppK) {   // applications
                    if(current->children[0]->kind==ConstK) {
                        fprintf(errOut, "Error: cannot apply a constant to any argument.\n");
                        fprintf(errOut, "\t\t");
                        printExpression(current,errOut);
                        deleteTree(state);
                        return NULL;
                    }else if(current->children[0]->kind==IdK) {
                        // find function from builtin and standard library
                        TreeNode* fun = resolveFunction(current->children[0]->name);
                        if(fun==NULL) {
                            fprintf(errOut, "Error: %s is not a predefined function.\n", current->children[0]->name);
                            deleteTree(state);
                            return NULL;
                        }
                        deleteTree(current->children[0]);
                        current->children[0] = fun;
                        break;
                    }
                    current=betaReduction(current);
                    *previous = current;
                    break;
                }else if(current->kind==PrimiK) {  // primitive application
                    // only perform primitive operation if operands are constants
                    if(current->children[0]->kind==ConstK
                        && current->children[1]->kind==ConstK) {
                        TreeNode* tmp  = evalPrimitive(current);
                        deleteTree(current);
                        current = tmp;
                        *previous = current;
                        break;
                    } else {
                        fprintf(errOut, "Error: %s can only be applied on constants.\n", current->name);
                        deleteTree(state);
                        return NULL;
                    }
                }else {
                    fprintf(errOut,"Error: Cannot evaluate unkown expression kind.\n");
                    deleteTree(state);
                    return NULL;
                }
            }
        }
        #ifdef DEBUG
            // print intermediate steps
            if(!isValue(state)) {
                fprintf(out,"-> ");
                printExpression(state,out);
                fprintf(out,"\n");
            }
        #endif
    }
    return state;
}

In the code, variable state keeps the intermediate expression during the evaluation, so it is used to print the intermediate steps. Variable current points to the subexpression we are currently working with, and previous is a pointer to the current pointer, so after a beta-reduction or primitive evaluation is performed on the current subexpression, we can make the parent of current point to the new expression by changing the value of previous. The state and previous variable save part of the evaluation context information, and if others are needed, such as parent expression of the redex, they can be added easily. There are two loops in the code. The outer one runs until the evaluation result is a value, and the inner one implements the post-order traversal to find and evaluate the redex. The isValue() function tests if an expression is a value, which is identifier, constant or abstraction for now. The resolveFunction() function finds a predefined function from libraries in the order of builtin library and standard library. After each step, the intermediate expression is printed if the DEBUG flag is set.

Let's try a simple example:
$ ./main
Welcome to Lambda Calculus Evaluator.
Press Ctrl+C to quit.

> (lambda x * 2 x) ((lambda x x) 3)
-> (lambda x * 2 x) 3
-> * 2 3
-> (lambda x (lambda y x `*` y)) 2 3
-> (lambda y 2 `*` y) 3
-> 2 `*` 3
-> 6

Textual Machine
In a real computer, a set of registers is used to keep the computation states, such as PC, and a list of instructions like ADD and MOV can be performed on the state to transform the machine to a new state. Our evaluator can be considered as a machine too. In contrast, the state is the expression, and the instructions are the reductions that transform expressions to expressions. We call this a textual machine, because expressions are represented as texts. This is the first textual machine we have implemented, and to improve it, a few more will be introduced in later posts.

2011年5月16日星期一

Recursion and Y Combinator

Today, I will talk about an important concept: recursion. It means a function could call itself inside its body. With recursion, many difficult problems can be solved in an easily understood way. A good case in point is the problem of Tower of Hanoi. In this post, a much simpler problem is used to demonstrate the concept of recursion and how we implement it in our evaluator.

Till now, our evaluator is capable of doing basic arithmetics, but that's not enough for a real programming language. Let's consider this problem: how to compute the factorial of n? For 3, we can compute it using (* 3 (* 2 1)); For 4, it is (* 4 (* 3 (* 2 1))). However, for n, we cannot write out the expression, because n is unknown when we are defining the function. By observing factorial 3 and 4, we can easily find out that factorial 4 is the multiplication of integer 4 and factorial 3. We can derive a recursive solution like this: to compute factorial of n, we can multiply n by factorial of n-1; to compute factorial of n-1, we can multiply n-1 by factorial of n-2; and so on. Until we meet factorial 1, whose result we know is 1. This is called the base case, where the recursion is stopped at. Start with this strategy, we can compute the factorial of n without writing out the whole expression, because the recursion will take care of it for us.

Recursion
Based on the idea described before, we can define the factorial function recursively as follows:
    factorial  := (lambda n (= n 1) 1 (* n (factorial (- n 1))))

We use the ":=" symbol to assign a name to a function to assist explanation and it has nothing to do with our lambda calculus evaluator. The = function checks the equality of two integers and returns a boolean value. This is not a correct expression because lambda calculus doesn't allow a function to refer to itself. Moreover, we are not able to write out such an expression, because when defining a function we need itself inside the body. Though we don't have the function when we are defining it, it will be available later. Thus, instead of referring to itself, the factorial function could have us supply a factorial function t as an argument. Using this strategy, the function we define is no longer a factorial function, but a factorial maker function: it takes some factorial function and produce a factorial function. The mkfactorial function is defined as:
    mkfactorial := (lambda t (lambda n (= n 1) 1 (* n (t (- n 1)))))

This is a legal expression in our lambda calculus evaluator, but it still sounds strange. We are defining a factorial function, but we rely on another factorial function. Obviously, we still don't have a factorial function. Think again, what if we change the definition of mkfactorial to which requires a maker function as an argument instead of a factorial functioin? We assume the mkfactorial function apply on a maker function t would produce a factorial function, and for every occurrence of a factorial function in the body we use (t t), because applying a maker function to itself produces a factorial function. We redefine the function as follows:
    mkfactorial' := (lambda t (lambda n (= n 1) 1 (* n ((t t) (- n 1)))))

And we could get factorial function by applying mkfactorial' to itself:
    factorial := mkfactorial' mkfactorial'

To examine whether the solution works, we are going to reduce factorial 3 and factorial n. To save space, I will use the names for the functions defind above during the reduction and only expand them when necessary. Remember, when evaluated in our evaluator, only the expanded expressions are kept. For factorial 3:
factorial 3
    = mkfactorial' mkfactorial' 3
    = (lambda t (lambda n (= n 1) 1 (* n ((t t) (- n 1))))) mkfactorial' 3
    => (lambda n (= n 1) 1 (* n ((mkfactorial' mkfactorial') (- n 1)))) 3
    => (= 3 1) 1 (* 3 ((mkfactorial' mkfactorial') (- 3 1)))
    => (* 3 ((mkfactorial' mkfactorial') 2))
    = (* 3 ((lambda t (lambda n (= n 1) 1 (* n ((t t) (- n 1))))) mkfactorial' 2))
    => (* 3 ((lambda n (= n 1) 1 (* n ((mkfactorial' mkfactorial') (- n 1)))) 2))
    => (* 3 ((= 2 1) 1 (* 2 ((mkfactorial' mkfactorial') (- 2 1)))))
    => (* 3 (* 2 ((mkfactorial' mkfactorial') 1)))
    = (* 3 (* 2((lambda t (lambda n (= n 1) 1 (* n ((t t) (- n 1))))) mkfactorial' 1)))
    => (* 3 (* 2 ((lambda n (= n 1) 1 (* n ((mkfactorial' mkfactorial') (- n 1)))) 1)))
    => (* 3 (* 2 ((= 1 1) 1 (* 1 ((mkfactorial' mkfactorial') (- 1 1))))))
    => (* 3 (* 2 1))
    => 6

And factorial n:
factorial n
    = mkfactorial' mkfactorial' n
    = (lambda t (lambda n (= n 1) 1 (* n ((t t) (- n 1))))) mkfactorial' n
    => (lambda n (= n 1) 1 (* n ((mkfactorial' mkfactorial') (- n 1)))) n
    => (= n 1) 1 (* n ((mkfactorial' mkfactorial') (- n 1)))
    => (* n ((mkfactorial' mkfactorial') (- n 1))

The meaning of (mkfactorial' mkfactorial') (- n 1) is factorial of n-1, and the result is what we want for factorial of n.

Y Combinator
Using the technique in the last section, we can construct the expressions for recursive functions from scratch. For each function, we always need to go through the following process: writing a function that refers to itself, changing it to a maker function, revising the maker function to make it accepts another makers function, and finally applying the maker function to itself. This is boring and clumsy. The significant part that makes each recursive function different is the maker function. Can we abstract this process into a function and the only thing we need to provide is a maker function?

The function we are eager to have is one that accepts a maker function and produces a recursive function. A maker function consists of two parts: a base case and a recursive case. The recursive stops when reaching the base case. Let's call the target function mk, and it is defined as:
    mk := (lambda t t (mk t))

Since argument t is a maker function, apply t to a recursive function, which we can make from (mk t) here, would again produce a recursive function, which is the result of mk. This is how we come up the definition of mk. Still, it is an illegal expression because mk refers to itself. Using the same technique as before, we can derive the maker function:
    mkmk' := (lambda k (lambda t t ((k k) t)))

and the mk function:
    mk := mkmk' mkmk'

We omitted the mkmk function. Now, we can define the factorial function as (mk mkfactorial). Let's check its behavior:
factorial
    = mk mkfactorial
    = (lambda k (lambda t t ((k k) t))) mkmk' mkfactorial
    => (lambda t t ((mkmk' mkmk') t)) mkfactorial
    => mkfactorial ((mkmk' mkmk') mkfactorial)
    = mkfactorial (mk mkfactorial)

According to the definition of mk, (mk mkfactorial) makes a factorial function, so the last step above is a factorial function. This is what we want.

With this mk function, we can define other recursive functions by providing a maker function. If we want to compute the sum of 1...n, we can define the function as follows:
    sum := mk (lambda t (lambda n (= n 1) 1 (+ n (t (- n 1)))))

In lambda calculus, mk is just one of such kind of functions. The more famous one is called Y:
    Y := (lambda f (lambda x f (x x)) (lambda x f (x x)))

Sometimes, when we are talking about Y combinator, it doesn't just mean the Y function, but a collection of functions like Y and mk.

Standard Library
The Y combinator is very useful for defining recursive functions. However, it is very inconvenient to write out the long expression every time we need it. So I would like to implement it as a predefined function in our evaluator, and we only need to call the Y function to use it. Unfortunately, the definition above cannot be used directly in our evaluator. Let's take a look at the reason:
Y g
    = (lambda f (lambda x f (x x)) (lambda x f (x x))) g
    => (lambda x g (x x)) (lambda x g (x x))
    => g ((lambda x g (x x)) (lambda x g (x x)))

In an eager evaluator, it won't progress in last step above, because the underlined part is an infinite loop and it can't be evaluated to a value. Since the argument is evaluated before applied to the function, g is never called in our current evaluator. This problem doesn't exist in a lazy evaluator. To solve this, we can change the underlined part to an abstraction expression. Do you still remember the η-conversion(that means λx. f x => f)? We can use the inverse-η conversion to change each application in Y into an abstraction.
    Y := (lambda f
            (lambda a
                (lambda x f (lambda g (x x) g))
                (lambda x f (lambda g (x x) g)) a))

Let's try to apply this new Y on mkfactorial:
Y mkfactorial
    = (lambda f (lambda a (lambda x f (lambda g (x x) g)) (lambda x f (lambda g (x x) g)) a)) mkfactorial
    => (lambda a (lambda x mkfactorial (lambda g (x x) g)) (lambda x mkfactorial (lambda g (x x) g)) a)
    => (lambda x mkfactorial (lambda g (x x) g)) (lambda x mkfactorial (lambda g (x x) g))
    => mkfactorial (lambda g ((lambda x mkfactorial (lambda g (x x) g)) (lambda x mkfactorial (lambda g (x x) g))) g)

Look at the last step, the underlined part is a function now, so we can apply it to the mkfactorial function. To save space, let's mark the underlined part as symbol T:
    => (lambda t (lambda n (= n 1) 1 (* n (t (- n 1))))) T
    => (lambda n (= n 1) 1 (* n (T (- n 1))))

This is the recursive function we finally get. Let's apply it on integer 1:
Y mkfactorial 1
    => (= 1 1) 1 (* 1 (T (- 1 1)))
    => (lambda x (lambda y x)) 1 (* 1 (T (- 1 1)))
    => (lambda y 1) (* 1 (T (- 1 1)))

In a lazy evaluator, the result 1 is returned from last step. While in our current eager evaluator, we won't get the expected result because of our implementation of if-statement using booleans. Currently, the boolean value is encoded using a function that cosumes two arguments. The (= 1 1) expression is evaluated to (lambda x (lambda y x)). Though the underlined part is never used in the function, we still have to evaluated it, and we will go into an infinite loop when doing so. From this example, we can conclude that Y combinator doesn't work in an eager evaluator which implements the if-statement as a pure function. To solve this, one way is using a lazy evaluation strategy. Another way is treating if-statement as a special form like Scheme. We choose the first option here, so all the things we talk later are implemented in our call-by-name evaluator.(If you use the second option, the Y function defined before will work without further modifications.)

Before adding Y function into our evaluator, I'd like to say a few words about builtin library and standard library. Till now, all the predefined functions in our evaluator are builtin functions, such as +, -, > and =. All the functions require the support of primitive operators by the evaluator. We cannot implement them using pure lambda calculus expression. While for standard functions, they are implemented as lambda calculus expressions. Without the builtin library, some features of the evaluator are lost, but without the standard library, programmers can still achieve the same effect by implementing the standard functions themselves using basic expressions. So the standard library is not mandatory and usually for programmer's convenience. The Y function is for such purpose, so we define it in the standard library.

The standard library is implemented in stdlib.h and stdlib.c files. It is similar to the implementation of builtin functions, so I won't duplicate the code here. The tricky part is that the yyparse() method is used to parse the expressions for standard functions into tree structure, so it could be used by the evaluator. The simple version of Y function is used because it works in our call-by-name evaluator. The evaluate function is also updated to search functions by name in the standard library. Please refer to source code for more information.

Some other functions, such as not, or and and, are added into the standard library in the same way. As an ending, let's try the factorial, sum and a more complex fibnacci function in our evaluator:
$ ./main
Welcome to Lambda Calculus Evaluator.
Press Ctrl+C to quit. 

# factorial 3
> Y (lambda t (lambda n (= n 1) 1 (* n (t (- n 1))))) 3
-> 6

# sum 4
> Y (lambda t (lambda n (= n 1) 1 (+ n (t (- n 1))))) 4
-> 10

# fibnacci 7
> Y (lambda t (lambda n (or (= n 1) (= n 2)) 1 (+ (t (- n 1)) (t (- n 2))))) 7
-> 13

2011年5月14日星期六

Eager and Lazy Evaluation

In the post that solves the partially reduced problem, we talked about a defective evaluation and a robust one. Though they are quite different, there is one principle that is kept by both of them: evaluating the argument expression before applying it to the function. This is called eager evaluation. In our evaluator, the argument won't be applied to the function until it is evaluated to a value. Many programming languages use this eager evaluation strategy, such as C, Java and Scheme. Most programming languages have a if-statement: if condition then-clause else else-clause. When condition is true, the then-clause is executed; otherwise, the else-clause is excecuted. At any time, only one of them will be executed. In our evaluator, we use the boolean encoding, which is a function that accepts two parameters, to simulate such if-statement. However, it has a different semantic. Since we use eager evaluation, both arguments will be evaluated before applying to the function. Though only one argument is needed, still both are evaluated. This is the problem of implementing if-statement using pure functions in an eager evaluation manner(that's why Scheme provide if as a special form instead of a pure function). To solve this, we need the lazy evaluation. That is, the argument expression is not evaluated unless it is actually needed in the evaluation of the function body.

These two evaluation strategies will be discussed in detail in the following sections and we will show how to implement them by revising our current evaluator as well.

Eager Evaluation
As stated above, in eager evaluation, the argument to a function is always evaluated completely before applied to the function. Regarding to whether evaluating the body of a function, it can be further divided to applicative order and call by value.

Applicative Order
Though our defective evaluator has "partially reduced problem", we can add a tiny fix to make it work. Since the problem only happens after evaluating the application expression, we can decide whether to keep evaluating after the beta-reduction by checking its result: if it is primitive expression, or it is an application expression and its left child is an abstraction expression, which means the result is reducible, we will continue evaluating the result. Here is the code:
// file: eval.c
TreeNode * evaluate(TreeNode *expr) {
    TreeNode* result = expr;
    if(expr!=NULL) {
        switch(expr->kind) {
            case IdK:
            case ConstK:
                return expr;
            case AbsK:
                expr->children[1] = evaluate(expr->children[1]);
                return expr;
            case AppK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                if(expr->children[0]->kind==IdK) {
                    // expand tree node for builtin functions
                    BuiltinFun* fun = lookupBuiltinFun(expr->children[0]->name);
                    if(fun!=NULL) {
                        expr->children[0] = (fun->expandFun)();
                    }
                }
                result = betaReduction(expr);
                // beta-reduction may result in primitive operations
                if(result->kind==PrimiK ||
                    (result->kind==AppK && result->children[0]->kind==AbsK)) {
                    result = evaluate(result);
                }
                return result;
            case PrimiK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                // only perform primitive operation if operands are constants
                if(expr->children[0]->kind==ConstK
                    && expr->children[1]->kind==ConstK) {
                    result = evalPrimitive(expr);
                }
                return result;
            default:
                fprintf(errOut,"Unkown expression kind.\n");
        }
    }
    return expr;
}
In this fixed evaluator, the body of an abstraction expression is evaluated first, and arguments are evaluated from left to right(our syntax gurantees this). We call this applicative order(or leftmost innermost). As discussed in the post "Solving Partially Reduced Problem", this strategy can not reduce some expressions though they could be, such as (lambda x 7) (lambda x (lambda y y y) (lambda y y y)). Since the body of an abstraction is evaluated first, it will go into an infinite loop when evaluating the body of the argument, though it is never used in the function (lambda x 7).

Call by Value
The robust evaluation strategy we use is very common in modern programming languages. Unlike applicative order, it never evaluates the body of an abstraction expression. This is called call-by-value evaluation, because every argument must be reduced to a value before calling the function. Obviously, this call-by-value strategy can evaluate the expression referred at the end of last section, because it doesn't evaluate the body of an abstraction expression, but it still has the similar problem. Take (lambda x 7) ((lambda y y y) (lambda y y y)) as an example, this time we have an application expression as the argument, which cannot be reduced to a value. Apparently, call-by-value evaluation won't compute a value for such cases.

Lazy Evaluation
From the failed cases in the previous section, we learned that there are cases that argument expressions are never used in the function, so it is not necessary to evaluate them before applying to the function. Why don't we just pass them to the function and evaluate them only when we actually need them? Lazy evaluation is such a strategy that doesn't evaluate arguments until they are actually needed in the evaluation of the function body.

Normal Order
Normal order(or leftmost outermost) is the counterpart of applicative order in lazy evaluation. Though it doesn't evaluate arguments, it still evaluates the body of a function first, because its goal is to reduce the expression as much as possible. When implementing the evaluator, the right child of the application expression is not evaluated, and it is used directly to substitute the free occurences of the parameter in the body of the function. So if the argument is not used in the function, it will be ignored and never evaluated. Here is the full listing of code:
// file: eval.c
TreeNode * evaluate(TreeNode *expr) {
    TreeNode* result = expr;
    if(expr!=NULL) {
        switch(expr->kind) {
            case IdK:
            case ConstK:
                return expr;
            case AbsK:
                expr->children[1] = evaluate(expr->children[1]);
                return expr;
            case AppK:
                expr->children[0] = evaluate(expr->children[0]);
                if(expr->children[0]->kind==IdK) {
                    // expand tree node for builtin functions
                    BuiltinFun* fun = lookupBuiltinFun(expr->children[0]->name);
                    if(fun!=NULL) {
                        expr->children[0] = (fun->expandFun)();
                    }
                }
                result = betaReduction(expr);
                // beta-reduction may result in primitive operations
                if(result->kind==PrimiK ||
                    (result->kind==AppK && result->children[0]->kind==AbsK)) {
                    result = evaluate(result);
                }
                return result;
            case PrimiK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                // only perform primitive operation if operands are constants
                if(expr->children[0]->kind==ConstK
                    && expr->children[1]->kind==ConstK) {
                    result = evalPrimitive(expr);
                }
                return result;
            default:
                fprintf(errOut,"Unkown expression kind.\n");
        }
    }
    return expr;
}
This strategy can evaluates more expressions than those eager ones, including the two failed expressions referred before. You could try them out.

Call by Name
In contrast to normal order, call-by-name doesn't evaluate the body of functions. The name of parameter is bound to the argument expression. So as normal order evaluation, every occurence of the parameter in the body of a function is substituted by the argument without evaluation. We implement this evaluator based on the robust evaluation, and only a tiny modification is needed to achieve such a call-by-name evaluator:
// file: eval.c
TreeNode * evaluate(TreeNode *expr) {
    TreeNode* result = expr;
    if(expr!=NULL) {
        switch(expr->kind) {
            case IdK:
            case ConstK:
            case AbsK:
                return expr;
            case AppK:
                expr->children[0] = evaluate(expr->children[0]);
                if(expr->children[0]->kind==IdK) {
                    // expand tree node for builtin functions
                    BuiltinFun* fun = lookupBuiltinFun(expr->children[0]->name);
                    if(fun!=NULL) {
                        expr->children[0] = (fun->expandFun)();
                    } else {
                        fprintf(errOut, "Error: %s is not a builtin function.\n", expr->children[0]->name);
                        return expr;
                    }
                }
                return evaluate(betaReduction(expr));
            case PrimiK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                // only perform primitive operation if operands are constants
                if(expr->children[0]->kind==ConstK
                    && expr->children[1]->kind==ConstK) {
                    result = evalPrimitive(expr);
                } else {
                    fprintf(errOut, "Error: %s can only be applied on constants.\n", expr->name);
                }
                return result;
            default:
                fprintf(errOut,"Unkown expression kind.\n");
        }
    }
    return expr;
}
An imperfect part of call-by-name strategy is that if an argument is used several times in the function, it is re-evaluated each time. So it is often slower than call-by-value. Take (lambda x + x x) (+ 1 2) as an example, the parameter x is used twice in the function body, so we need to evaluate (+ 1 2) twice, while it is evaluated once in the call-by-value evaluation.

Call by Need
To solve the performance problem of call-by-name, we can save the result of each argument when first time evaluated, and when it is used again, we don't need to evaluate it any more. Call-by-need is a memoized version of call-by-name, when the argument is evaluated, the result is stored for subsequent uses. It produces the same result as call-by-name. To implement it, some data structure, such as symbol table, are needed to store the values. Another challenge is arguments are not evaluated in the contexts where they are define. To do this correctly when they are needed, the context in which the argument expression is defined should be attached to it. This requires another important concept called environment and I'd like to implement it in the future.

Now, we have various evalutors with different evaluation strategies in our toolbox. In the future, if I don't emphasize a particular evaluator, that means we are talking about the call-by-value evaluation. Otherwise, I will specify which evaluation strategy we are talking about.

This post only mentioned some common evaluation strategies. For a complete list, please refer to Evaluation strategy. Remember you can still reach the full list of code from here.

2011年4月16日星期六

Solving Partially Reduced Problem

We met the "partially reduced problem" when we were adding the builtin function feature. That is, the primitive expression is returned unevaluated, though it can still be reduced. Actually, it doesn't only happen on the primitive expression, but also exists for ordinary expressions. Let's try another one:
$ ./main
Welcome to Lambda Calculus Evaluator.
Press Ctrl+C to quit.

> (lambda x (lambda y y x)) 1 (lambda x x)
-> (lambda x x) 1
Obviously, the result can be further reduced to 1. Let's do a simulation of the evaluation algorithm to see why we have such problem. The expression is evaluated from left to right. First, we need to evaluate (lambda x (lambda y y x)) 1. The left hand side is an abstraction, whose body is an abstraction too. We need to evaluate its body first. The inner body y x is evaluated to y x, so it remains unchanged. The right hand side is a constant and the result is itself. After performing a beta-reduction, the expression turns to (lambda y y 1). Then, we have (lambda y y 1) (lambda x x). Both left and right hand sides are abstraction and keeps unchanged after evaluating their bodies. Finally a beta-reduction is performed and it results in (lambda x x) 1. It is not a primitive expression, so the evaluation terminates and result (lambda x x ) 1 is returned.

This kind of "partially reduced problem" exists in many expressions, so we need to fix it. However, before doing that, we will define what "totally reduced" is, as opposed to "partially reduced".

Evaluation Values
Partially reduced result is generated because the evaluator terminates at a point where it shouldn't. So essentially we are talking of the termination condition. That means our evaluator should only terminate where some conditions are met. Otherwise, it should keep evaluating the resulted expression. So if the resulted expression doesn't meet the conditions, it should never be returned. I'd like to call the expressions that can be returned as evaluation values, values for short, because they are the only results our evaluator can give out.

From our current evaluation algorithm, we can easily identify that identifiers and constants are values. For an abstraction, though we may evaluate its body first(it turns out to be a bad strategy, we will get rid of this later), we can still return it as a result. For applications and primitives, we will always perform reductions on them. So we define the evaluation values as:

    V = identifier | constant | abstraction

The evaluator terminates only if it can return the above values. It should continue to evaluate the expression if it is an application or primitive expression. However, the evaluator doesn't always guarantee to return a value for every expression. Take x y as an example, it is an application expression but it can not be further reduced because x is not a defined function. Another example is + (lambda x x) 1, with the + function can only be applied on constants. It doesn't make sense to apply + on abstractions, so it cannot be reduced to a value. Those two are invalid expressions, and we will add error handling mechanism for them in the future. Another set of expression is those that can be reduced but it never stops, such as (lambda x x x) (lambda x x x). It is an infinite loop, and the evaluator runs forever. The evaluator needs to take both of these expressions into consideration, too.

Defective Evaluation Strategy
I never inspected our current evaluation strategy in the past posts, so I am going to discuss it in detail and find the defects. Currently, the evaluator obeys the following rules:
    1) Evaluate the body of an abstraction before returning it;
    2) Evaluate both parts of the application first, then perform beta-reduction for once, and return the result.

In rule 2), I perform beta-reduction for only once because I assume after performing rule 1) to an abstraction expression, it will be simple enough so no reducible expressions could be generated by applying it to any expression. For example, for expression (lambda x (lambda y y) x) 1, before applying the abstraction to 1, it will be simplified to (lambda x x). After that, we apply it to 1 and we get the result 1. However, this is not always true. The expression mentioned at the beginning of this post is a good counterexample. And we probably have this problem when providing an abstraction as a parameter to another abstraction.

The defective assumption for rule 2) is the root cause of "partially reduced problem". Besides that, rule 1) is also a stupid choice. Consider expression (lambda x 7) (lambda x (lambda y y y) (lambda y y y)). Apparently, the result is 7, but if we evaluate the body of an abstraction first, we will go into an endless loop.

From the above analysis, we learned that performing beta-reduction only once on an application may not always give a value, and evaluating the body of an abstraction first is not a good strategy. Based on this, we are going to propose a robust evaluation strategy.

Robust Evaluation Strategy
Our robust evaluation strategy states the following rules:
    1) Return an abstraction expression directly, without evaluating its body;
    2) Keeps evaluating the expression until we get a value.

We can easily change the evaluator to use this new strategy. Here is the code:
// file: eval.c
TreeNode * evaluate(TreeNode *expr) {
    TreeNode* result = expr;
    if(expr!=NULL) {
        switch(expr->kind) {
            case IdK:
            case ConstK:
            case AbsK:
                return expr;
            case AppK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                if(expr->children[0]->kind==IdK) {
                    // expand tree node for builtin functions
                    BuiltinFun* fun = lookupBuiltinFun(expr->children[0]->name);
                    if(fun!=NULL) {
                        expr->children[0] = (fun->expandFun)();
                    }
                }
                return evaluate(betaReduction(expr));
            case PrimiK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                // only perform primitive operation if operands are constants
                if(expr->children[0]->kind==ConstK
                    && expr->children[1]->kind==ConstK) {
                    result = evalPrimitive(expr);
                }
                return result;
            default:
                fprintf(errOut,"Unkown expression kind.\n");
        }
    }
    return expr;
}

Abstractions are returned just like identifiers and constants, and we also evaluate the result of a beta-reduction. One thing I need to mention here is that we have expressions which cannot be reduced to values, as described in "Evaluation Values" section. For those invalid expressions, the evaluator could identify it. For example, for x y, we will treat x as an identifier of a builtin function and we will fail because it is not. For + (lambda x x) 1, after expanding + we will finally try to evaluate the primitive expression (lambda x x) `+` 1, and by checking its operands we will know it is invalid. We will add error handling mechanisms to keep it consistent with current computation model for these invalid expressions. For now, the evaluator just prints an error message and stop evaluating. For the expressions causing endless loops, I don't think we can find any method to determine whether an expression runs forever, because it is a halting problem. So our evaluator doesn't handle this case, and it will run forever. It is the programmer's responsibility to make sure there is no endless loop in the code. Here is the revised code:
// file: eval.c
TreeNode * evaluate(TreeNode *expr) {
    TreeNode* result = expr;
    if(expr!=NULL) {
        switch(expr->kind) {
            case IdK:
            case ConstK:
            case AbsK:
                return expr;
            case AppK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                if(expr->children[0]->kind==IdK) {
                    // expand tree node for builtin functions
                    BuiltinFun* fun = lookupBuiltinFun(expr->children[0]->name);
                    if(fun!=NULL) {
                        expr->children[0] = (fun->expandFun)();
                    } else {
                        fprintf(errOut, "Error: %s is not a builtin function.\n", expr->children[0]->name);
                        return expr;
                    }
                }
                return evaluate(betaReduction(expr));
            case PrimiK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                // only perform primitive operation if operands are constants
                if(expr->children[0]->kind==ConstK
                    && expr->children[1]->kind==ConstK) {
                    result = evalPrimitive(expr);
                } else {
                    fprintf(errOut, "Error: %s can only be applied on constants.\n", expr->name);
                }
                return result;
            default:
                fprintf(errOut,"Unkown expression kind.\n");
        }
    }
    return expr;
}

With this robust evaluation strategy, we are able to get rid of the "partially reduced problem".

2011年4月6日星期三

Constants and Builtin Functions

In last series of posts "A Simple Lambda Calculus Evaluator", we implemented a very simple evaluator for pure lambda calculus. Though it performs reductions to lambda calculus expressions, it is still far from a real programming language. In this series of posts, I would like to add some important features to make it evolve towards a real programming language, including constants and builtin functions, a new evaluation strategy, recursion and text machines. Since this series is inspired by the paper "Programming Languages and Lambda Calculi", I will follow the terms defined in it.

Natural numbers could be encoded using basic lambda calculus. However, this is not very convenient. In this post, constants and builtin functions will be added to the evaluator as in many real programming languages, though it destroys the purity.

Constants
First, we will add the most common constants, integer, to our evaluator. To make the evaluator recognize integer values, both positive and negative, the regular expression and rules are added to file scanner.l:
// file: scanner.l

lambda    "lambda"
integer   [+-]?[0-9]+
// others remain as before

%%
    /* reserved words */
{lambda}        {return LAMBDA;}

    /* constants */
{integer}       {return INT;}

// others remain as before
%%
And the token definition in parser.y:
// file: parser.y

%token  INT

%%
A new expression type named ConstK should be added for constants. The tree node structure should also be updated to hold the integer value. I will not display the code here. The parsing rule is very straightforward: create a tree node of type ConstK, and save the value in it.
// file: parser.y

expression      : ID    
                    {
                        $$ = newTreeNode(IdK);
                        $$->name = stringCopy(yytext);
                    }
                | INT
                    {
                        $$ = newTreeNode(ConstK);
                        $$->value = atoi(yytext);
                    }
                | // other rules are ignored here
Now, we need to update our two main algorithms, FV and substitute, for this new expression. For FV, since it is not a variable, the result should be an empty set. For substitute, we don't do any substitution and just return the constant:
    c[x := N] = c, if c is a constant
It's time to update our implementation. The FV() function will be updated to:
// file: eval.c

static VarSet * FV(TreeNode *expr) {
    VarSet* set = NULL;
    VarSet* set1 = NULL;
    VarSet* set2 = NULL;
    switch(expr->kind) {
        case IdK:
            set = newVarSet();
            addVar(set,expr->name);
            break;
        case ConstK:
            set = newVarSet();
            break;
        // ...
    }
    return set;
}
And for substitute(), we just return the constant expression:
// file: eval.c

static TreeNode *substitute(TreeNode *expr, TreeNode *var, TreeNode *sub) {
    // ...
    switch(expr->kind) {
        case IdK:
            // ...
        case ConstK:
            return expr;
        // ...
    }
    return expr;
}
Finally, if a constant expression is evaluated, itself should be returned:
// file: eval.c

TreeNode * evaluate(TreeNode *expr) {
    if(expr!=NULL) {
        switch(expr->kind) {
            case IdK:
            case ConstK:
                return expr;
            // ...
        }
    }
    return expr;
}
We have finished adding constant to our lambda calculus evaluator, nice and easy. And don't forget to update the printExpression() function to print constants to user.

Builtin Functions
We have constants now, but it is still useless because we can't do computation on them. We are going to add two sets of operators: arithmetic operators and comparison operators. The arithmetic operator set includes six basic ones: plus(+), minus(-), times(*), over(/), modulo(%) and power(^). And the comparison operator set includes less than(<), equal(=), greater than(>), less than or equal(<=), not equal(!=) and greater than or equal(>=). Each of them is a binary operator, and we could simple add a primitive operator for each one which takes two arguments. However, this is not a good idea. In our current lambda calculus model, every function can only be applied on one argument. If we add the primitive operators directly to the evaluator, it would make our model inconsistent. Our goal is to make these operators behave as a normal function so programmers aren't required to distinguish whether this is a function or primitive operator.

As stated in the past post, any function that takes two arguments can be treated as a function that takes one argument and returns another function that takes one argument. So we can apply the same method to the operators. Our solution is: when an operator is encountered, expand it as a function that takes one argument and returns another function, which take one argument and returns the result of applying the corresponding operator to those arguments. Take + as an example:
     +    => (lambda x (lambda y x `+` y))
The `+`(enclosed with backticks) stands for primitive plus. We still have primitive operators internally, but the programmer doesn't need to be aware of it. I'd like to call the operator before expanding as builtin function, and the binary operator as primitive operator. The builtin function looks the same as others functions defined by the programmer.

Now, let's implement the builtin functions.

Arithmetic Functions
First, we should make our scanner recognize the operator symbols. Since an operator symbol represents a builtin function, we will treat it as an identifier - identifier for a function :). The identifier definition in scanner.l is changed to:
// file: scanner.l

// ...
identifier  [A-Za-z_]+|[+\-*/%^<=>]
// ...

%%

Now, the arithmetic builtin functions are analyzed as IdK tokens. Nothing should be updated for the parser and we get a tree node of type IdK for each builtin function.

The evaluation logic needs to be rework for builtin functions. When evaluating an application expression, the left child tree node should be expanded if it is a builtin function. So the evaluate() function is revised to:
// file: eval.c
TreeNode * evaluate(TreeNode *expr) {
    if(expr!=NULL) {
        switch(expr->kind) {
            case IdK:
            case ConstK:
                return expr;
            case AbsK:
                expr->children[1] = evaluate(expr->children[1]);
                return expr;
            case AppK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                if(expr->children[0]->kind==IdK) {
                    BuiltinFun* fun = lookupBuiltinFun(expr->children[0]->name);
                    if(fun!=NULL) {
                        expr->children[0] = (fun->expandFun)();
                    }
                }
                return betaReduction(expr);
            default:
                fprintf(errOut,"Unkown expression kind.\n");
        }
    }
    return expr;
}
We examine the left child tree node and replace it with the expanded tree if it is of IdK type and is a builtin function. The lookupBuiltinFun() and other definitions related to builtin functions are all defined in file builtin.c. For each builtin function, we provide a corresponding expand function which returns the expanded tree for it. To save space, I won't duplicate the codes here. Take + as an example, it will be expanded as follows:
+ => (lambda x (lambda y x`+` y)):
       AbsK
        /\
       /  \
 IdK("x") AbsK
           /\
          /  \
     IdK("y") ?

We haven't add the primitive node yet! It is a tree node and we need to add a new expression type PrimiK for it.
// file: globals.h

/* expression types */
typedef enum { IdK, ConstK, AbsK, AppK, PrimiK } ExprKind;
We could reuse the current tree node structure to represent a primitive operator: set operator name as the node name and keep the left hand side operand in the first child node and right operand in the second one. So the "?" place in the above tree can be replaced with:
x `+` y:
        PrimiK("+")
           /\
          /  \
    IdK("x") IdK("y")

This PrimiK expression is hidden from the programmer, so no FV() or substitute() could be applied on it. When evaluating a PrimiK expression, we only perform primitive operation if both operands are reduced to constants, because it doesn't make sense to do arithmetic on variables or functions.
// file: eval.c
reeNode * evaluate(TreeNode *expr) {
    TreeNode* result = expr;
    if(expr!=NULL) {
        switch(expr->kind) {
            case IdK:
            case ConstK:
                return expr;
            case AbsK:
                expr->children[1] = evaluate(expr->children[1]);
                return expr;
            case AppK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                if(expr->children[0]->kind==IdK) {
                    BuiltinFun* fun = lookupBuiltinFun(expr->children[0]->name);
                    if(fun!=NULL) {
                        expr->children[0] = (fun->expandFun)();
                    }
                }
                return betaReduction(expr);
            case PrimiK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                // only perform primitive operation if operands are constants
                if(expr->children[0]->kind==ConstK
                    && expr->children[1]->kind==ConstK) {
                    result = evalPrimitive(expr);
                }
                return result;
            default:
                fprintf(errOut,"Unkown expression kind.\n");
        }
    }
    return expr;
}
The primitive evaluation is implemented in file primitive.c. For each arithmetic operator, a tree node of type PrimiK is required as input, a tree node of type ConstK is returned after applying the operator to its operands. For +, we have:
// file: primitive.c
static TreeNode* plus(TreeNode* node) {
    TreeNode* result = newTreeNode(ConstK);
    result->value = node->children[0]->value + node->children[1]->value;
    return result;
}
The printExpression() should also be updated to print PrimiK expressions. Now, let's try it.
$ ./main
Welcome to Lambda Calculus Evaluator.
Press Ctrl+C to quit.

> + 1 2
-> 1 `+` 2
Unfortunately, the final answer doesn't show up. The result is an expression of PrimiK type. That's because after applying the second argument 2 to the inner function of +, we end up with the PrimiK expression, and it is returned without further evaluation. I call this a "partially reduced problem" because the returned expression is a redex, which means it can be reduced. Actually, this is a defect of our evaluator and it doesn't just happen for the current case. However, I will postpone the detail discussion to next post, and for now I am going to add a fix for our special case. Evaluate the expression if it is a PrimiK expression.
// file: eval.c
TreeNode * evaluate(TreeNode *expr) {
    TreeNode* result = expr;
    if(expr!=NULL) {
        switch(expr->kind) {
            case IdK:
            case ConstK:
                return expr;
            case AbsK:
                expr->children[1] = evaluate(expr->children[1]);
                return expr;
            case AppK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                if(expr->children[0]->kind==IdK) {
                    BuiltinFun* fun = lookupBuiltinFun(expr->children[0]->name);
                    if(fun!=NULL) {
                        expr->children[0] = (fun->expandFun)();
                    }
                }
                result = betaReduction(expr);
                // beta-reduction may result in primitive operations
                if(result->kind==PrimiK) {
                    result = evaluate(result);
                }
                return result;
            case PrimiK:
                expr->children[0] = evaluate(expr->children[0]);
                expr->children[1] = evaluate(expr->children[1]);
                // only perform primitive operation if operands are constants
                if(expr->children[0]->kind==ConstK
                    && expr->children[1]->kind==ConstK) {
                    result = evalPrimitive(expr);
                }
                return result;
            default:
                fprintf(errOut,"Unkown expression kind.\n");
        }
    }
    return expr;
}

Let's try it again:
$ ./main
Welcome to Lambda Calculus Evaluator.
Press Ctrl+C to quit.

> + 1 2
-> 3

Comparison Functions
Adding comparison functions is very similar to that of arithmetic functions. The only difference is the primitive evaluation method. For arithmetic functions, the return value should be a constant expression. For our current implementation, it is an integer. However, comparison functions should return a boolean value. Though we don't support builtin boolean values, we could use the method in post I to define booleans:

    TRUE   := (λx. (λy. x))
    FALSE  := (λx. (λy. y))

So the primitive evaluation for comparison functions will return an abstraction expression. And we can use it to construct an if-like expression.
$ ./main
Welcome to Lambda Calculus Evaluator.
Press Ctrl+C to quit.

> < 1 2
-> (lambda x (lambda y x))

> (< -1 0) 0 1
-> 0

Now, our evaluator can work as a simple calculator. Try more expressions and have fun. In the next post, we are going to solve the "partially reduced problem".

The full listing of code could be found here.