Translating Failing Transformations

After deliberating about it for a long time (several years even), I finally implemented a new translation scheme for the Stratego Compiler. The old scheme used the C feature of setjmp/longjmp to deal with failing transformations. This provided the opportunity to go from using C as an assembly language, where an entire Stratego program was compiled to a single C function using gotos for control-flow, to a more idiomatic style of C programs in which each strategy definition was compiled to a C function. The setjmp/longjmp feature elegantly dealt with the notion of a failure by declaring a choice point (with setjmp) and jumping to it from anywhere (with longjmp). However, since choice points are the control-flow mechanism in Stratego, the speed of programs depends heavily on the cost of this feature. On Intel machinery (running Linux) this is not a big issue, but on Apples and Suns (RISC machines) the number of registers saved at each choicepoint is quite expensive; at least that is a theory about possibilities for improving the performance of Stratego programs.

Eelco Dolstra suggested a long time ago to return NULL to indicate failure of a strategy. Indeed, this representation closely matches the formal operational semantics of the language, in which the set of terms is extended with a failure value; exactly the ATerm data-type extended with an extra value (NULL). While conceptually simple, the idea seemed too disruptive to the run-time system, compiler, and native parts of the library to start work on, and there were always plenty of other things to do. (Especially considering the fact that the change does not add one bit of functionality.)

After the recent Stratego Core refactoring and clean up of the intermediate representation and the compiler, and solving lots of outstanding issues, this translation scheme refactoring came into view again. Interestingly, it took less then a week real time to achieve, which included a camping trip, a visit to Philips Research, and reading a couple of theses and articles. So either the problem was never that problematic to start with, or the recent drastic refactorings of the Stratego/XT source tree, configuration, and build system has paid off. Also the change to baseline-based bootstrapping (from self-based bootstrapping) has enormously simplified the process of changing the foundations of the compiler. Finally, the reliance on a solid continous build and release system gives one much more confidence in committing to a new version to bootstrap against. If there is a general lesson here: refactoring and continuous integration pay off.

As for the new translation itself; it is pretty standard fair. Have a look at s2c-ng.str and compare it to the classic s2c.str. The only flaw is that I had to resort to producing code with gotos. Noteworthy about the new version is the use of concrete syntax of Stratego and C almost everywhere, which makes a difference between night and day in maintaining the translation scheme. For example, the following rule defines the translation of the crucial guarded choice construct:

  TranslateStrat(|S,F) :
    |[ s1 < s2 + s3 ]| ->
    stm|[
      {
        ATerm ~id:x = t;
        ~stm:<translate-strat(|Next,F')>s1
        ~stm:<translate-strat(|S,F)>s2
        ~id:F' : t = ~id:x; 
        ~stm:<translate-strat(|S,F)>s3
      }
    ]|
    where <not(Next)> S; new => x; new => F'

Another interesting new feature is the collection of code fragments using dynamic rules, and the synthesis of the target program from these fragments afterwards; in contrast to the old method in which the source program was traversed for each type of fragment `driven by' the target program.

I'm writing this blog while waiting for the bootstrap build to go through, but from the regression tests for the compiler the implementation seems to work fine. What is not clear yet, is the performance improvement this will bring, if any. Another feature of the current translation scheme that seems to have negative impact on the performance of Stratego programs, especially on Apples and Suns, is the use of nested functions in the target C code. This is a feature of GCC, and therefore fraught with portability and performance problems. The implementation using trampolines also does not go very well with the tendency to forbid executable code on the stack. So this aspect of the translation is next in line to be changed. The basic idea is again simple and the use of fragment collection was partly motivated by this change. More later.