xtc

The xtc (eXTensible Compiler) project is exploring novel programming languages and tools to improve the expressiveness, safety, and efficiency of complex systems.

• Features
• Publications
• Distribution
• Mailing Lists
• Acknowledgements

Features

Highlights of xtc’s features include:

• Rats!, a parser generator supporting modular syntax and declarative AST specifications.
• Grammars for C with common gcc extensions and for Java, versions 1.4, 1.5, and 1.7.
• SuperC, a tool for preprocessing and parsing C across all configurations.
• Support for statically and dynamically typed ASTs and traversing them with dynamically dispatched visitors.
• Support for generating parse trees that can be processed with the same visitors as the corresponding ASTs.
• Support for programming language types and type checking C and Java programs.
• Support for concrete syntax, i.e., the ability to create AST fragments with source code instead of programmatically building them.
• Jeannie, which nests Java and C statements and expressions within each other to simplify multi-lingual programming.
• Blink, a portable debugger for both C and Java.
• Jinn, a dynamic bug detector for the Java Native Interface.

Additionally, xtc includes support for common tool tasks, such as processing command line flags, managing file search paths, setting character encodings, and reporting program errors.

Discontinued components include:

• Typical, a domain-specific language for implementing a compiler’s semantic analysis.
• XForm, an AST query and transformation language.
• A front-end for Overlog, which enables the declarative specification of overlay networks.
• C4, the Cross-Cutting C Compiler, which adds aspects to the C language.

These components are still accessible through version 1.15.0 of the source release (see below).

Publications

Select publications:

• SuperC: Parsing all of C by taming the preprocessor.
  By Paul Gazzillo and Robert Grimm. To appear in Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’12), June 2012.
• Jinn: Synthesizing dynamic bug detectors for foreign language interfaces.
  By Byeongcheol Lee, Ben Wiedermann, Martin Hirzel, Robert Grimm, and Kathryn S. McKinley. In Proceedings of the ACM SIGPLAN 2010 Conference on Programming Language Design and Implementation (PLDI ’10), pp. 36–49, June 2010.
• Debug all your code: Portable mixed-environment debugging.
  By Byeongcheol Lee, Martin Hirzel, Robert Grimm, and Kathryn S. McKinley. In Proceedings of the 2009 ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA ’09), pp. 207–226, October 2009.
• Declarative syntax tree engineering (or, one grammar to rule them all).
  By Robert Grimm. NYU CS Technical Report TR2007-905, November 2007.
• Typical: Taking the tedium out of typing.
  By Robert Grimm, Laune Harris, and Anh Le. NYU CS Technical Report TR2007-904, November 2007.
• Jeannie: Granting Java native interface developers their wishes.
  By Martin Hirzel and Robert Grimm. In Proceedings of the 2007 ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA ’07), pp. 19–38, October 2007.
• Better extensibility through modular syntax.
  By Robert Grimm. In Proceedings of the ACM SIGPLAN 2006 Conference on Programming Language Design and Implementation (PLDI ’06), pp. 38–51, June 2006. Talk.
• patch (1) considered harmful
  By Marc Fiuczynski, Robert Grimm, Yvonne Coady, and David Walker. In Proceedings of the 10th Workshop on Hot Topics in Operating Systems (HotOS ’05), pp. 91–96, June 2005.

See Robert Grimm’s publications for a list of all papers and talks on xtc.

User guides:

• Rats!
• SuperC
• Jeannie (PDF)
• Blink (PDF)

xtc Distribution

xtc is mostly released under the GNU GPL version 2. The main exception is the Rats! runtime, which is released under the GNU LGPL version 2.1. See xtc’s sources and Javadoc documentation for a detailed list of these classes.

xtc’s current distribution is version 2.3.1, which was released 4/4/12. It includes the following files:

• xtc-core.zip: The source distribution.
• xtc-testsuite.zip: The test suite.
• xtc.jar: The binary distribution.
• rats.jar: The Rats! stand-alone binary (including runtime).
• rats-runtime.jar: The runtime for parsers generated by Rats! with the -lgpl command line option.
• history.html: The release history and notes.

Note that the binary distribution xtc.jar contains all classes contained in the parser binary rats.jar and the parser runtime rats-runtime.jar.

Version 1.8.0 of the source distribution is still available to ensure the repeatability of the experimental results reported in the PLDI ’06 paper “Better Extensibility through Modular Syntax.” Version 1.15.0 is the last version to contain several discontinued tools. We preserve both source code and test suite.

The source distribution comes with extensive documentation, including installation instructions (step-by-step.html in xtc’s root directory), an overview of our setup (development.html), and Javadoc-generated class documentation (execute “make doc”).

This page provides an introduction to Rats!, our parser generator. It is part of xtc’s Javadoc documentation (hence the @link mark-up) and uses examples from Rats!’ own grammar as well as from our Java and C grammars (look in the src/xtc/parser and src/xtc/lang directories in the source distribution).

The binary distribution is invoked from the command line by typing:

java -cp xtc.jar tool-class-name

where valid tool class names include:

• xtc.parser.Rats for the Rats! parser generator.
• xtc.lang.FactoryFactory for the concrete syntax tool.
• xtc.lang.cpp.SuperC for the SuperC tool.
• xtc.lang.jeannie.Jeannie for the Jeannie compiler.

When invoked without additional arguments, each tool prints usage instructions including recognized command line arguments.

xtc’s dependencies include the following Java packages:

• JUnit for performing unit tests.
  • junit.jar
• JavaNCSS for collecting source statistics.
  • ccl.jar
  • javacc.jar
  • javancss.jar
  • jhbasic.jar
• JFlex for building the SuperC scanner.
  • JFlex.jar
• JavaBDD for reasoning about conditionals in SuperC.
  • javabdd.jar

For the source release, the corresponding jar files should be placed into the bin directory of the distribution. The JUnit and JavaBDD jar files should also be added to the class path.

For your convience, dependencies.zip contains the jar files for JUnit 4.8.2, JavaNCSS 32.53, JFlex 1.4.3, and JavaBDD 1.0b2.

The source distribution of the Marco macro processor is available as a separate download.

Mailing Lists

• xtc-announce: A low-volume, moderated mailing list for important announcements about xtc and Rats! (including new releases).
• xtc-develop: A closed mailing list only for active xtc developers with CVS access.

Acknowledgements

xtc is currently being developed by Paul Gazzillo, Robert Grimm, and Robert Soulé.

Major contributors are Marc Fiuczynski, Buğra Gedik, Laune Harris, Martin Hirzel, Anh Le, Byeongcheol Lee, Kathryn McKinley, Joe Pamer, Ben Wiedermann, and Marco Yuen.

Further contributors are Ken Britton, Chris Capel, Christopher Conway, Christine Flood, Hunter Freyer, Marek Gilbert, Ben Goldberg, Eric Hielscher, Thomas Huston, Chris Jones, Yonas Jongkind, Dejan Jovanović, Adam Kravetz, Stacey Kuznetsov, Petar Maymounkov, Thomas Moschny, William Moy, Janus Dam Nielsen, Karen Osmond, Robin Lee Powell, Matt Renzelmann, Sukyoung Ryu, Jacob Shufro, Uwe Simm, and Mengtao Sun.

xtc is used by:

• Fortress: A new language for high-performance computing.
• Matchete: Java extended with pattern matching.

This material is based in part upon work supported by the National Science Foundation under grants CNS-0448349, CNS-0615129, and CCF-1017849. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.