Demand-driven compilation

Demand-driven compilation, demand-driven computation, or query-based architectures appear to be a current trend in the community interested in compilers and programming language R&D. This novel model intends to makes IDEs more performant and easier to implement by rejecting the traditional pipeline approach to compilers, in favour of lazy (lazy as in non-strict and caching) computation. Laziness allows the user to make a small change to a single file, before requesting "compile my entire project." A demand-driven compiler might take this request, and further request "parse each file in this project." For each parse request, the query handler will recognise which files have changed, and simply return the cached ASTs for those which haven't. This request/cache model is used for all kinds of tasks within the compiler, including every pass.

Demand-driven compilers greatly resemble build systems. The Rock library used in SydML and Sixten is itself a build system.

Examples

Further reading

• rustc demand-driven-compilation

• Query-based compiler architectures

• Three Architectures for a Responsive IDE

• Practical Algorithms for Incremental Software Development Environments

• Query-based compiler architectures

• A journey through incremental computation

• Making Fast Incremental Compiler for Huge Codebase

• How to load 1m lines of Ruby in 5s

• Compilers are Databases

• Anders Hejlsberg on Modern Compiler Construction

• Responsive compilers