stencil

Stencil

Stencil is a programming language and data model for describing precise and efficient data structure for interchange between different languages.

Stencil is designed to give developers control over the layouts of data structures and generate efficient code in

The goal is to generate types that wrap buffers to enable interacting with data structures from disk approaching the performance of mmapping a C struct from disk, but with added safety and well-formedness checks.

Architecture

Type declaration frontend

Stencil has 3 frontends planned:

1. YaAST - shitty Yaml-based frontend I am using for testing. see examples/demo.yml
2. Stencil native - a native Stencil language that allows full expressiveness, see examples/ip.stencil for a rough plan
3. SSZ - convert the Pythonic SSZ definitions into portable definitions to take advantage of SSZ's unique benefits

Only the first exists today. I am focused on making the core of Stencil robust and powerful enough to suit its target use cases before exposing the full expressiveness of its internal representation. Right now we can suffer through that nonsense.

Middle end intermediate representations

Stencil frontends target the layout reprensentation. This representation reasons more explicitly about the positions of fields within a structure and how different structures relate to each other. This is where we generate different instances of types when we need to talk about the same abstract type in different contexts, like builders for complex types.

From the layout representation, we generate Stencil MIR, or "middle intermediate representation". This describes types more concretely and generates function implementations for constructors, getters, setters, etc. We also do typechecking in this stage to ensure our generated code is self-consistent.

This is also the stage where we can inject functions for introspecting the types in specialized ways, like generating the merkle hasher functions for SSZ types.

Langauge-specific backend

There are back ends for each language we target.

Each backend needs a LangIR that it converts to from MIR. This is where we can resolve type names to local versions and make final checks. This wouldn't be a full native AST, but can represent a subset of the target language syntax that's useful for Stencil.

We can apply additional language transformations to the LangIR (adding annotations, attributes, instrumentation, etc, as desired) for additional functionality.

Then we can actually generate code from it. This phase isn't that sophisticated. It will be more sophisticated in the future when we have need for that.

Supported backends

The top priorities are:

• Python (mostly implemented, but being iterated on)
• Rust

These are what I have planned for my own uses.

Secondary priorities:

These I may get to if I have some desire to figure it out.

• untyped JavaScript (I don't want to put up with npm so just this)
• Java / Kotlin
• C
• Go (icky but necessary since Go serialization sucks)

Contributions encouraged:

I either don't want to or don't trust myself to write/test code in these languages, but I see them as being very valuable to have:

• TypeScript
• C#
• Swift
• Racket
• C++
• Bash
• OCaml

or any other language you personally have a use for.