pyc

The strict dialect

pyc --strict compiles a dialect of Python 3 defined by a single guarantee: your program has CPython's semantics, or it does not compile.

The contract

Static compilation of Python always runs into programs that CPython can run and that no static type assignment can represent without boxing every value. Compilers usually handle these programs in one of three ways: box everything and lose the speed, deviate quietly from CPython, or ask for annotations. Strict pyc does none of these:

Strict mode:

every accepted program prints exactly what CPython prints. Every program that can't meet that is refused at compile time, with a diagnostic. A gap in this guarantee is a bug, tracked as a silent wrong answer.

A dialect by restriction

Strict pyc adds no syntax. Every strict program is an ordinary Python 3 program that runs unchanged under CPython, so you can develop and test with CPython and ship a pyc binary. The dialect is the subset of Python 3 whose programs are monotonic: every variable, field and container element can be given one concrete representation for the life of the program.

That is a larger subset than it sounds. Polymorphism is fine, as long as each object is consistent. A function called with ints and with strings gets two compiled versions, a class used with several field types gets several layouts, and related classes dispatch through a shared base. The analysis finds all of this without annotations. What's excluded is a single slot holding unrelated types, and the features that defeat whole-program analysis (eval, exec, run-time setattr).

Strict today

  • Every type violation is fatal, and no accommodation applies.
  • The end-to-end suite is compiled under --strict and compared with CPython. A strict corpus sweep checks the 77 shedskin programs the same way. Every program must match CPython or be refused.
  • Integer literals past 64 bits are refused rather than wrapped.
  • Python 2 idioms that CPython 3 rejects, such as raise "message", are refused.
counter.py
class Counter:
    def __init__(self):
        self.total = 0
    def add(self, v):
        self.total = self.total + v

c = Counter()
c.add(1)
c.add(2.5)
print(c.total)
$ pyc --strict counter.py
counter.py:5:69: error: expression has mixed basic types:( int64 float64 )
            self.total = self.total + v
                                       ^
  called from counter.py:8
...
fail: program does not type

The fix is the one that makes your intent explicit, and it changes nothing under CPython: self.total = 0.0.

Accommodations

The default mode, --permissive, exists to compile more existing code. It allows a short list of accommodations: behaviour that goes beyond CPython 3 or beyond static typing in order to accept a program. Every accommodation follows the same three rules:

  1. It sits behind a named flag, on under permissive and off under strict.
  2. It warns wherever it changes what the program would do under CPython.
  3. It reaches the generic analysis only through a callback interface. The analysis library itself knows nothing about Python.

Examples: widening an int/float slot to float, and wrapping a Python 2 string raise in an Exception. The complete audit is issues/171.

Toward a full dialect

The guarantee is in place. What makes strict pyc a language, rather than a compiler mode, is everything around it: a definition you can read, a promise that it won't move under you, and tools that speak it.

A written specification planned

Today, the strict language is whatever the analysis accepts. A dialect needs rules a programmer can read: which unions are representable, what int means, how None combines with other types, when a container element is “one type.” The rules should be stated in terms of what the types are, never in terms of how the analysis happens to find them.

Stability planned

A program that version N accepts under --strict, version N+1 accepts too, with the same output. The shedskin corpus and the end-to-end suite become a conformance suite, run against every release and against CPython.

Diagnostics as the interface under way

In a strict dialect, the refusal message is the type system's user interface. pyc already names both sources of a numeric union (“the int comes from line 3, the float from line 5”). Every refusal should point at where the values meet, cite the spec rule, and say what change would make the program typeable.

Checked annotations planned

Annotations parse today and, as in CPython, have no effect. Under --strict they will be verified against what inference found. They stay optional, since inference still does the work, but they let a module state its interface and turn a confusing downstream error into a clear one at the boundary.

A strict standard library planned

Every module in pyc_lib/ compiles under --strict with no accommodation, and its surface is documented in dialect terms. A strict program can then use the whole library with the same guarantee as its own code.

Integers, decided planned

int is 64 bits. Literals that don't fit are already refused under --strict. Arithmetic that overflows at run time is the remaining gap, and the dialect has to close it explicitly: either trap on overflow, or use a wider representation where the program demands one. Wrapping silently is not an option.

Tooling planned

The compiler already knows the type of every expression, and --html renders the type-annotated program. Those facts belong in the editor: a language server with inferred types on hover, refusals as live diagnostics, and go-to-definition that follows the resolved call graph.

The sequencing is on the roadmap.

What it will not do

  • Box. A slot that holds unrelated types has no representation in the dialect. Boxing would bring back the cost static compilation exists to remove, and it would hide the very unions the dialect is meant to surface.
  • Deviate to type a program. If a change makes pyc agree with some other compiler and disagree with CPython, it's wrong. CPython is the reference.
  • Add syntax. Strict programs stay valid Python, so CPython remains your interpreter, debugger and test runner.