Skip to content

0013. Build: config-gen is Python, and its protobuf is pinned

Date: 2026-09-29

config-gen moved from a C++ tool to a Python script whose only dependency, protobuf, is pinned. Switching parsers also exposed two default-config files that were malformed JSON. Both sections are extracted verbatim from Tag Firmware Build Reproducibility.

Decision and evidence

default_config.c was produced by config-gen.cc, a C++ tool linked against whichever libprotobuf a developer's vcpkg baseline supplied. The manifest hashed config-gen.cc and recorded nothing about that library, so a committed, shipped file was governed by a per-machine dependency no check could see -- the same shape of hole as fmpp, with a much larger library behind it.

It is now config-gen.py, depending on protobuf alone, pinned in embedded/proto-c/requirements.txt to 5.28.1. That version is not a preference: it is the protobuf the pinned nanopb 0.4.9.1 distribution itself runs on, whose generator-bin ships protoc 28.1. One protobuf now governs both generated outputs in those directories instead of two free to drift.

Two decisions make the pin cheap to hold:

  • The schema comes from a descriptor set, not generated modules. Generated Python carries a gencode version the runtime checks and can refuse, which makes an old protoc against a newer runtime a standing hazard. A descriptor set is wire-format data with no such gate.
  • protoc comes from the nanopb distribution, beside nanopb_generator, so one is present exactly when regeneration is possible and nothing is installed for it.

The build creates its own virtualenv from requirements.txt rather than using whichever interpreter is on PATH. That is not fastidiousness: on the machine this was developed on, python3, pip and CMake's find_package(Python3) resolved to three different installations, and nanopb's binary distribution bundles a fourth interpreter of its own. An uncontrolled interpreter choosing the version that renders a committed file is precisely the defect being removed, so it is not an acceptable way to remove it.

The switch was measured, not assumed. Output is byte-identical to config-gen.cc's for all five distributed variants -- verified against a rebuild of the C++ tool, and under protobuf 4.25.3, 5.28.1, 5.29.5 and 7.36.2. The pin is therefore conservative rather than load-bearing: if it has to move, it can.

Consequences: two config files parsed only by accident

Switching parsers found a defect that had been in the tree invisibly. prestag-proto-c/default-config.json and prestagraw-proto-c/default-config.json both ended with a trailing comma before the closing brace, which RFC 8259 disallows. The C++ JSON parser accepted it; a strict parser does not.

Those files parsed only because of a leniency nobody chose, nothing recorded, and no check could observe. A developer on a stricter libprotobuf would have hit a hard failure on one distributed variant out of five with nothing in the tree to explain why. Both are corrected, and neither correction moves an emitted byte -- the comma changed what the file hashed to and nothing about what it meant.

Worth recording as the concrete form of a general risk: an unpinned generator can make a malformed input look valid, and the freshness check cannot see it, because that check compares recorded input hashes rather than re-rendering output. See the build worklist.