Software Factory Best Practices from Open-Source Projects (3/4): Factories That Filter by Evidence

Sat, 10 Oct 2026 10:00:00 +0900

“Your test is NOT VALID if it passes with USE_SYSTEM_BUN=1.” – Bun CLAUDE.md¹

Dohyun Jung - Principal Consultant, ROBOCO

This is the third post in the four-part series “Software Factory Best Practices from Open-Source Projects.”

The series

The Criteria and 12 Repositories: what counts as a factory, and where factories are running
Who Merges? Risk-based delegation and permission structures
Factories That Filter by Evidence: proving a test “fails before the fix” (this post)
Turning Failures into Rules: self-improvement loops and their limits

The quote at the top comes from the agent instructions in the Bun repository. USE_SYSTEM_BUN=1 is a setting that runs the tests against the existing Bun release, before the fix. In other words, the rule says that “a test that also passes on the unfixed code proves nothing.”

Testing on ROBOCO

Software Factory Best Practices from Open-Source Projects (3/4): Factories That Filter by Evidence

Dohyun Jung - Principal Consultant, ROBOCO