Bitforge Precision Lab

Verified project card

# BitForge 1-bit

BitForge trains and compares a full-precision digit classifier, a strict one-bit
weight model, and a ternary-weight model. The low-bit students use straight-through
quantization during training and learn from both labels and the full-precision
teacher's softened output distribution.

The binary deployment artifact stores each matrix weight as one packed sign bit,
plus one floating-point scale per output channel and floating-point biases. The
project reports both classification accuracy and the measured inference payload.
Activations, scales, and biases remain floating point, so this is specifically a
one-bit **matrix-weight** experiment rather than a claim that every operation or
parameter is one bit.

## Verified results

| Variant | Test accuracy | Accuracy change |
| --- | ---: | ---: |
| FP32 teacher | 95.78% | reference |
| Packed binary matrix weights | 94.22% | -1.56 points |
| Ternary matrix weights | 95.11% | -0.67 points |

Each network has 4,810 parameters, including 4,736 matrix weights. The measured
inference payload fell from 19,240 bytes for FP32 parameters to 1,184 bytes for
packed signs, per-channel scales, and biases, a 16.25 times reduction. The `.npz`
container itself is 3,270 bytes because it also carries names, shapes, and archive
metadata. An independent reload of the packed signs reproduced 94.22% accuracy.

## Reproduce

```powershell
uv run python projects/bitforge-1bit/train.py
```

Evaluation snapshot

{
  "benchmark": "BitForge 1-bit",
  "parameters_per_variant": 4810,
  "matrix_weight_count": 4736,
  "test": {
    "fp32": {
      "accuracy": 0.9577777777777777,
      "cross_entropy": 0.1542476937174797
    },
    "binary_weight": {
      "accuracy": 0.9422222222222222,
      "cross_entropy": 0.2073044627904892
    },
    "ternary_weight": {
      "accuracy": 0.9511111111111111,
      "cross_entropy": 0.19603287428617477
    }
  },
  "storage": {
    "fp32_parameter_payload_bytes": 19240,
    "packed_payload_bytes": 1184,
    "container_bytes": 3270,
    "measured_payload_compression": 16.25
  },
  "precision_boundary": {
    "matrix_weights": "one packed bit in binary variant",
    "scales": "float32 per output channel",
    "biases": "float32",
    "activations": "float32"
  }
}

Backed-up artifact tree

• README.md
• __pycache__/app.cpython-311.pyc
• __pycache__/model.cpython-311.pyc
• __pycache__/packing.cpython-311.pyc
• __pycache__/train.cpython-311.pyc
• app.py
• artifacts/bitforge-1bit/binary_qat.safetensors
• artifacts/bitforge-1bit/binary_weights.npz
• artifacts/bitforge-1bit/evaluation.json
• artifacts/bitforge-1bit/fp32.safetensors
• artifacts/bitforge-1bit/ternary_qat.safetensors
• data/split_manifest.parquet
• model.py
• packing.py
• requirements.txt
• train.py