StartLux-Decision v1.1 4B Instant, W4A4 MSE

Saved symmetric W4A4 checkpoint for StartLux-Decision v1.1 4B Instant, single-pass readout. Weights are per-output-channel int4 codes in [-7, 7], stored in int8 tensors. Activations use the 4-bit grid: the static MSE threshold comes from w4a4-rotated-scales.json (layers[name]["mse"] / 7). Those thresholds were searched again on the same rotated snapshot, prompts, and sampler as the 8-bit book, with reconstruction qmax=7.

The base model is startlux-models/StartLux-Decision-v1.1-4B-Instant at revision 416088f77cfce09633906a3675c7cd2e5dabcf10, licensed CC-BY-NC-4.0. This repository is an adaptation of that model and keeps the same license.

W4A8 uses the same weight file with the 8-bit MSE book: River555/quantdecision-w4a8-mse. The W8A8 MSE pack is River555/quantdecision.

What is in this repo

File Role
decoder-w4.safetensors 248 decoder linear layers, *.weight_q int8 codes in [-7, 7] and *.weight_scale float32
w4a4-rotated-scales.json Calibration book. W4A4 MSE inference reads the mse field and divides by 7
manifest.json Export record and SHA-256 checks
quantization.json Variant pin: static W4A4, activation key mse

SHA-256:

  • decoder-w4.safetensors: 3c0c30cf75c88f4ee6fb8d2f8323d0d5f9b41861a7c79f84cce0f8a83d6d3318
  • w4a4-rotated-scales.json: c6406fdf9e0bfa39072356fb9255a45c508a5b47d3c593b505f6fb4d9244dc67

The pack was exported from the Smooth_fc_fc + fixed Hadamard R1 BF16 snapshot used for the November 9 Instant ablation (/root/work/instant-ablation-20261106/rotated-bf16). File sizes and hashes of that snapshot are listed in manifest.json under snapshot_files. The rounding matches FakeW8A8Linear with weight_bits=4 and activation_bits=4. That BF16 snapshot is not stored here.

This is not a Transformers or compressed-tensors checkpoint. AutoModel.from_pretrained will not apply these int4 weights by itself.

Load

Code: HeXiao-55/quant-decision.

from static_w8a8 import FakeW8A8Linear

# activation_absmax is layers[name]["mse"] from w4a4-rotated-scales.json
linear = FakeW8A8Linear(
    source_linear, activation_absmax,
    mode="static", backend="qdq", weight_bits=4, activation_bits=4,
)

weight_q and weight_scale in this file are the tensors that constructor stores.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for River555/quantdecision-w4a4-mse

Quantized
(6)
this model

Collection including River555/quantdecision-w4a4-mse