communitylabs/cascadia-phi-4-int4

Phi-4 — Cascadia int4 shards

Pre-exported Cascadia inference shards for microsoft/phi-4, so operators can deploy without running an export themselves.

These are OpenVINO IR artifacts for Cascadia's ov-runtime engine. They are not loadable by transformers — see Using these below.

Presets

Preset	Path	Stages	Layers	Size
1	int4/stages-1	1	40	7.6 GB

Each preset is a separate export: a dense model's layer split is fixed at export time, so a 2-stage tree cannot serve a 4-node pipeline. Pick the one matching your fleet.

int4/stages-N/
  pipeline_config.json     # model geometry: layers, heads, rope, arch tag
  stage_0/ ... stage_N-1/  # each: openvino_model.xml/.bin + stage_config.json
  tokenizer/               # tokenizer.json + configs

cascadia.json at the repo root is the machine-readable index (sizes, checksums, export version) that Cascadia's model registry reads.

Using these

hf download communitylabs/cascadia-phi-4-int4 --local-dir ./cascadia-phi-4-int4
cascadia worker --model ./cascadia-phi-4-int4/int4/stages-1 --engine ov-runtime

The worker takes a local path, never a HuggingFace id — Cascadia workers never download or convert models at serve time.

Provenance

Exported with Cascadia's tools/export_shards.py, export_version: v5_canonical_inputs, int4 via NNCF (group size 128).

Quantization is lossy. For anything quality-sensitive, benchmark against the source model rather than assuming parity.

License

MIT, inherited from microsoft/phi-4. These shards are a derivative work; the upstream license and its terms apply.