HuggingFaceBio/Carbon-8B-GGUF

Carbon-8B GGUF

Running with llama.cpp

Carbon is a base DNA model (not instruction-tuned). Use the raw completion front-end so llama.cpp does not apply any chat templating:

llama-completion \
  -hf HuggingFaceBio/Carbon-8B-GGUF:Q8_0 \
  --prompt "<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG" \
  -n 128 --temp 0 --no-display-prompt

If you use llama-cli instead, pass -no-cnv so it does not wrap the prompt in assistant-style markers (otherwise the model may emit annotation tags such as <protein_coding_region> instead of DNA).

Available files

Quant	File	Size	SHA256
BF16	carbon-8b-bf16.gguf	15.4 GB	dd9773602737f9bf89672d56e4d55146edcdb8ac98e6392f3aada8e997d5b238
Q8_0	carbon-8b-q8_0.gguf	8.2 GB	ba5f7794d0768e639fcb0fe860ff7340b09bfaad5e0e2f2c021a4f49029352dd
Q6_K	carbon-8b-q6_k.gguf	6.6 GB	db26cf21b45ed2c07722002ec93c7f7ab707df8bb5e2a12c06c8139e2412e6e7
Q5_K_M	carbon-8b-q5_k_m.gguf	5.9 GB	5d0f351362bd1dd9a5944e1a1cf5d7769fe5d356e9475a6e73f935133a0bdd8c
Q4_K_M	carbon-8b-q4_k_m.gguf	5.2 GB	b4fa68a7722b446a55198d1eceb6f00899af12b47e23fab11dc7d7b6a34e905c

K-quants are produced with --token-embedding-type Q8_0 --output-tensor-type Q8_0 so the sensitive embedding/output tensors stay at Q8_0 precision (important for this model's tight DNA-vs-text logit margins).

Requires a recent llama.cpp

HybridDNATokenizer support was merged in ggml-org/llama.cpp#23410, so any build from master after that works:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build -j

Files

File	Quant	Size
carbon-8b-bf16.gguf	bf16 (lossless from source)	16 GB

Usage

Download

hf download HuggingFaceBio/Carbon-8B-GGUF carbon-8b-bf16.gguf --local-dir .

Basic DNA completion

./build/bin/llama-completion -m carbon-8b-bf16.gguf \
    -p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
    -n 64 --temp 0 -no-cnv

Metadata-conditioned generation

./build/bin/llama-completion -m carbon-8b-bf16.gguf \
    -p '<vertebrate_mammalian><protein_coding_region><dna>ATGCGCTAG' \
    -n 64 --temp 0 -no-cnv

Speculative decoding with Carbon-500M draft (~2x speedup)

The 500M shares the HybridDNA vocab, so it's a near-ideal draft. Measured ~2.1x speedup at temp=0 with 87% accept rate on DNA prompts. Grab the draft GGUF first:

hf download HuggingFaceBio/Carbon-500M-GGUF carbon-500m-bf16.gguf --local-dir .

Then run the standalone tool with --model-draft:

./build/bin/llama-speculative \
    -m  carbon-8b-bf16.gguf \
    -md carbon-500m-bf16.gguf \
    -p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
    -n 256 --temp 0

Or serve the 8B with the 500M draft (llama-server accepts the same -md flag):

./build/bin/llama-server \
    -m  carbon-8b-bf16.gguf \
    -md carbon-500m-bf16.gguf \
    --draft-max 16 --draft-min 1 \
    --port 8080

curl -s http://localhost:8080/completion -d '{
  "prompt": "<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG",
  "n_predict": 256,
  "temperature": 0
}' | jq -r .content

Likelihood scoring

The source card's Python score() function computes mean log-prob per DNA token. In llama.cpp the closest tools are llama-perplexity for corpus-level perplexity (perplexity = exp(-mean_logprob)):

# one prompt per line in dna_corpus.txt, each wrapped in <dna>...</dna>
./build/bin/llama-perplexity -m carbon-8b-bf16.gguf -f dna_corpus.txt --ppl-stride 0

Or llama-server with logprobs for per-token log-probabilities:

./build/bin/llama-server -m carbon-8b-bf16.gguf --port 8080 &
curl -s http://localhost:8080/completion -d '{
  "prompt": "<dna>GGGCTATAAAGGCCATCGATCGATCGATCGATCGATCGATCG</dna>",
  "n_predict": 0,
  "n_probs": 1
}' | jq '.completion_probabilities'

Long context with YaRN (65k tokens ≈ 393 kbp)

Mirrors the source card's rope_scaling = {type: yarn, factor: 4.0, original_max_position_embeddings: 32768}:

./build/bin/llama-completion -m carbon-8b-bf16.gguf \
    -c 65536 --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 32768 \
    -p '<dna>...' -n 64 --temp 0 -no-cnv

Base-pair-level generation (FNS branch) — not supported

The revision="fns" example from the source card needs custom modeling code (factorized nucleotide supervision head), which only the Python transformers path can load. llama.cpp can't run that branch.

See also

• Source weights: HuggingFaceBio/Carbon-8B
• Other GGUF variants: 500M · 3B · 8B

License

Apache-2.0, inherited from the source model.