Like 59

Follow

Hugging Face Biology Research 204

Text Generation

Transformers

Safetensors

English

qwen3

dna

genomics

biology

variant-effect-prediction

conversational

text-generation-inference

License: apache-2.0

Model card Files Files and versions xet

Community 2

Deploy

Copy to bucket new

Use this model Instructions to use HuggingFaceBio/Carbon-X with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

Transformers How to use HuggingFaceBio/Carbon-X with Transformers:

Use a pipeline as a high-level helper

from transformers import pipeline

pipe = pipeline("text-generation", model="HuggingFaceBio/Carbon-X") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)

pip install -U transformers accelerate

Load model directly

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("HuggingFaceBio/Carbon-X") model = AutoModelForCausalLM.from_pretrained("HuggingFaceBio/Carbon-X", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks

Google Colab

Kaggle

Local Apps Settings

vLLM How to use HuggingFaceBio/Carbon-X with vLLM:

Install vLLM from pip:

pip install vllm

Start the vLLM server:

vllm serve "HuggingFaceBio/Carbon-X"

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:8000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "HuggingFaceBio/Carbon-X", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Use Docker

docker model run hf.co/HuggingFaceBio/Carbon-X

SGLang How to use HuggingFaceBio/Carbon-X with SGLang:

Install SGLang from pip:

pip install sglang

Start the SGLang server:

python3 -m sglang.launch_server
--model-path "HuggingFaceBio/Carbon-X"
--host 0.0.0.0
--port 30000

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "HuggingFaceBio/Carbon-X", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Use Docker images

docker run --gpus all
--shm-size 32g
-p 30000:30000
-v ~/.cache/huggingface:/root/.cache/huggingface
--env "HF_TOKEN=<secret>"
--ipc=host
lmsysorg/sglang:latest
python3 -m sglang.launch_server
--model-path "HuggingFaceBio/Carbon-X"
--host 0.0.0.0
--port 30000

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "HuggingFaceBio/Carbon-X", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Docker Model Runner How to use HuggingFaceBio/Carbon-X with Docker Model Runner:

docker model run hf.co/HuggingFaceBio/Carbon-X

Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Carbon-X Table of Contents

Model Summary Key features

How to use Serving with vLLM

Tokenizer: working with DNA inputs

Prompting tips

Evaluation

Training Model

Vocabulary extension

Mid-training

Supervised fine-tuning

Reinforcement learning

Software

Using Carbon-X as a tool for an agent Tips for agent use

Limitations

License

Acknowledgements

Carbon-X

An open 14B language model for DNA understanding and generation, from the Carbon family.

Table of Contents

Model Summary

How to use

Evaluation

Training

Using Carbon-X as a tool for an agent

Limitations

License

Acknowledgements

Model Summary

Each comparison model is shown at the reasoning effort with its highest average score. Costs use standard API pricing; for Carbon-X, measured throughput per H200-hour at $5/hour.

Carbon-X is a 14.8B-parameter decoder-only language model trained jointly on DNA sequences and natural-language text. You give it raw DNA inside an ordinary chat conversation and it answers in English: it predicts whether mutations are benign or pathogenic, identifies regions that control gene activity, classifies sequences by organism group, predicts the next bases of a sequence, and explains its reasoning. Responses can contain both DNA and natural language.

DNA and text share a single token stream in a single decoder. There is no separate DNA encoder. A hybrid tokenizer keeps a BPE vocabulary for text and encodes DNA as non-overlapping 6-mers, so one DNA token covers 6 base pairs.

Across five DNA benchmarks, which cover variant-effect prediction, sequence understanding and next-base prediction, Carbon-X averages 66.7% with thinking disabled. That is 27 points above Qwen3-14B, the model it starts from, and ahead of every other open model we evaluated. It is close to GPT-6.1 Sol (67.5%) at about 1/1,800 of the inference cost per problem, and it leads every model we tested on coding-variant effects and next-base prediction (Evaluation). Carbon-X answers directly, without extended reasoning, so each problem costs few output tokens. We estimate that analysing the 2.4 million variants of uncertain significance in ClinVar would cost about $300 with Carbon-X, compared with $32,000 to $4 million with the frontier models we evaluated. The weights fit on a single 80 GB GPU in bfloat16, so researchers can keep sensitive or unpublished sequences on their own infrastructure.

Key features

14.8B parameters: 40 layers, hidden size 5120, GQA (40 query heads, 8 KV heads), SwiGLU, RMSNorm with QK-norm.

Hybrid tokenizer: the Qwen3 BPE vocabulary for text plus 5,515 DNA tokens. These are 4,096 6-mers, 1,364 tail tokens for sequences whose length is not a multiple of 6, and the <dna>, </dna> and <oov> markers. DNA in a prompt is detected and tagged automatically.

Chat model: post-trained and evaluated with thinking disabled.

Context window: 40,960 tokens.

Training: 302B tokens of mid-training on DNA, biology text, English and instruction data, then supervised fine-tuning and reinforcement learning with verifiable rewards on DNA tasks.

Efficient inference: serves with vLLM on a single GPU.

How to use

Carbon-X uses a standard causal LM architecture. The DNA tokenizer is custom code, so pass trust_remote_code=True when you load the tokenizer. Carbon-X requires Transformers v5 or later.

pip install -U transformers

import torch from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "HuggingFaceBio/Carbon-X"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")

messages = [ { "role": "user", "content": ( "Reference: ATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG\n" "Variant: ATGGTGCATCTGACTCCTGTGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG\n\n" "Is this variant likely to be benign or pathogenic? Explain your reasoning." ), } ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True, ).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=1024) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Carbon-X was post-trained and evaluated with enable_thinking=False, and we recommend keeping it that way. generate() uses the sampling defaults from generation_config.json (temperature 0.6, top-p 0.95, top-k 20), which are also the settings we used for every reported result.

Serving with vLLM

pip install -U vllm vllm serve HuggingFaceBio/Carbon-X --trust-remote-code

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") response = client.chat.completions.create( model="HuggingFaceBio/Carbon-X", messages=[{"role": "user", "content": "What does this sequence encode?\n\nATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTT"}], extra_body={"chat_template_kwargs": {"enable_thinking": False}}, ) print(response.choices[0].message.content)

vLLM also applies the sampling defaults from generation_config.json unless you override them in the request.

Tokenizer: working with DNA inputs

The tokenizer switches between BPE for text and 6-mers for DNA. You can paste sequences directly into a message:

BioReason and GenomeQA come from the literature; we built FrontierVEP and NextBase.

Carbon-X also remains a capable assistant. On IFEval it reaches 79.3% prompt-level strict accuracy, or 84.0% averaged over the four IFEval metrics.

All training data was decontaminated against these benchmarks with three checks: whole-span DNA overlap, text n-gram overlap and sequence homology search.

Training

Model

Architecture: decoder-only Transformer with 40 layers, hidden size 5120, FFN size 17408, 40 attention heads with GQA (8 KV heads), head dimension 128, SwiGLU, RMSNorm with QK-norm and untied embeddings.

Positional embedding: RoPE with base θ = 1 × 10^6. The context window is 40,960 tokens.

Vocabulary: 157,184 tokens: 151,669 BPE tokens and 5,515 DNA tokens, including padding to a multiple of 128.

Precision: bfloat16.

Vocabulary extension

Carbon-X starts from Qwen3-14B. We add the DNA tokens to its vocabulary with randomly initialised embeddings. We then train for about 10B tokens with the transformer body frozen, so that only the new DNA embeddings learn, before unfreezing the whole model.

Mid-training

The full model is then trained for 302B tokens with a sequence length of 8,192 and a global batch size of 512 sequences (≈ 4.2M tokens per step, 72,000 steps). The learning rate follows a cosine schedule from 5e-5 to 5e-6 after 2,000 warmup steps. In our ablations, learning rate was the dominant knob at this stage. A low learning rate preserves English but leaves DNA at chance, and a high one learns DNA but forgets English. The data mixture is:

Supervised fine-tuning

The mid-trained model is fine-tuned for 3 epochs on a mixture of 20 dataset subsets. The learning rate follows a cosine schedule from 1e-5 to 1e-6, with a global batch size of 32, examples of up to 9,984 tokens and the loss computed on assistant tokens only. The mixture covers:

Reinforcement learning

The fine-tuned model is then trained with asynchronous GRPO and verifiable rewards in two rounds, with thinking disabled:

Both rounds use 16 samples per prompt, completions of up to 4,096 tokens, a learning rate of 1e-6, asymmetric clipping (ε = 0.2 / 0.28) and a maximum gradient norm of 0.2. Rewards are task-specific, code-checked verifiers. They cover variant-effect prediction (with a penalty for ungrounded rationales), sequence-feature classification, basic DNA operations, DNA skills, locating positions within sequences, biology multiple-choice questions and instruction following.

Software

Post-training: TRL for supervised fine-tuning and asynchronous GRPO, with vLLM for generation.

Using Carbon-X as a tool for an agent

A general-purpose assistant can call Carbon-X when a question involves DNA. Serve Carbon-X with vLLM (Serving with vLLM) and wrap it in a function that takes a question and named DNA sequences:

from openai import OpenAI

carbon = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

def ask_carbon(question: str, sequences: dict[str, str], context: str = "") -> str: """Ask Carbon-X a question about one or more named DNA sequences.""" parts = [context, question] if context else [question] parts += [f"{name}: <dna>{seq}</dna>" for name, seq in sequences.items()] response = carbon.chat.co