MetaGraph: Metagenome Graph Project

Scalable indexing and querying of annotated genome graphs, from a handful of genomes to petabase-scale sequence repositories.

Index your reads or assemblies once. Then search them for matching samples, alignments and source positions.

For a sense of scale, MetaGraph Online can search an index representing 7.2 trillion bases from about 4,200 samples for matches to a 10 kbp DNA sequence in under a second.

A MetaGraph index has two parts: a de Bruijn graph storing the indexed k-mers and an annotation matrix linking them to labels, such as samples or sequence headers. Optional payloads record counts or source positions.

flowchart LR
    F["FASTA / FASTQ<br/>reads, assemblies, transcripts"]
    F -->|build| G[("de Bruijn graph<br/>graph.dbg")]
    F -->|annotate| M[("annotation matrix<br/>*.annodbg")]
    G --> Q(("query / align"))
    M --> Q
    Qseq["query sequence"] --> Q
    Q --> R["matches<br/>labels, counts, positions"]

Loading

Features

• Search public archives at metagraph.ethz.ch. See MetaGraph Online.
• Index your own data. Build a k-mer index over reads, assemblies or transcripts. Then query it for matching labels.
• Optional per-k-mer payloads. Attach counts (abundance, e.g. expression or coverage) or coordinates (positions that losslessly encode source sequences and return per-target hit positions).
• Sequence alignment. Align sequences to the full annotated graph, with sub-k seeding for arbitrarily short queries.
• Scalable graph cleaning. Strip sequencing errors out of very large de Bruijn graphs.
• Differential assembly. Extract sequences present in one group of samples and absent from another.
• Python API & HTTP server. Drive MetaGraph from Python or query a running instance over HTTP.

Under the hood

• Succinct data structures: the default succinct (BOSS) graph representation uses only 2–4 bits per k-mer.
• Modular annotation formats: ColumnCompressed, RowDiff<Multi-BRWT>, RowSparse, Rainbowfish, plus count- and coordinate-aware variants. Pick the compression/speed tradeoff that fits your scale.
• Custom alphabets: {A,C,G,T}, {A,C,G,T,N}, amino acids, case-sensitive DNA, or compile-time custom alphabets.
• Extensible by design: generic interfaces let developers add new graph/annotation representations or algorithms with little code.
• Memory-mapped loading: use --mmap to open supported graphs and annotations without first reading them fully into RAM. This enables fast startup and queries on indexes larger than available memory. Use fast local SSD storage.
• Scales to trillions of k-mers and millions of labels: see the petabase-scale indexing study.
• Batched operations: annotation queries group lookups to reduce repeated access to compressed data structures.

Full documentation: https://metagraph.ethz.ch/static/docs/index.html (offline copy in metagraph/docs/source).

MetaGraph Online

Try MetaGraph without installing anything at https://metagraph.ethz.ch/search. It searches indexes representing over 50 petabases of public sequencing data, including SRA collections, RefSeq, UHGG, Tara Oceans and UniParc. See the list of databases. Paste a sequence and pick the indexes to search.

Prefer local analysis? The prebuilt indexes are also published on AWS Open Data for download (no AWS account needed) and offline querying. See Preconstructed indexes.

Quick start

1. Install

With Conda and Git installed, install the CLI and the current workflow wrapper:

conda create -n metagraph --override-channels -c conda-forge -c bioconda \
    --strict-channel-priority python=3.12 pip "metagraph>=0.5.2"
conda activate metagraph

git clone --depth 1 --branch master https://github.com/ratschlab/metagraph.git
cd metagraph
python -m pip install ./metagraph/workflows

The shallow checkout supplies both the wrapper and the example data. No submodule initialization or C++ build is needed. Both metagraph and metagraph-workflows are available while the environment is active.

2. Build an index

metagraph-workflows build <(ls metagraph/tests/data/*.fa) -o out/ --primary -p 2 --mem-gb 1

--primary stores one orientation per reverse-complement pair, approximately halving the graph compared with indexing both strands. Use it when the strand of the reads is unknown, as in most short-read sequencing.

The pipeline builds the graph, annotates it, applies row-diff compression, then clusters and relaxes the BRWT. It writes graph.dbg, a smaller graph representation with slower access (graph_small.dbg), and the RowDiff<Multi-BRWT> annotation graph.relax.row_diff_brwt.annodbg. The Quick start guide explains each step.

Real-workload example with file list and hardware budget

metagraph-workflows build samples.txt -o out/ --primary \
  -p 34 --mem-gb 70 --disk-swap-dir /scratch/swap

The positional argument accepts a file list (one path per line), a directory, or process substitution. Per-stage memory caps, thread packing, and BRWT parameters are derived from the budget. See metagraph-workflows build --help for all options.

3. Query

Query the index with a FASTA file containing sequences at least as long as the graph's k-mer length. For a first try, use the bundled test data:

metagraph query --query-mode matches -p 2 \
    -i out/graph.dbg -a out/graph.relax.row_diff_brwt.annodbg \
    metagraph/tests/data/transcripts_100.fa

Other ways to use the index:

• metagraph align: align sequences to the graph and report the alignment itself. With a coordinate-aware annotation it acts as a read mapper, returning source positions.
• metagraph query --align: align each query to the graph first, then report the labels of the best-scoring alignment instead of requiring exact k-mer matches. Useful for divergent or noisy queries.
• metagraph server_query: start an HTTP server that answers queries over a REST API (also drivable from the Python client).

The Minimal example below walks through each step on a smaller dataset.

Minimal example

Step-by-step walkthrough with the metagraph CLI (build → annotate → query → stats)

A hands-on demo using metagraph directly (no workflow wrapper). A label is whatever tag you want each k-mer associated with: a filename, a FASTA header, or a custom string. --anno-header below produces one label per FASTA record.

cd metagraph/tests/data

# 1. Build a de Bruijn graph (k-mer index) with k=31
metagraph build -v -p 4 -k 31 -o samples metasub_fake_data_simple.fa

# 2. Construct an annotation matrix linking each k-mer to its source label.
metagraph annotate -v -p 4 -i samples.dbg --anno-header \
                   -o samples metasub_fake_data_simple.fa

# 3. Query: for each query sequence, report all labels whose k-mers cover ≥80% of it.
metagraph query --query-mode matches \
                -i samples.dbg -a samples.column.annodbg \
                --min-kmers-fraction-label 0.8 \
                metasub_fake_data_simple.fa

# 4. Print graph + annotation stats.
metagraph stats -a samples.column.annodbg samples.dbg

Outputs samples.dbg (the de Bruijn graph) and samples.column.annodbg (3 labels, one per FASTA record). Sample query output (tab-separated: query index, query header, then one <label>:k-mer-count entry per matching label):

0   kl_sample   <kl_sample>:330
1   zh_sample   <kl_sample>:243   <zh_sample>:243
2   tk_sample   <kl_sample>:207   <tk_sample>:207

Each query matches at least its own label. zh_sample has 243 k-mers, and all of them are in kl_sample. The same holds for tk_sample (207 of 207). Both clear the 80% threshold. kl_sample matches only itself, because neither shorter sample covers its 330 k-mers.

More recipes

Direct CLI: align / assemble / differential assembly / stats

# Build a de Bruijn graph from a fasta (single sample)
metagraph build -v -p 8 -k 31 --mem-cap-gb 10 -o graph data.fa.gz

# Build using disk swap (limits RAM at the cost of disk I/O)
metagraph build -v -p 8 -k 31 --mem-cap-gb 10 --disk-swap /scratch/swap -o graph data.fa.gz

# Annotate a graph with file-based labels (one label per input file)
metagraph annotate -v -p 8 --anno-filename -i graph.dbg -o annotation data.fa.gz

# Align sequences to the graph (plain sequence-to-graph alignment, no labels).
metagraph align -v -i graph.dbg query.fa

# Labeled alignment: with -a, the walk continues only while its k-mers share
# at least 1 label. It reports those shared labels.
metagraph align -v -i graph.dbg -a annotation.row_diff_brwt.annodbg reads.fq

# Same, but with a coordinate-aware annotator: the walk is additionally
# coordinate-consistent (positions step by ±1 per node), so hits map to
# source positions. Functions as a read mapper over indexed genomes.
metagraph align -v -i graph.dbg -a annotation.row_diff_brwt_coord.annodbg reads.fq

# query --align: aligns each query to the graph WITHOUT label constraints,
# then fetches labels for the highest-scoring walk. Use when exact k-mer
# matching is too strict (divergent or noisy queries).
metagraph query --align -i graph.dbg -a annotation.row_diff_brwt.annodbg query.fa

# Assemble unitigs from a graph (writes assembled.fasta.gz)
metagraph assemble -v graph.dbg -o assembled --unitigs

# Differential assembly: JSON rules define which label groups must be present vs. absent.
# Sample rule files: metagraph/tests/data/example.diff.json, example_simple.diff.json.
metagraph assemble -v graph.dbg --unitigs \
    -a annotation.column.annodbg \
    --diff-assembly-rules diff_assembly_rules.json \
    -o diff_assembled

# Stats: graph only, annotation only, or both
metagraph stats graph.dbg
metagraph stats -a annotation.column.annodbg
metagraph stats -a annotation.column.annodbg graph.dbg

See the online docs for the full subcommand reference, alphabet builds, and the row-diff / BRWT clustering pipeline.

Build from KMC k-mer counters (e.g., to filter low-abundance k-mers)

KMC is a fast k-mer counter that can pre-process inputs and filter by abundance.

Configure with cmake .. -DBUILD_KMC=ON to build the standalone KMC executable used below. Reading existing KMC files is always supported.

Build a graph from k-mers occurring at least 5 times:

K=31
# Count k-mers with KMC (drops k-mers with abundance < 5)
./KMC/kmc -ci5 -t4 -k$K -m5 -fq input.fastq.gz input.cutoff_5 ./KMC

# Build the graph directly from the KMC counter
metagraph build -v -p 4 -k $K -o graph input.cutoff_5.kmc_pre

Add --count-kmers to keep the KMC abundance counts as a weight vector alongside the graph (graph.dbg.weights). The weighted graph is the input for metagraph clean and for indexing k-mer counts.

Common query flags

# Filter labels with low k-mer coverage (default 0.7)
metagraph query --min-kmers-fraction-label 0.8 ...

# Delimiter joining labels in `--query-mode labels` output (default ":")
metagraph query --query-mode labels --labels-delimiter ", " ...

# Per-k-mer presence/absence bitmask per matching label
metagraph query --query-mode signature ...

# Indexed k-mer counts (requires count-aware annotation, see online docs)
metagraph query --query-mode counts ...

# k-mer coordinates / per-target positions (requires coordinate-aware annotation)
metagraph query --query-mode coords ...

# Server mode (Python API / HTTP queries)
metagraph server_query -i graph.dbg -a annotation.row_diff_brwt.annodbg --port 5555 --parallel 8

More install options

MetaGraph runs on Linux and macOS. Bioconda ships the DNA and Protein alphabets. metagraph is a link to metagraph_DNA. The Docker image adds DNA5. For other alphabets, build from source.

Docker

docker pull ghcr.io/ratschlab/metagraph:master
docker run -v ${HOME}:/mnt ghcr.io/ratschlab/metagraph:master \
    metagraph build -v -k 10 -o /mnt/transcripts_1000 /mnt/transcripts_1000.fa

Replace ${HOME} with the host directory you want to expose under /mnt. The default image uses the DNA alphabet {A,C,G,T}. The same image runs metagraph_DNA5 and metagraph_Protein. See the published container images for other versions.

Advanced Docker usage

Inspect what's mounted in the container:

docker run -v ${HOME}:/mnt ghcr.io/ratschlab/metagraph:master ls /mnt

Run the DNA5 / Protein variants:

docker run -v ${HOME}:/mnt ghcr.io/ratschlab/metagraph:master \
    metagraph_Protein build -v -k 10 -o /mnt/graph /mnt/protein.fa

Drop into an interactive shell for multi-step workflows:

docker run -it --entrypoint /bin/bash -v ${HOME}:/mnt ghcr.io/ratschlab/metagraph:master
# inside container:
metagraph --version
metagraph build -v -k 31 -o /mnt/graph /mnt/data.fa.gz

Install from source

See the installation guide for custom alphabet / debug builds.

Contributing

Developer notes: docker build, releases

Build a docker container. Run docker build .

Releases.

1. Bump the version in package.json.
2. Tag the commit with that version.
3. Create a GitHub release.

Citation

If MetaGraph or its index resources are useful in your work, please cite:

Karasikov M, Mustafa H, Danciu D, Kulkov O, Zimmermann M, Barber C, Rätsch G, Kahles A. Efficient and accurate search in petabase-scale sequence repositories. Nature. 2025;647: 1036–1044. https://www.nature.com/articles/s41586-025-09603-w

BibTeX

@article{karasikov2025metagraph,
  title={Efficient and accurate search in petabase-scale sequence repositories},
  author={Karasikov, Mikhail and Mustafa, Harun and Danciu, Daniel and Kulkov, Oleksandr and Zimmermann, Marc and Barber, Christopher and R{\"a}tsch, Gunnar and Kahles, Andr{\'e}},
  journal={Nature},
  volume={647},
  number={8091},
  pages={1036--1044},
  year={2025},
  publisher={Nature Publishing Group},
  doi={10.1038/s41586-025-09603-w}
}

License

MetaGraph is distributed under the GPLv3 License (see LICENSE). See also AUTHORS and COPYRIGHT.