Threadle: A Memory-Efficient Network Storage and Query Engine for Large, Multilayer, and Mixed-mode Networks

We present Threadle, an open-source, high-performance, and memory-efficient network storage and query engine written in C#. Designed for working with full-population networks derived from administrative register data, which represent very large, multilayer, mixed-mode networks with millions of nodes and billions of edges, Threadle addresses a fundamental limitation of existing network libraries: the inability to efficiently handle two-mode (bipartite) data at scale. Threadle's core innovation is a pseudo-projection approach that allows two-mode layers to be queried as if they were projected into one-mode form, without ever materializing the memory-prohibitive projection. We demonstrate that a network with 20 million nodes containing layers equivalent to 8 trillion projected edges can be stored in approximately 20 GB of RAM -- a compression ratio exceeding 2000:1 compared to materialized projection. Additionally, Threadle provides native support for multilayer mixed-mode networks, an integrated node attribute manager, and a CLI frontend with 50+ commands for the construction, processing, file handling, and management of very large heterogeneous networks. Threadle is freely available at https://www.threadle.dev and can either be obtained as precompiled binaries for Win, macOS and Linux, or compiled directly from source. Supplementing Threadle is threadleR, an R frontend that enables advanced sampling- and traversal-based analyses on very large, heterogeneous, multilayer, mixed-mode population-scale networks.

• Publication year

  2026

• Venue

  Unknown venue

• Publication date

  2026-02-24

• Fields of study

  Computer Science

• Identifiers
  arXiv 2603.04446
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• The graph-tool python library

  2014cited by this paper
• NetworKit: A tool suite for large-scale complex network analysis

  2014cited by this paper
• Exploring Network Structure, Dynamics, and Function using NetworkX

  2008cited by this paper
• The igraph software package for complex network research

  2006cited by this paper
• Random graphs

  2006cited by this paper
• Efficient generation of large random networks.

  2005cited by this paper
• The collegial phenomenon. The social mechanisms of cooperation among peers in a corporate law partnership

  2003cited by this paper
• Emergence of Scaling in Random Networks

  1999cited by this paper
• Collective dynamics of ‘small-world’ networks

  1998cited by this paper

• No citing papers are available for this paper.