The Leiden algorithm is the modern replacement for Louvain for community detection in graphs. It guarantees well-connected communities (Louvain doesn't) and runs faster. I needed it for the GraphRAG feature in Arcana, but there were no Elixir bindings. So I built Leidenfold.
What it does
Given a graph (nodes and edges), Leidenfold finds communities: groups of nodes that are more densely connected to each other than to the rest of the network. Think clustering, but for graph structures.
edges = [{0, 1}, {1, 2}, {2, 0}, {3, 4}, {4, 5}, {5, 3}, {2, 3}]
{:ok, result} = Leidenfold.detect_from_edges(edges)
result.n_communities # => 2
result.membership # => [0, 0, 0, 1, 1, 1]
result.quality # => optimization score
Two clear clusters connected by a single bridge edge. The algorithm finds them automatically.
Under the hood
Leidenfold is a NIF wrapping libleidenalg (the reference C++ implementation) via igraph. Both are statically linked in the precompiled binaries, so installation is just mix deps.get on supported platforms (macOS ARM64, Linux x86_64/ARM64).
For other platforms, set LEIDENFOLD_BUILD=true and it compiles from source against your local igraph and libleidenalg installations.
Quality functions
The algorithm supports six optimization objectives:
- CPM (default): Constant Potts Model. Resolution parameter controls community granularity
- Modularity: the classic Newman-Girvan metric
- Significance and Surprise: information-theoretic approaches
- RBER and RBC: Reichardt-Bornholdt variants with different null models
{:ok, result} = Leidenfold.detect(sources, targets,
objective: :modularity,
resolution: 1.0,
iterations: 2,
seed: 42
)
How Arcana uses it
This is the piece that makes Arcana's GraphRAG work. When you ingest documents with graph: true, Arcana extracts entities (people, organizations, technologies) and their relationships into a knowledge graph. But a flat graph of entities isn't enough for good retrieval. You need structure.
Leidenfold clusters the entity graph into communities: groups of closely related entities. Each community gets a summary. At search time, Arcana can traverse not just individual entity connections but entire communities of related concepts, then fuse those results with vector search using Reciprocal Rank Fusion.
So when you ask "Who leads OpenAI's safety research?", the graph traversal finds not just entities named in the query but the entire community of people, roles, and organizations clustered around OpenAI's safety team. That context gets combined with the semantic search results for a much richer answer.
Community detection is what turns GraphRAG from entity linking into actual knowledge structure.
Beyond RAG
Leidenfold is a general-purpose library. Social network analysis, citation networks, biological networks, recommendation systems: anywhere you have a graph and want to find structure.
{:leidenfold, "~> 0.2.0"}
