Bend: Automatic Parallelism via Interaction Combinators

In mid-2024, a GitHub repository called Bend briefly went viral with a claim that sounded impossible: write Python-like sequential code, run it in parallel across every CPU core and GPU thread automatically, with no explicit threads, no locks, no synchronization primitives. The repo got tens of thousands of stars in days.

The underlying idea is real and theoretically deep. The current implementation is research-grade. Both things are true simultaneously — and understanding why requires understanding interaction combinators, one of the most unusual computational models you’ll encounter.

What Bend Is

Bend is a high-level programming language that compiles to HVM2 (Higher-order Virtual Machine 2), a runtime built by HigherOrderCO (founded by Victor Taelin). The pitch:

• Write code in Python-like syntax
• The runtime automatically finds and exploits parallelism
• Run on CPU (multi-core) or GPU (CUDA) without changing your code
• No data races by construction — the computational model makes them impossible

Bend and HVM2 are open source (MIT licensed), actively developed, and genuinely novel in their approach. They are also pre-1.0, with a small ecosystem, limited I/O, and real performance tradeoffs that the viral marketing glossed over.

If you want to understand the future of parallel computing, or you’re curious about the intersection of programming language theory and hardware, Bend is worth understanding. If you need to parallelize a production workload today, use Rust’s rayon or CUDA directly.

HVM2: Interaction Combinators as a Runtime

Most runtimes are built on lambda calculus or a stack machine. HVM2 is built on interaction combinators — a different computational foundation invented by Yves Lafont in 1990.

What interaction combinators are

An interaction combinator system consists of:

1. A set of cells (nodes), each with one principal port and zero or more auxiliary ports
2. Wires connecting ports
3. Interaction rules that fire when two cells connect principal-to-principal

Lafont showed that with just three cell types — constructor (con), duplicator (dup), and eraser (era) — you can compute anything a Turing machine can compute.

The key insight for parallelism: interaction rules are local. When two cells connect principal-to-principal, they reduce according to their rule, touching nothing else in the graph. Two independent reductions in different parts of the graph can happen simultaneously with no coordination.

This isn’t just “oh we can parallelize some loops.” It’s a computational model where parallelism falls out of the structure of computation itself.

The six interaction rules

// Constructor-Constructor annihilation
(a b) ⋈ (c d)  →  a-c  b-d   (cross-connect the aux ports)

// Duplicator-Duplicator annihilation
{a b} ⋈ {c d}  →  a-c  b-d

// Eraser-Eraser annihilation
* ⋈ *          →  (nothing, both removed)

// Constructor-Duplicator: the expensive one
(a b) ⋈ {c d}  →  creates new cells, 4 wires

// Constructor-Eraser
(a b) ⋈ *      →  a and b both connect to new erasers

// Duplicator-Eraser
{a b} ⋈ *      →  a and b both connect to new erasers

The constructor-duplicator interaction is where sharing gets eliminated. In a conventional lambda calculus interpreter, sharing an expression means multiple parts of the program point to the same memory — so you need reference counting or garbage collection. In HVM2, when a duplicator interacts with a constructor, it creates independent copies. No sharing, no data races, no GC pauses (HVM2 uses no garbage collector).

Why this matters for parallelism

In a conventional program:

result = f(x) + g(x)

If f and g are pure functions, they could run in parallel — but figuring that out requires alias analysis, dependency tracking, and careful compiler work.

In HVM2, f(x) and g(x) reduce to independent subgraphs. The runtime discovers independent redexes at runtime and distributes them across threads. The parallelism is structural, not inferred.

The Bend Language

Bend provides two syntax styles:

• Imp (imperative): Python-like, the default for new users
• Fun (functional): Haskell-like, more direct mapping to interaction nets

Hello World

1
2

def main():
  return "Hello, Bend!"

Run it:

1
2
3
4

bend run main.bend          # interpreter (slow, for debugging)
bend run -s main.bend       # compiled, sequential
bend run -p main.bend       # compiled, parallel (all CPU cores)
bend run --cuda main.bend   # compiled, GPU (requires CUDA)

Types and pattern matching

Bend uses algebraic data types:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

type Shape:
  Circle { radius }
  Rectangle { width, height }
  Triangle { base, height }

def area(shape):
  match shape:
    case Shape/Circle:
      return 3.14159 * shape.radius * shape.radius
    case Shape/Rectangle:
      return shape.width * shape.height
    case Shape/Triangle:
      return 0.5 * shape.base * shape.height

def main():
  shapes = [
    Shape/Circle { radius: 5.0 },
    Shape/Rectangle { width: 3.0, height: 4.0 },
    Shape/Triangle { base: 6.0, height: 8.0 },
  ]
  return map(shapes, area)

Lists and recursion

Bend’s built-in List type is a recursive algebraic type:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

def sum(list):
  match list:
    case List/Nil:
      return 0
    case List/Cons:
      return list.head + sum(list.tail)

def map(list, f):
  match list:
    case List/Nil:
      return List/Nil
    case List/Cons:
      return List/Cons { head: f(list.head), tail: map(list.tail, f) }

def filter(list, pred):
  match list:
    case List/Nil:
      return List/Nil
    case List/Cons:
      if pred(list.head):
        return List/Cons { head: list.head, tail: filter(list.tail, pred) }
      else:
        return filter(list.tail, pred)

The bend and fold Constructs

This is where Bend diverges most from Python. Bend has no for or while loops in the traditional sense. Instead, it has bend and fold.

bend: building a tree structure

bend creates a recursive tree structure from a seed value:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# Generate a binary tree of depth n
def make_tree(depth):
  bend d = depth:
    when d > 0:
      # Branch: create two subtrees
      return Tree/Node {
        left: fork(d - 1),
        right: fork(d - 1),
      }
    else:
      # Leaf
      return Tree/Leaf { value: 0 }

The fork keyword refers back to the bend recursion. Each call to fork is an independent branch of the tree — and independent branches can be computed in parallel.

This is the key: bend doesn’t loop sequentially, it builds a tree of work. Trees of work parallelize naturally — you can process left and right subtrees simultaneously.

fold: consuming a tree structure

1
2
3
4
5
6

def sum_tree(tree):
  fold tree:
    case Tree/Leaf:
      return tree.value
    case Tree/Node:
      return tree.left + tree.right  # left and right are already folded

The fold construct recursively reduces a tree. Again: left and right subtrees are independent, so they can be processed in parallel.

The loop equivalent

Here’s how you’d express “add 1 to every element of a range” using bend:

1
2
3
4
5
6
7
8
9

def range_map(n, f):
  bend i = 0, acc = List/Nil:
    when i < n:
      return fork(i + 1, List/Cons { head: f(i), tail: acc })
    else:
      return acc

def main():
  return range_map(1000000, lambda x: x * 2)

Compare to how you’d write this imperatively in Python:

1

result = [x * 2 for x in range(1000000)]

The Bend version is more verbose, but the bend structure exposes the tree shape of the computation that the runtime can parallelize.

What Actually Parallelizes

Not everything benefits from Bend’s parallel execution. Here’s a realistic breakdown:

Parallelizes well

Tree-recursive algorithms: Merge sort, quicksort (on Bend’s tree-based lists), tree traversals, divide-and-conquer algorithms. These naturally express as trees of independent work.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

def merge_sort(list):
  match list:
    case List/Nil:
      return List/Nil
    case List/Cons:
      match list.tail:
        case List/Nil:
          return list
        case List/Cons:
          # Split and sort both halves independently (parallelizes!)
          (left, right) = split(list)
          return merge(merge_sort(left), merge_sort(right))

Map over large collections: Applying an expensive pure function to every element of a list.

Functional graph transformations: Any computation that can be expressed as rewriting an interaction net directly.

Parallelizes poorly

Sequential data dependencies: Computing a running sum where each element depends on the previous:

1
2
3
4
5
6
7

# This is inherently sequential — Bend won't help
def running_sum(list):
  match list:
    case List/Nil: return List/Nil
    case List/Cons:
      rest = running_sum(list.tail)  # must complete first
      return List/Cons { head: list.head + ..., tail: rest }

I/O-bound code: File reads, network requests, database queries. HVM2 has minimal I/O support currently.

Tight numeric loops with state: Matrix multiplication with the typical triple-nested loop. Bend’s list-based data structures add overhead that outweighs parallel gains for cache-friendly numeric code. For this use case, CUDA or AVX intrinsics are still better.

Performance Reality

The viral benchmarks showed Bend achieving near-linear scaling across CPU cores on tree-recursive workloads. That part is real.

What the benchmarks didn’t highlight:

Constant factor overhead: HVM2’s interaction combinator reduction has overhead compared to native code execution. A simple recursive Fibonacci in Bend is slower than the same function in C, even on one core.

Memory layout: HVM2 allocates interaction combinator nodes on a heap. This is flexible but cache-unfriendly for sequential access patterns. Tight numeric loops that fit in L1/L2 cache in C don’t fit in HVM2’s node graph.

The parallelism sweet spot: You need enough independent work to amortize the overhead of thread scheduling. For small inputs, the sequential interpreter is faster.

Realistic speedup: On an 8-core machine with a highly parallel-friendly workload (large tree recursion), 4-6x speedup is achievable. Linear scaling to 8x requires very little inter-thread coordination. For mixed workloads, 2-3x is more typical.

GPU Execution

1

bend run --cuda main.bend

When compiled for CUDA, HVM2 distributes interaction net reductions across GPU threads. Each thread takes a pending redex from a work queue and executes its reduction rule.

This is conceptually elegant. In practice:

• Requires CUDA toolkit and an NVIDIA GPU
• GPU overhead (kernel launch, data transfer) is only worth it for very large computations
• The CUDA backend is less mature than the CPU backend
• AMD/Metal/Vulkan are not supported

For GPU-parallel numeric work (deep learning, physics simulation), the traditional CUDA/cuBLAS approach with explicit kernels still has better performance. Bend’s GPU path is most interesting for algorithms that don’t fit CUDA’s SIMT model well — highly irregular tree computations, for example.

Current Limitations

Be clear-eyed about what Bend can’t do today:

No standard I/O ecosystem: File I/O is basic, network I/O is experimental. You can’t build a web server in Bend yet.

No package manager or ecosystem: There are no Bend libraries to install. You write everything from scratch or fork from examples in the repo.

Language instability: Bend is pre-1.0. Syntax and semantics may change between versions. Don’t build production systems on it.

Numeric performance: For float-heavy computation (ML, simulations), Bend’s interaction combinator representation is not competitive with C or CUDA using traditional data layouts.

Debugging: Error messages are improving but still terse. The interaction combinator model makes it hard to reason about performance in the traditional way.

Memory usage: Interaction combinator nodes use more memory than equivalent C structs, especially for numeric arrays.

Installing and Running Bend

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

# Install Rust first (Bend is implemented in Rust)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install Bend
cargo install bend-lang

# Verify
bend --version

# Run a file
bend run example.bend

# Run in parallel mode (all CPU cores)
bend run -p example.bend

# Run with CUDA (requires CUDA toolkit)
bend run --cuda example.bend

A complete working example: parallel sum of squares

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

# sum_squares.bend
# Compute sum of squares from 1 to N in parallel

def square(x):
  return x * x

# Build a list from 1 to n
def range(n):
  bend i = 1, acc = List/Nil:
    when i <= n:
      return fork(i + 1, List/Cons { head: i, tail: acc })
    else:
      return acc

# Sum a list
def sum(list):
  match list:
    case List/Nil:
      return 0
    case List/Cons:
      return list.head + sum(list.tail)

def main():
  n = 1000000
  nums = range(n)
  squares = map(nums, square)
  return sum(squares)

1
2
3
4
5

# Sequential
bend run -s sum_squares.bend

# Parallel (uses all cores)
bend run -p sum_squares.bend

Parallel merge sort

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

# mergesort.bend

def split(list):
  match list:
    case List/Nil:
      return (List/Nil, List/Nil)
    case List/Cons:
      match list.tail:
        case List/Nil:
          return (list, List/Nil)
        case List/Cons:
          (left, right) = split(list.tail.tail)
          return (
            List/Cons { head: list.head, tail: left },
            List/Cons { head: list.tail.head, tail: right }
          )

def merge(a, b):
  match a:
    case List/Nil:
      return b
    case List/Cons:
      match b:
        case List/Nil:
          return a
        case List/Cons:
          if a.head <= b.head:
            return List/Cons { head: a.head, tail: merge(a.tail, b) }
          else:
            return List/Cons { head: b.head, tail: merge(a, b.tail) }

def sort(list):
  match list:
    case List/Nil:
      return List/Nil
    case List/Cons:
      match list.tail:
        case List/Nil:
          return list
        case List/Cons:
          (left, right) = split(list)
          # These two recursive calls are independent — HVM2 runs them in parallel
          return merge(sort(left), sort(right))

def main():
  unsorted = [5, 2, 8, 1, 9, 3, 7, 4, 6, 0]
  return sort(unsorted)

The Functional Syntax (Fun)

For users coming from Haskell or ML, Bend also supports a more direct functional syntax:

# Fun syntax (Haskell-like)
(Double n) = (* n 2)

(Factorial 0) = 1
(Factorial n) = (* n (Factorial (- n 1)))

(Main) = (Factorial 10)

The Fun syntax maps more directly to interaction nets and is useful for understanding what’s actually happening at the HVM2 level.

Theoretical Significance

Bend is a practical experiment in a deep theoretical question: what would programming look like if we built it on optimal lambda reduction instead of Turing machines?

The key papers behind HVM2:

• Lafont (1990): “Interaction Nets” — the original interaction combinator paper
• Lamping (1990): “An Algorithm for Optimal Lambda Calculus Reduction” — optimal (beta-normal-form-in-polynomial-time) reduction
• Asperti & Guerrini (1998): “The Optimal Implementation of Functional Programming Languages” — the book-length treatment

Why does optimal reduction matter? In a conventional interpreter, reducing (λx. x + x) (expensive_computation) evaluates expensive_computation twice (or requires sharing via thunks). Optimal reduction evaluates it once, automatically, at the cost of duplication nodes in the interaction net. HVM2 makes this practical.

The interaction combinator model also has implications for hardware. If you built a chip where each processing element handles one interaction rule, you’d have a natural parallel machine with no von Neumann bottleneck. Several research groups have explored “interaction combinator chips.” Bend is the first practical language targeting this model.

Comparison to Other Parallel Approaches

Bend’s niche is: automatically parallel without any user annotation, for algorithms that express naturally as trees. No other system in this table achieves that for general-purpose code.

Where Bend Makes Sense Today

Use Bend for:

• Learning about interaction combinators and optimal reduction — Bend is the most accessible entry point to this theory
• Experimenting with parallel algorithms expressed as tree recursion
• Research into alternative computational models
• Writing highly parallel functional algorithms where you don’t want to think about thread management
• Contributing to an interesting open-source project at an early stage

Don’t use Bend for:

• Production services (no ecosystem, breaking changes, limited I/O)
• Numeric/ML computation (worse cache behavior than C/CUDA for arrays)
• I/O-heavy workloads (minimal I/O support)
• Anywhere you need to hire engineers who can maintain the code

Getting Involved

The Bend and HVM2 repositories are at github.com/HigherOrderCO. The community is active on Discord and GitHub Issues. At this stage of the project, filing bug reports and contributing examples is genuinely valuable.

The theoretical foundations are worth studying regardless of whether Bend itself becomes mainstream. Interaction combinators, optimal reduction, and linear logic are deep ideas that have influenced programming language research for three decades — Bend is the first real attempt to bring them to a general-purpose programmer’s workflow.

If the interaction combinator model turns out to be efficient enough on modern hardware (and the HVM2 team is actively working on this), Bend could represent a genuinely different way of writing parallel code: not “figure out which parts can run in parallel and annotate them,” but “write the algorithm naturally and let the structure of computation determine the parallelism.” That’s a worthwhile goal even if the current implementation has rough edges.

The Bottom Line

Bend is a research project with production ambitions, backed by real theoretical depth. The viral moment was ahead of the current implementation — but the ideas are solid. Interaction combinators are a legitimate computational model, HVM2’s automatic parallelism is real (not just marketing), and the tree-recursive performance is genuinely impressive for the right workloads.

For practical parallel programming today: use rayon, CUDA, or Go. For understanding a genuinely different approach to computation: install Bend and sort a list with it. The experience of writing code that runs in parallel without a single thread-management primitive is worth having, even in an alpha-quality tool.