LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

Elixir and the BEAM: Concurrency That Actually Scales

elixirbeamotpphoenixliveviewconcurrencydistributed-systemsfunctional-programming
Contents

There is a class of problem that breaks most mainstream runtimes: systems that need hundreds of thousands of simultaneous long-lived connections, that must remain available under partial failure, that need to route messages between processes running on different machines, and that need to do all of this while keeping latency predictable and low. Node.js handles concurrency with a single-threaded event loop that collapses under CPU pressure. Go’s goroutines are excellent but its runtime was built for a different set of tradeoffs. Python’s GIL has been relaxed in 3.13 but it is still not a concurrency-first runtime. Ruby is beautiful but slow for concurrent work.

The BEAM — the Erlang virtual machine — was built specifically for this class of problem in 1986, and it still has properties that no other mainstream runtime has replicated. Elixir is a language that runs on the BEAM while making it dramatically more approachable. This post covers both, deeply, with code you can actually run.


1. The BEAM: What It Is and Why It Matters

History

In 1986, Ericsson needed software to control telephone exchanges — systems that needed to handle hundreds of thousands of concurrent calls, upgrade software without dropping calls, and keep running even as individual hardware nodes failed. The constraints were extreme: a telephone exchange could not go down. Lives could depend on it.

Joe Armstrong, Robert Virding, and Mike Williams built Erlang, and with it the BEAM virtual machine, to meet those constraints. The language went open source in 1998. The AXD301 ATM switch, built with Erlang, famously achieved nine nines of availability — 99.9999999% uptime, which equates to roughly 31 milliseconds of downtime per year.

WhatsApp ran 2 million concurrent connections per server on Erlang. Discord uses Elixir to serve tens of millions of concurrent users. Bet365 processes massive sports betting traffic on the BEAM. RabbitMQ is written in Erlang. The pattern is consistent: when you need to handle massive concurrency with strong fault isolation and predictable latency, the BEAM is the right tool.

The BEAM’s Unique Properties

Preemptive scheduler. The BEAM does not rely on processes yielding control voluntarily. It uses reduction counting — each process is given a budget of “reductions” (roughly, function calls) and the scheduler preempts processes that exhaust their budget. This means a tight loop or a CPU-intensive computation cannot starve other processes. This is fundamentally different from cooperative multitasking, which is what Node.js and Python asyncio use. In Node.js, a CPU-bound callback blocks the entire event loop. In the BEAM, it just slows down one process.

Millions of lightweight processes. A BEAM process is not an OS thread or even a goroutine in the Go sense. BEAM processes start with around 2–4 KB of stack space and are multiplexed across OS threads (typically one per CPU core). It is normal and idiomatic to have hundreds of thousands of BEAM processes running simultaneously. Each process is isolated — it has its own heap, its own mailbox, and its own execution state.

Per-process heap and garbage collection. This is the BEAM’s most unusual property, and the source of its latency predictability. Each process has its own heap and is garbage collected independently. When a process dies, its memory is immediately reclaimed — no global GC scan needed. When a process does need to be GC’d, only that process pauses, not the entire runtime. There are no stop-the-world GC pauses in the BEAM. A garbage collection event on one process does not add latency to any other process. This is why BEAM-based systems can maintain consistent tail latencies under load in ways that JVM-based or Go-based systems cannot.

Built-in distribution. BEAM nodes can connect to each other and communicate as if they were on the same machine. You can call a GenServer running on a different physical host using the same API as calling one on localhost. Distribution is a first-class feature of the runtime, not a library bolt-on.

Hot code upgrades. The BEAM supports loading new versions of modules while the system is running and while processes are actively executing old code. This is how telephone exchanges were upgraded without dropping calls. In practice most modern Elixir systems use rolling restarts instead, but the capability is there and tools like Distillery and Mix Releases support it.

The “let it crash” philosophy. This is perhaps the hardest mental shift for engineers coming from Python, Ruby, or Go. In most languages, defensive programming is standard: check every error, handle every nil, write try/catch around anything that might fail. The BEAM’s answer to failure is different: let the process crash, and have a supervisor restart it. Process isolation means a crashing process cannot corrupt state in other processes. Supervisors can restart failed processes in milliseconds. The result is systems that are more reliable precisely because they do not try to handle every error inline.

Compare these two approaches:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
# Python: defensive programming
def process_message(msg):
    try:
        result = parse_and_validate(msg)
        if result is None:
            logger.error("Null result from parsing")
            return None
        processed = apply_business_logic(result)
        if processed is None:
            logger.error("Business logic returned None")
            return None
        return save_to_db(processed)
    except ParseError as e:
        logger.error(f"Parse failed: {e}")
        return None
    except DatabaseError as e:
        logger.error(f"DB error: {e}")
        return None
    except Exception as e:
        logger.error(f"Unexpected: {e}")
        return None
1
2
3
4
5
6
# Elixir: let it crash
def process_message(msg) do
  result = parse_and_validate!(msg)   # raises on failure
  processed = apply_business_logic!(result)
  save_to_db!(processed)
end

In Elixir, if parse_and_validate! raises, the calling process crashes. The supervisor notices, logs the crash with a full stacktrace, and restarts the process. The error is not silently swallowed. The system recovers automatically. For anything that can be retried (and most things can), this produces more reliable software with less code.


2. Elixir’s Role

José Valim’s Creation

José Valim was a prominent Ruby on Rails core contributor. In 2011, while exploring concurrency solutions for Rails, he discovered the BEAM and recognized that it solved problems the Ruby runtime could not. Rather than use Erlang directly — which has a syntax many find off-putting — he built Elixir: a language that compiles to BEAM bytecode, has full interoperability with Erlang libraries, but uses a Ruby-inspired syntax with significant improvements.

Elixir’s key innovations over Erlang:

  • Friendly, consistent syntax with Unicode identifiers, do/end blocks, and familiar module patterns
  • A powerful macro system that allows extending the language without runtime overhead
  • The pipe operator (|>) which dramatically improves readability for data transformation chains
  • Protocols for polymorphism (similar to Go interfaces or Haskell typeclasses)
  • First-class tooling with Mix and Hex from the start
  • String handling that actually makes sense (Erlang’s strings are lists of integers)

Erlang Interop

Elixir and Erlang share the same runtime. Elixir code compiles to the same BEAM bytecode as Erlang code. You can call any Erlang function from Elixir with the :erlang_module.function() syntax:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Calling Erlang's :crypto module directly
:crypto.hash(:sha256, "hello world")

# Erlang's :ets (Erlang Term Storage)
table = :ets.new(:my_table, [:set, :public])
:ets.insert(table, {:key, "value"})
:ets.lookup(table, :key)

# Erlang's :timer module
:timer.sleep(1000)
:timer.send_after(5000, self(), :timeout)

The entire Erlang/OTP standard library is available in Elixir. The Hex package manager hosts both Elixir and Erlang packages.

Recent Elixir Features (1.16/1.17)

Elixir 1.16 and 1.17 (released in 2024) brought several notable improvements:

  • Improved compiler diagnostics with better error messages that point directly to the problematic code with context
  • dbg/2 enhancements in IEx for pipeline inspection
  • Mix install improvements for scripting use cases
  • Process.set_label/1 for naming processes without registering them globally
  • Kernel.then/2 and tap/2 for cleaner pipelines
  • Stacktrace improvements in ExUnit for clearer test failures
  • Duration and Calendar.ISO updates for date/time arithmetic

3. Core Language

Immutability

All data in Elixir is immutable. Variables do not hold mutable state — they hold references to immutable values. When you “update” a data structure, you get a new data structure sharing as much structure as possible with the original (persistent data structures).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# "Mutating" a map creates a new map
original = %{name: "Alice", age: 30}
updated = Map.put(original, :age, 31)

IO.inspect(original)  # %{name: "Alice", age: 30} — unchanged
IO.inspect(updated)   # %{name: "Alice", age: 31}

# Lists are immutable linked lists
list = [1, 2, 3]
new_list = [0 | list]   # prepend — efficient
IO.inspect(list)        # [1, 2, 3] — unchanged
IO.inspect(new_list)    # [0, 1, 2, 3]

Immutability is not a constraint — it is what makes concurrent programming safe. Multiple processes can hold references to the same data structure without coordination because neither can mutate it.

Pattern Matching: The Central Idiom

In Elixir, = is not assignment — it is the match operator. It attempts to match the right side against the left side, binding variables in the process. This is the most important concept in Elixir.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
# Simple binding
x = 42          # x is now bound to 42

# Matching a tuple
{:ok, result} = {:ok, "hello"}   # result = "hello"
{:ok, result} = {:error, "fail"} # raises MatchError!

# Matching a list
[head | tail] = [1, 2, 3, 4]
IO.inspect(head)  # 1
IO.inspect(tail)  # [2, 3, 4]

# Matching nested structures
%{user: %{name: name, role: :admin}} = %{
  user: %{name: "Alice", role: :admin, age: 30}
}
IO.inspect(name)  # "Alice"

# The pin operator ^ prevents rebinding
x = 5
^x = 5   # matches — x is still 5
^x = 6   # raises MatchError — 6 does not match 5

Pattern matching in function heads is how Elixir achieves what other languages do with if/else chains and type dispatch:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
defmodule HttpHandler do
  # Match specific status codes in function heads
  def handle_response({:ok, %{status: 200, body: body}}) do
    {:ok, parse_body(body)}
  end

  def handle_response({:ok, %{status: 404}}) do
    {:error, :not_found}
  end

  def handle_response({:ok, %{status: status}}) when status >= 500 do
    {:error, {:server_error, status}}
  end

  def handle_response({:ok, %{status: status}}) do
    {:error, {:unexpected_status, status}}
  end

  def handle_response({:error, reason}) do
    {:error, {:network_error, reason}}
  end

  defp parse_body(body), do: Jason.decode!(body)
end

This is exhaustive, readable, and requires no isinstance checks or null guards. The compiler will warn you if you have unreachable clauses.

case, cond, with

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# case — match a value against multiple patterns
case Map.fetch(config, :timeout) do
  {:ok, timeout} when timeout > 0 ->
    {:ok, timeout}
  {:ok, timeout} ->
    {:error, "timeout must be positive, got: #{timeout}"}
  :error ->
    {:ok, 5000}  # default
end

# cond — like a chain of if/else if
cond do
  System.get_env("PROD") == "true" ->
    :production
  System.get_env("STAGING") == "true" ->
    :staging
  true ->
    :development
end

# with — chain operations that may fail, short-circuiting on error
# This is the idiomatic way to sequence fallible operations
def create_user(params) do
  with {:ok, changeset} <- User.changeset(params),
       {:ok, user}      <- Repo.insert(changeset),
       {:ok, _email}    <- Mailer.send_welcome(user.email) do
    {:ok, user}
  else
    {:error, %Ecto.Changeset{} = cs} -> {:error, {:validation, cs}}
    {:error, :email_failed}          -> {:error, :email_failed}
    # Unmatched errors bubble up automatically
  end
end

The Pipe Operator

The pipe operator |> passes the result of one expression as the first argument to the next function. It makes data transformation pipelines readable in the order operations are applied:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# Without pipes — reads inside-out
result = Enum.join(Enum.map(String.split(String.downcase("Hello World"), " "), &String.capitalize/1), "-")

# With pipes — reads left to right, top to bottom
result =
  "Hello World"
  |> String.downcase()
  |> String.split(" ")
  |> Enum.map(&String.capitalize/1)
  |> Enum.join("-")

# result = "Hello-World"

# Real-world pipeline: processing an HTTP request body
def process_order(raw_body) do
  raw_body
  |> Jason.decode!()
  |> validate_order_params()
  |> apply_discount_rules()
  |> calculate_totals()
  |> persist_order()
  |> notify_fulfillment_service()
end

Modules and Functions

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
defmodule Accounts do
  # Module attribute (compile-time constant)
  @min_password_length 12
  @max_failed_attempts 5

  # Public function
  def register(email, password) do
    with :ok <- validate_email(email),
         :ok <- validate_password(password),
         {:ok, user} <- create_user(email, password) do
      {:ok, user}
    end
  end

  # Guard clauses
  def lock_account(user) when user.failed_attempts >= @max_failed_attempts do
    {:ok, %{user | locked: true}}
  end

  def lock_account(user), do: {:ok, user}

  # Private function
  defp validate_password(password) when byte_size(password) < @min_password_length do
    {:error, "Password must be at least #{@min_password_length} characters"}
  end

  defp validate_password(_password), do: :ok

  defp validate_email(email) do
    if String.contains?(email, "@") do
      :ok
    else
      {:error, "Invalid email format"}
    end
  end

  defp create_user(email, password) do
    hashed = Bcrypt.hash_pwd_salt(password)
    Repo.insert(%User{email: email, password_hash: hashed})
  end
end

Anonymous Functions and Captures

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# Anonymous function
double = fn x -> x * 2 end
double.(5)  # 10

# Shorthand capture syntax
double = &(&1 * 2)
double.(5)  # 10

# Capturing named functions
add = &Kernel.+/2
add.(3, 4)  # 7

# &Module.function/arity
upcase = &String.upcase/1
["hello", "world"] |> Enum.map(upcase)  # ["HELLO", "WORLD"]

# Functions are first-class
apply_twice = fn f, x -> f.(f.(x)) end
apply_twice.(&(&1 + 1), 5)  # 7

Enum and Stream

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# Enum — eager evaluation, returns a new list
numbers = 1..100

squares_of_evens =
  numbers
  |> Enum.filter(&(rem(&1, 2) == 0))
  |> Enum.map(&(&1 * &1))
  |> Enum.take(5)
# [4, 16, 36, 64, 100]

# Common Enum operations
Enum.reduce([1, 2, 3, 4, 5], 0, &+/2)   # 15
Enum.group_by(["alice", "bob", "al"], &String.first/1)
# %{"a" => ["alice", "al"], "b" => ["bob"]}

Enum.chunk_every([1,2,3,4,5,6], 2)       # [[1,2], [3,4], [5,6]]
Enum.flat_map([[1,2],[3,4]], & &1)        # [1, 2, 3, 4]
Enum.zip([1,2,3], [:a, :b, :c])          # [{1,:a}, {2,:b}, {3,:c}]

# Stream — lazy evaluation, composes without intermediate lists
# Essential for large datasets or infinite sequences
result =
  File.stream!("huge_log.txt")              # reads line by line
  |> Stream.filter(&String.contains?(&1, "ERROR"))
  |> Stream.map(&parse_log_line/1)
  |> Stream.reject(&is_nil/1)
  |> Enum.take(100)                          # only materializes 100 items

# Infinite stream
Stream.iterate(0, &(&1 + 1))
|> Stream.filter(&(rem(&1, 3) == 0))
|> Enum.take(5)
# [0, 3, 6, 9, 12]

Comprehensions

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
# for comprehension with filter
squares = for n <- 1..10, rem(n, 2) == 0, do: n * n
# [4, 16, 36, 64, 100]

# Multiple generators (cartesian product)
pairs = for x <- [:a, :b], y <- [1, 2], do: {x, y}
# [{:a, 1}, {:a, 2}, {:b, 1}, {:b, 2}]

# into: accumulate into a map
freq_map =
  for char <- String.graphemes("mississippi"),
      reduce: %{} do
    acc -> Map.update(acc, char, 1, &(&1 + 1))
  end
# %{"i" => 4, "m" => 1, "p" => 2, "s" => 4}

# Generate a lookup map from a list of structs
user_index = for user <- users, into: %{}, do: {user.id, user}

4. Processes and Message Passing

Spawning Processes

Every concurrent operation in Elixir happens in a process. Processes are the unit of concurrency and the unit of failure isolation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# Spawn a process — returns a PID
pid = spawn(fn ->
  IO.puts("Running in process #{inspect(self())}")
  :timer.sleep(1000)
  IO.puts("Done")
end)

IO.inspect(pid)  # #PID<0.123.0>

# spawn_link — crash propagates to parent
pid = spawn_link(fn ->
  raise "something went wrong"
end)
# The parent process will also crash unless it traps exits

# spawn_monitor — get a monitor reference
{pid, ref} = spawn_monitor(fn ->
  :timer.sleep(100)
end)

send, receive, and Mailboxes

Every BEAM process has a mailbox — an ordered queue of messages. The send/2 function is non-blocking; receive blocks until a matching message arrives.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
defmodule Counter do
  def start do
    spawn(fn -> loop(0) end)
  end

  defp loop(count) do
    receive do
      {:increment, amount} ->
        loop(count + amount)

      {:get, caller_pid} ->
        send(caller_pid, {:count, count})
        loop(count)

      :reset ->
        loop(0)

      :stop ->
        IO.puts("Counter stopping with final count: #{count}")
        # Process exits normally — no more recursive call
    end
  end
end

# Using the counter
pid = Counter.start()

send(pid, {:increment, 5})
send(pid, {:increment, 3})
send(pid, {:get, self()})

receive do
  {:count, n} -> IO.puts("Count: #{n}")  # Count: 8
after
  5000 -> IO.puts("Timeout!")  # after clause for timeouts
end

send(pid, :stop)

Process Monitoring and Linking

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
# Monitor — receive a message when process dies, non-fatal
ref = Process.monitor(some_pid)

receive do
  {:DOWN, ^ref, :process, pid, reason} ->
    IO.puts("Process #{inspect(pid)} died: #{inspect(reason)}")
end

# Link — bidirectional, crash propagates both ways
Process.link(some_pid)

# Trapping exits — convert EXIT signals into messages
# This is how supervisors work under the hood
Process.flag(:trap_exit, true)

spawn_link(fn -> raise "crash" end)

receive do
  {:EXIT, pid, reason} ->
    IO.puts("Linked process #{inspect(pid)} crashed: #{inspect(reason)}")
    # Handle the crash without dying yourself
end

The Actor Model in Practice

The actor model is the theoretical basis for BEAM concurrency: everything is an actor (process), actors communicate only through messages, and actors can create new actors. There is no shared mutable state between actors.

This eliminates an entire class of bugs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# In Go, you need explicit synchronization for shared state
# mutex.Lock()
# counter++
# mutex.Unlock()

# In Elixir, state lives inside a process — only accessible through messages
defmodule RateLimiter do
  def start_link(max_per_second) do
    pid = spawn_link(fn -> init(max_per_second) end)
    {:ok, pid}
  end

  defp init(max_per_second) do
    state = %{
      max: max_per_second,
      count: 0,
      window_start: System.monotonic_time(:second)
    }
    loop(state)
  end

  defp loop(state) do
    now = System.monotonic_time(:second)

    state =
      if now > state.window_start do
        %{state | count: 0, window_start: now}
      else
        state
      end

    receive do
      {:check, caller} ->
        if state.count < state.max do
          send(caller, :allowed)
          loop(%{state | count: state.count + 1})
        else
          send(caller, :denied)
          loop(state)
        end
    end
  end

  def check(pid) do
    send(pid, {:check, self()})
    receive do
      result -> result
    after
      100 -> :timeout
    end
  end
end

All state mutations happen in a single process. No locks needed. No race conditions possible.


5. OTP: Open Telecom Platform

OTP is not just a library — it is a set of design principles, behaviors, and abstractions that encode decades of hard-won experience building fault-tolerant distributed systems. Understanding OTP is the difference between writing Elixir that looks like Erlang with nicer syntax and writing Elixir that actually leverages the BEAM’s unique strengths.

GenServer: The Workhorse

GenServer (Generic Server) is the most commonly used OTP behavior. It abstracts the receive loop into a structured pattern with callbacks.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
defmodule Cache do
  use GenServer

  # Client API (runs in caller's process)

  def start_link(opts \\ []) do
    name = Keyword.get(opts, :name, __MODULE__)
    GenServer.start_link(__MODULE__, %{}, name: name)
  end

  def get(server \\ __MODULE__, key) do
    GenServer.call(server, {:get, key})
  end

  def put(server \\ __MODULE__, key, value, ttl_ms \\ :infinity) do
    GenServer.cast(server, {:put, key, value, ttl_ms})
  end

  def delete(server \\ __MODULE__, key) do
    GenServer.cast(server, {:delete, key})
  end

  def stats(server \\ __MODULE__) do
    GenServer.call(server, :stats)
  end

  # Server callbacks (run in GenServer's process)

  @impl true
  def init(_args) do
    # Schedule periodic cleanup
    Process.send_after(self(), :cleanup, 60_000)
    {:ok, %{entries: %{}, hits: 0, misses: 0}}
  end

  @impl true
  def handle_call({:get, key}, _from, state) do
    case Map.get(state.entries, key) do
      nil ->
        {:reply, {:error, :not_found}, %{state | misses: state.misses + 1}}

      {value, :infinity} ->
        {:reply, {:ok, value}, %{state | hits: state.hits + 1}}

      {value, expires_at} ->
        if System.monotonic_time(:millisecond) < expires_at do
          {:reply, {:ok, value}, %{state | hits: state.hits + 1}}
        else
          new_entries = Map.delete(state.entries, key)
          {:reply, {:error, :expired}, %{state | entries: new_entries, misses: state.misses + 1}}
        end
    end
  end

  @impl true
  def handle_call(:stats, _from, state) do
    stats = %{
      size: map_size(state.entries),
      hits: state.hits,
      misses: state.misses,
      hit_rate: if(state.hits + state.misses > 0, do: state.hits / (state.hits + state.misses), else: 0.0)
    }
    {:reply, stats, state}
  end

  @impl true
  def handle_cast({:put, key, value, ttl_ms}, state) do
    expires_at =
      case ttl_ms do
        :infinity -> :infinity
        ms -> System.monotonic_time(:millisecond) + ms
      end

    new_entries = Map.put(state.entries, key, {value, expires_at})
    {:noreply, %{state | entries: new_entries}}
  end

  @impl true
  def handle_cast({:delete, key}, state) do
    new_entries = Map.delete(state.entries, key)
    {:noreply, %{state | entries: new_entries}}
  end

  @impl true
  def handle_info(:cleanup, state) do
    now = System.monotonic_time(:millisecond)

    new_entries =
      Enum.reject(state.entries, fn
        {_key, {_value, :infinity}} -> false
        {_key, {_value, expires_at}} -> now >= expires_at
      end)
      |> Map.new()

    Process.send_after(self(), :cleanup, 60_000)
    {:noreply, %{state | entries: new_entries}}
  end
end

The critical distinction between call and cast:

  • GenServer.call/3 is synchronous — it blocks the caller until the server responds. Use it when you need a result or need backpressure.
  • GenServer.cast/2 is asynchronous — fire and forget. Use it for side effects where you do not need confirmation.

Supervisor

Supervisors watch over worker processes and restart them according to a defined strategy.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
defmodule MyApp.Supervisor do
  use Supervisor

  def start_link(opts) do
    Supervisor.start_link(__MODULE__, opts, name: __MODULE__)
  end

  @impl true
  def init(_opts) do
    children = [
      # Worker with default child spec
      Cache,

      # Worker with explicit configuration
      {RateLimiter, max_per_second: 1000},

      # Worker with a name and restart policy
      %{
        id: :session_cleaner,
        start: {SessionCleaner, :start_link, [[interval: 300_000]]},
        restart: :permanent,
        shutdown: 5_000,
        type: :worker
      },

      # Nested supervisor
      {JobQueue.Supervisor, [pool_size: 10]}
    ]

    # Supervision strategies:
    # :one_for_one  — restart only the failed child (most common)
    # :one_for_all  — restart all children if any fails (when children are interdependent)
    # :rest_for_one — restart the failed child and all children started after it
    Supervisor.init(children, strategy: :one_for_one, max_restarts: 3, max_seconds: 5)
  end
end

The max_restarts and max_seconds parameters implement circuit-breaker logic at the supervisor level: if a child crashes more than 3 times in 5 seconds, the supervisor itself crashes and propagates the failure up the supervision tree.

Supervision Trees

The power of OTP comes from composing supervisors into trees. The root supervisor starts your application. It supervises other supervisors, which supervise workers. Failures are contained and handled at the appropriate level.

MyApp.Application
├── MyApp.Supervisor
│   ├── Cache (GenServer)
│   ├── RateLimiter (GenServer)
│   └── JobQueue.Supervisor
│       ├── JobQueue.Worker (1)
│       ├── JobQueue.Worker (2)
│       └── JobQueue.Worker (3)
├── MyAppWeb.Endpoint (Phoenix)
│   ├── Cowboy.Supervisor
│   └── Phoenix.PubSub
└── MyApp.Repo (Ecto)

Each node in this tree has a defined fault isolation boundary. If a JobQueue.Worker crashes, the JobQueue.Supervisor restarts it without affecting the Cache or the HTTP endpoint. If the entire JobQueue.Supervisor crashes repeatedly, the MyApp.Supervisor might crash, but MyAppWeb.Endpoint continues serving requests.

Application

An Application is the top-level OTP behavior — the entry point for your system:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
defmodule MyApp.Application do
  use Application

  @impl true
  def start(_type, _args) do
    children = [
      # Start the Ecto repository
      MyApp.Repo,

      # Start the Telemetry supervisor
      MyAppWeb.Telemetry,

      # Start DNS cluster for distributed operation
      {DNSCluster, query: Application.get_env(:my_app, :dns_cluster_query) || :ignore},

      # PubSub for Phoenix
      {Phoenix.PubSub, name: MyApp.PubSub},

      # Finch HTTP client
      {Finch, name: MyApp.Finch},

      # Application supervisor
      MyApp.Supervisor,

      # Phoenix endpoint (last — depends on everything above)
      MyAppWeb.Endpoint
    ]

    opts = [strategy: :one_for_one, name: MyApp.Supervisor.Root]
    Supervisor.start_link(children, opts)
  end

  @impl true
  def config_change(changed, _new, removed) do
    MyAppWeb.Endpoint.config_change(changed, removed)
    :ok
  end
end

ETS: Erlang Term Storage

ETS is an in-memory key-value store built into the BEAM. Unlike a GenServer, ETS allows concurrent reads from multiple processes without message passing:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
defmodule SessionStore do
  @table :sessions

  def init do
    :ets.new(@table, [
      :named_table,
      :set,             # unique keys
      :public,          # any process can read/write
      read_concurrency: true,   # optimized for concurrent reads
      write_concurrency: true   # optimized for concurrent writes
    ])
  end

  def put(session_id, data, ttl_seconds) do
    expires_at = System.os_time(:second) + ttl_seconds
    :ets.insert(@table, {session_id, data, expires_at})
    :ok
  end

  def get(session_id) do
    now = System.os_time(:second)
    case :ets.lookup(@table, session_id) do
      [{^session_id, data, expires_at}] when expires_at > now ->
        {:ok, data}
      [{^session_id, _data, _expires_at}] ->
        :ets.delete(@table, session_id)
        {:error, :expired}
      [] ->
        {:error, :not_found}
    end
  end

  def delete(session_id) do
    :ets.delete(@table, session_id)
    :ok
  end

  def cleanup_expired do
    now = System.os_time(:second)
    # Match spec: delete all entries where expires_at <= now
    match_spec = [{{:_, :_, :"$1"}, [{:"=<", :"$1", now}], [true]}]
    deleted = :ets.select_delete(@table, match_spec)
    {:ok, deleted}
  end
end

ETS gives you O(1) or O(log n) lookups without process overhead. It is the right tool when many processes need concurrent read access to shared data (session stores, caches, configuration, routing tables).


6. Phoenix Framework

Phoenix is the dominant web framework in the Elixir ecosystem. It was built by Chris McCord and sits on top of Plug (a composable middleware specification) and Cowboy (an Erlang HTTP server). Phoenix is consistently among the fastest web frameworks in benchmarks like TechEmpower — not because of micro-optimizations, but because the BEAM’s architecture handles concurrent connections exceptionally well and Phoenix does not add unnecessary overhead.

The Architecture

Request → Cowboy (HTTP server) → Plug pipeline → Phoenix Router → Controller → View/Template

Phoenix 1.7 restructured views with Verified Routes and function components.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# router.ex
defmodule MyAppWeb.Router do
  use MyAppWeb, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_live_flash
    plug :put_root_layout, html: {MyAppWeb.Layouts, :root}
    plug :protect_from_forgery
    plug :put_secure_browser_headers
    plug MyAppWeb.Plugs.SetCurrentUser
  end

  pipeline :api do
    plug :accepts, ["json"]
    plug MyAppWeb.Plugs.AuthenticateToken
  end

  scope "/", MyAppWeb do
    pipe_through :browser

    get "/", PageController, :index
    live "/dashboard", DashboardLive
    live "/dashboard/:id", DashboardLive, :show

    resources "/users", UserController, except: [:delete]
  end

  scope "/api/v1", MyAppWeb.API.V1 do
    pipe_through :api

    resources "/events", EventController, only: [:index, :create, :show]
    post "/events/:id/process", EventController, :process
  end
end

Controllers

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
defmodule MyAppWeb.UserController do
  use MyAppWeb, :controller

  alias MyApp.Accounts

  action_fallback MyAppWeb.FallbackController

  def index(conn, params) do
    users = Accounts.list_users(params)
    render(conn, :index, users: users)
  end

  def create(conn, %{"user" => user_params}) do
    with {:ok, user} <- Accounts.create_user(user_params) do
      conn
      |> put_status(:created)
      |> put_resp_header("location", ~p"/api/v1/users/#{user}")
      |> render(:show, user: user)
    end
  end

  def show(conn, %{"id" => id}) do
    with {:ok, user} <- Accounts.get_user(id) do
      render(conn, :show, user: user)
    end
  end
end

# Fallback controller handles {:error, reason} tuples from with
defmodule MyAppWeb.FallbackController do
  use Phoenix.Controller

  def call(conn, {:error, :not_found}) do
    conn
    |> put_status(:not_found)
    |> put_view(json: MyAppWeb.ErrorJSON)
    |> render(:"404")
  end

  def call(conn, {:error, %Ecto.Changeset{} = changeset}) do
    conn
    |> put_status(:unprocessable_entity)
    |> put_view(json: MyAppWeb.ChangesetJSON)
    |> render(:error, changeset: changeset)
  end
end

Channels for WebSockets

Phoenix Channels provide an abstraction over WebSockets (and other transports) with rooms/topics:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
defmodule MyAppWeb.ChatChannel do
  use Phoenix.Channel

  alias MyApp.Chat
  alias MyAppWeb.Presence

  def join("room:" <> room_id, _payload, socket) do
    if authorized?(socket, room_id) do
      send(self(), :after_join)
      {:ok, assign(socket, :room_id, room_id)}
    else
      {:error, %{reason: "unauthorized"}}
    end
  end

  def handle_info(:after_join, socket) do
    # Track presence
    {:ok, _} = Presence.track(socket, socket.assigns.user_id, %{
      online_at: inspect(System.system_time(:second))
    })

    # Send recent messages
    messages = Chat.recent_messages(socket.assigns.room_id, limit: 50)
    push(socket, "history", %{messages: messages})

    # Broadcast updated presence list
    push(socket, "presence_state", Presence.list(socket))

    {:noreply, socket}
  end

  def handle_in("new_message", %{"body" => body}, socket) do
    user_id = socket.assigns.user_id
    room_id = socket.assigns.room_id

    case Chat.create_message(room_id, user_id, body) do
      {:ok, message} ->
        broadcast!(socket, "new_message", %{
          id: message.id,
          user_id: user_id,
          body: message.body,
          inserted_at: message.inserted_at
        })
        {:reply, :ok, socket}

      {:error, changeset} ->
        {:reply, {:error, %{errors: format_errors(changeset)}}, socket}
    end
  end

  def handle_in("typing", _payload, socket) do
    broadcast_from!(socket, "user_typing", %{user_id: socket.assigns.user_id})
    {:noreply, socket}
  end

  defp authorized?(socket, room_id) do
    Chat.member?(socket.assigns.user_id, room_id)
  end
end

Phoenix Presence

Presence tracks which users are connected to which topics across a cluster — using CRDTs under the hood so it works without a central coordinator:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
defmodule MyAppWeb.Presence do
  use Phoenix.Presence,
    otp_app: :my_app,
    pubsub_server: MyApp.PubSub
end

# In a LiveView or Channel:
Presence.track(self(), "room:lobby", user_id, %{
  name: user.name,
  avatar_url: user.avatar_url,
  joined_at: DateTime.utc_now()
})

# Get current presence list
presences = Presence.list("room:lobby")
# %{
#   "user_123" => %{metas: [%{name: "Alice", ...}]},
#   "user_456" => %{metas: [%{name: "Bob", ...}]}
# }

The BEAM’s distribution makes this straightforward — presence information propagates through the cluster automatically.

Telemetry Integration

Phoenix emits telemetry events for every request, which you attach handlers to:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
defmodule MyAppWeb.Telemetry do
  use Supervisor
  import Telemetry.Metrics

  def start_link(arg) do
    Supervisor.start_link(__MODULE__, arg, name: __MODULE__)
  end

  def init(_arg) do
    children = [
      {:telemetry_poller, measurements: periodic_measurements(), period: 10_000},
      {TelemetryMetricsPrometheus, metrics: metrics()}
    ]
    Supervisor.init(children, strategy: :one_for_one)
  end

  def metrics do
    [
      # Phoenix metrics
      summary("phoenix.endpoint.start.system_time", unit: {:native, :millisecond}),
      summary("phoenix.endpoint.stop.duration", unit: {:native, :millisecond}),
      summary("phoenix.router_dispatch.stop.duration",
        tags: [:route],
        unit: {:native, :millisecond}
      ),

      # Ecto metrics
      summary("my_app.repo.query.total_time", unit: {:native, :millisecond}),
      summary("my_app.repo.query.decode_time", unit: {:native, :millisecond}),
      summary("my_app.repo.query.query_time", unit: {:native, :millisecond}),
      counter("my_app.repo.query.count"),

      # VM metrics
      last_value("vm.memory.total", unit: {:byte, :kilobyte}),
      last_value("vm.total_run_queue_lengths.total"),
      last_value("vm.total_run_queue_lengths.cpu")
    ]
  end

  defp periodic_measurements do
    [{__MODULE__, :dispatch_node_stats, []}]
  end

  def dispatch_node_stats do
    :telemetry.execute([:my_app, :node], %{
      connected_nodes: length(Node.list()),
      process_count: length(Process.list())
    })
  end
end

7. Phoenix LiveView

LiveView is the most distinctive feature of the Phoenix ecosystem. It enables interactive, real-time UIs driven entirely by server-side rendering, without writing JavaScript. The client-side library (~35KB) maintains a WebSocket connection and applies server-sent HTML diffs.

How It Works

  1. User requests a LiveView page over HTTP — the initial HTML is server-rendered and returned (good for SEO and first-paint)
  2. The client connects over WebSocket and the LiveView process mounts
  3. User interactions send events over WebSocket to the server
  4. The server updates state and calls render/1
  5. LiveView diffs the new render against the previous render and sends only the changed parts
  6. The client applies the diff — no full page reload

The diff-and-patch protocol is efficient enough that LiveView handles complex UIs with many concurrent users without noticeable latency.

A Complete LiveView Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
# lib/my_app_web/live/search_live.ex
defmodule MyAppWeb.SearchLive do
  use MyAppWeb, :live_view

  alias MyApp.Catalog

  @impl true
  def mount(_params, _session, socket) do
    socket =
      socket
      |> assign(:query, "")
      |> assign(:results, [])
      |> assign(:loading, false)
      |> assign(:filter, :all)

    {:ok, socket}
  end

  @impl true
  def handle_params(%{"q" => query}, _uri, socket) do
    socket = search(socket, query)
    {:noreply, socket}
  end

  def handle_params(_params, _uri, socket) do
    {:noreply, socket}
  end

  @impl true
  def handle_event("search", %{"query" => query}, socket) do
    socket =
      socket
      |> assign(:query, query)
      |> assign(:loading, true)

    # Push navigation so the URL updates without a page reload
    {:noreply, push_patch(socket, to: ~p"/search?q=#{query}")}
  end

  def handle_event("filter", %{"type" => filter}, socket) do
    filter = String.to_existing_atom(filter)
    results = apply_filter(socket.assigns.results, filter)
    {:noreply, assign(socket, filter: filter, filtered_results: results)}
  end

  # handle_info receives messages sent to the LiveView process
  @impl true
  def handle_info({:search_results, results}, socket) do
    {:noreply, assign(socket, results: results, loading: false)}
  end

  defp search(socket, query) when byte_size(query) < 2 do
    assign(socket, results: [], loading: false)
  end

  defp search(socket, query) do
    # Async search — sends a message to self when done
    pid = self()
    Task.start(fn ->
      results = Catalog.search(query)
      send(pid, {:search_results, results})
    end)

    assign(socket, loading: true)
  end

  defp apply_filter(results, :all), do: results
  defp apply_filter(results, filter), do: Enum.filter(results, &(&1.type == filter))
end
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
<%!-- lib/my_app_web/live/search_live.html.heex --%>
<div class="search-container">
  <form phx-submit="search" phx-change="search">
    <input
      type="text"
      name="query"
      value={@query}
      placeholder="Search..."
      phx-debounce="300"
      autofocus
    />
    <button type="submit">Search</button>
  </form>

  <div class="filters">
    <button phx-click="filter" phx-value-type="all"
            class={if @filter == :all, do: "active"}>All</button>
    <button phx-click="filter" phx-value-type="product"
            class={if @filter == :product, do: "active"}>Products</button>
    <button phx-click="filter" phx-value-type="article"
            class={if @filter == :article, do: "active"}>Articles</button>
  </div>

  <%= if @loading do %>
    <div class="spinner" aria-label="Loading..."></div>
  <% else %>
    <div class="results">
      <%= for result <- @results do %>
        <div class="result-card" id={"result-#{result.id}"}>
          <h3><%= result.title %></h3>
          <p><%= result.excerpt %></p>
          <span class="badge"><%= result.type %></span>
        </div>
      <% end %>

      <%= if @results == [] and @query != "" do %>
        <p class="empty-state">No results for "<%= @query %>"</p>
      <% end %>
    </div>
  <% end %>
</div>

Live Components

For reusable, stateful UI pieces:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
defmodule MyAppWeb.CounterComponent do
  use Phoenix.LiveComponent

  def render(assigns) do
    ~H"""
    <div class="counter" id={@id}>
      <button phx-click="decrement" phx-target={@myself}>-</button>
      <span><%= @count %></span>
      <button phx-click="increment" phx-target={@myself}>+</button>
    </div>
    """
  end

  def mount(socket) do
    {:ok, assign(socket, count: 0)}
  end

  def handle_event("increment", _params, socket) do
    {:noreply, update(socket, :count, &(&1 + 1))}
  end

  def handle_event("decrement", _params, socket) do
    {:noreply, update(socket, :count, &(&1 - 1))}
  end
end

# Used in a parent LiveView template:
# <.live_component module={CounterComponent} id="counter-1" />

Streams for Large Lists

LiveView Streams were introduced in LiveView 0.18 to handle large, dynamically updating lists efficiently — instead of re-rendering the entire list, only changed items are patched:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
defmodule MyAppWeb.MessagesLive do
  use MyAppWeb, :live_view

  @impl true
  def mount(_params, _session, socket) do
    if connected?(socket) do
      Phoenix.PubSub.subscribe(MyApp.PubSub, "messages")
    end

    messages = MyApp.Chat.list_recent_messages(limit: 100)

    socket =
      socket
      |> stream(:messages, messages)

    {:ok, socket}
  end

  @impl true
  def handle_info({:new_message, message}, socket) do
    # Prepend new message to the stream — only this item is sent to client
    {:noreply, stream_insert(socket, :messages, message, at: 0)}
  end

  def handle_info({:message_deleted, message}, socket) do
    {:noreply, stream_delete(socket, :messages, message)}
  end
end
1
2
3
4
5
6
<%!-- Only changed items are patched in the DOM --%>
<div id="messages" phx-update="stream">
  <div :for={{dom_id, message} <- @streams.messages} id={dom_id}>
    <strong><%= message.user.name %></strong>: <%= message.body %>
  </div>
</div>

When LiveView is the Right Choice

LiveView excels when:

  • You want real-time updates (dashboards, live feeds, collaborative tools)
  • The team is Elixir-focused and you want to avoid context switching to JavaScript
  • You need server-rendered HTML for SEO or first-paint performance
  • The latency to your users allows ~50–100ms round trips (usually true for users on same continent as server)

LiveView is not the right choice when:

  • You need offline functionality (PWA requirements)
  • Users are on high-latency connections where even small interaction delays are noticeable
  • You have a large existing JavaScript codebase and team
  • You need very complex client-side interactions (canvas drawing, local video/audio processing)

8. Ecto: Database Access Done Right

Ecto is Elixir’s database library, and it takes a different philosophy from ActiveRecord-style ORMs. Ecto separates concerns explicitly: schemas describe data structures, changesets handle validation and casting, repositories handle database access, and queries are composable data structures — not strings.

Schema and Changeset

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
defmodule MyApp.Accounts.User do
  use Ecto.Schema
  import Ecto.Changeset

  schema "users" do
    field :email, :string
    field :name, :string
    field :password, :string, virtual: true
    field :password_hash, :string
    field :role, Ecto.Enum, values: [:user, :admin, :moderator], default: :user
    field :confirmed_at, :utc_datetime
    field :failed_login_attempts, :integer, default: 0
    field :locked_at, :utc_datetime

    has_many :posts, MyApp.Blog.Post
    has_one :profile, MyApp.Accounts.Profile
    many_to_many :teams, MyApp.Teams.Team, join_through: "user_teams"

    timestamps()  # inserts inserted_at and updated_at
  end

  # Registration changeset — used when creating a new user
  def registration_changeset(user, attrs) do
    user
    |> cast(attrs, [:email, :name, :password])
    |> validate_required([:email, :name, :password])
    |> validate_format(:email, ~r/^[^\s]+@[^\s]+$/, message: "must be a valid email address")
    |> validate_length(:password, min: 12, max: 72)
    |> validate_length(:name, min: 2, max: 100)
    |> unique_constraint(:email)
    |> put_password_hash()
  end

  # Update changeset — used when updating profile
  def update_changeset(user, attrs) do
    user
    |> cast(attrs, [:name])
    |> validate_required([:name])
    |> validate_length(:name, min: 2, max: 100)
  end

  # Password reset changeset
  def password_changeset(user, attrs) do
    user
    |> cast(attrs, [:password])
    |> validate_required([:password])
    |> validate_length(:password, min: 12, max: 72)
    |> put_password_hash()
  end

  defp put_password_hash(%Ecto.Changeset{valid?: true, changes: %{password: pwd}} = changeset) do
    put_change(changeset, :password_hash, Bcrypt.hash_pwd_salt(pwd))
  end
  defp put_password_hash(changeset), do: changeset
end

The changeset pattern is more explicit than ActiveRecord’s before_validate callbacks. You define exactly what fields are castable, what constraints apply, and changesets are composable — you can pipe them through multiple functions.

Composable Queries

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
defmodule MyApp.Accounts do
  import Ecto.Query
  alias MyApp.Repo
  alias MyApp.Accounts.User

  # Base query — all active users
  defp base_query do
    from u in User,
      where: is_nil(u.locked_at),
      where: not is_nil(u.confirmed_at)
  end

  # Composable filter functions
  defp filter_by_role(query, nil), do: query
  defp filter_by_role(query, role) do
    from u in query, where: u.role == ^role
  end

  defp filter_by_search(query, nil), do: query
  defp filter_by_search(query, "") , do: query
  defp filter_by_search(query, search) do
    pattern = "%#{search}%"
    from u in query,
      where: ilike(u.name, ^pattern) or ilike(u.email, ^pattern)
  end

  defp with_post_count(query) do
    from u in query,
      left_join: p in assoc(u, :posts),
      group_by: u.id,
      select_merge: %{post_count: count(p.id)}
  end

  defp paginate(query, page, per_page) do
    offset = (page - 1) * per_page
    from u in query, limit: ^per_page, offset: ^offset
  end

  # Public API that composes the above
  def list_users(opts \\ []) do
    role = Keyword.get(opts, :role)
    search = Keyword.get(opts, :search)
    page = Keyword.get(opts, :page, 1)
    per_page = Keyword.get(opts, :per_page, 20)

    base_query()
    |> filter_by_role(role)
    |> filter_by_search(search)
    |> with_post_count()
    |> order_by([u], asc: u.name)
    |> paginate(page, per_page)
    |> Repo.all()
  end

  def get_user!(id) do
    Repo.get!(User, id)
  end

  def get_user_by_email(email) do
    Repo.get_by(User, email: String.downcase(email))
  end

  def create_user(attrs \\ %{}) do
    %User{}
    |> User.registration_changeset(attrs)
    |> Repo.insert()
  end

  def update_user(%User{} = user, attrs) do
    user
    |> User.update_changeset(attrs)
    |> Repo.update()
  end
end

This is dramatically more explicit than ActiveRecord. The query is a data structure built up by composing functions. There is no magic. You can inspect the query at any step with IO.inspect/1. The generated SQL is predictable.

Migrations

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
defmodule MyApp.Repo.Migrations.CreateUsers do
  use Ecto.Migration

  def change do
    create table(:users) do
      add :email, :string, null: false
      add :name, :string, null: false
      add :password_hash, :string, null: false
      add :role, :string, default: "user", null: false
      add :confirmed_at, :utc_datetime
      add :failed_login_attempts, :integer, default: 0, null: false
      add :locked_at, :utc_datetime

      timestamps()
    end

    create unique_index(:users, [:email])
    create index(:users, [:role])
    create index(:users, [:confirmed_at])
  end
end

# Zero-downtime migration pattern for adding columns to large tables
defmodule MyApp.Repo.Migrations.AddPreferencesToUsers do
  use Ecto.Migration

  # Disable advisory lock for large table migrations
  @disable_ddl_transaction true
  @disable_migration_lock true

  def change do
    # In PostgreSQL, adding a nullable column with no default is instant
    # (no table rewrite needed in modern Postgres)
    alter table(:users) do
      add_if_not_exists :preferences, :map, default: %{}
    end

    # Create index concurrently — doesn't lock the table
    create_if_not_exists index(:users, [:preferences],
      using: :gin,
      concurrently: true
    )
  end
end

The Repo Pattern

Ecto’s Repo is explicit about database access. There is no implicit loading of associations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# Explicit preloading — no N+1 by accident
users = Repo.all(from u in User, preload: [:posts, :profile])

# Or load associations after the fact
user = Repo.get!(User, id)
user = Repo.preload(user, [:posts, teams: :members])

# Transactions
Repo.transaction(fn ->
  {:ok, user} = create_user(attrs)
  {:ok, _profile} = create_profile(user, profile_attrs)
  user
end)

# Multi — named transaction steps
alias Ecto.Multi

Multi.new()
|> Multi.insert(:user, User.registration_changeset(%User{}, attrs))
|> Multi.insert(:profile, fn %{user: user} ->
  Profile.changeset(%Profile{user_id: user.id}, profile_attrs)
end)
|> Multi.run(:send_email, fn _repo, %{user: user} ->
  Mailer.send_welcome(user.email)
end)
|> Repo.transaction()
# Returns {:ok, %{user: user, profile: profile, send_email: email_result}}
# Or {:error, step_name, changeset_or_value, changes_so_far}

9. Distribution

The BEAM’s distributed computing support is built into the runtime. You connect two BEAM nodes and they can communicate as if they were the same node. This is not a library or framework feature — it is part of the virtual machine.

Connecting Nodes

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# Start a node with a name
# iex --sname node1 --cookie mysecretcookie
# iex --sname node2 --cookie mysecretcookie

# In node1:
Node.connect(:"node2@hostname")
Node.list()  # [:"node2@hostname"]

# Call a function on a remote node
Node.spawn(:"node2@hostname", fn ->
  IO.puts("Running on #{node()}")
end)

# Run a named GenServer on a specific node
GenServer.call({MyApp.Cache, :"node2@hostname"}, {:get, "key"})

Global Process Registration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# Register a process globally — accessible from any node in the cluster
:global.register_name(:my_global_server, self())

# Look up a globally registered process from any node
pid = :global.whereis_name(:my_global_server)
GenServer.call(pid, :some_request)

# Using :via tuple — works with GenServer natively
GenServer.start_link(MyServer, [], name: {:global, :my_global_server})
GenServer.call({:global, :my_global_server}, :some_request)

pg: Process Groups

pg (process groups, introduced in OTP 23) allows subscribing processes to named groups and broadcasting messages to all members:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Start pg scope (usually in your Application supervisor)
:pg.start_link(:my_scope)

# Join a group from any node
:pg.join(:my_scope, "user:#{user_id}", self())

# Get all processes in a group (across all nodes in cluster)
pids = :pg.get_members(:my_scope, "user:#{user_id}")

# Send to all members
Enum.each(pids, &send(&1, {:notification, message}))

# Leave a group
:pg.leave(:my_scope, "user:#{user_id}", self())

Phoenix PubSub uses pg under the hood for distributed pub/sub.

libcluster for Kubernetes

In a Kubernetes deployment, you need nodes to discover each other automatically. libcluster handles this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
# mix.exs
{:libcluster, "~> 3.3"}

# config/prod.exs
config :libcluster,
  topologies: [
    k8s: [
      strategy: Cluster.Strategy.Kubernetes.DNS,
      config: [
        service: "my-app-headless",
        application_name: "my_app",
        polling_interval: 10_000
      ]
    ]
  ]

# In Application.start/2
children = [
  {Cluster.Supervisor, [Application.get_env(:libcluster, :topologies), [name: MyApp.ClusterSupervisor]]},
  # ... rest of children
]

With a headless Kubernetes service and libcluster, new pods automatically join the cluster and are available for distributed GenServer calls, PubSub, and presence tracking.

What “Distributed by Default” Means

When your Phoenix application runs in a Kubernetes cluster with 3 replicas and libcluster:

  • Phoenix Channels subscribers on replica 1 receive messages published on replica 2 (via PubSub over distribution)
  • Presence state is consistent across all replicas via CRDT synchronization
  • You can call GenServers on specific replicas by node name
  • A process on replica 1 can monitor a process on replica 3 and receive a {:DOWN, ...} message if it crashes

This is not eventual consistency — it is synchronous distributed message passing with the same semantics as local messaging. The tradeoff is that network partitions can be catastrophic (this is not Raft or Paxos), which is why most production Elixir systems combine the BEAM’s distribution with Ecto’s database for state that must survive node failures.


10. Tooling

Mix: Build Tool and Task Runner

Mix is Elixir’s built-in build tool. Unlike Gradle or Maven, it is designed to be extended:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# Create a new project
mix new my_app
mix new my_app --sup  # with an OTP Application and Supervisor

# Phoenix project
mix phx.new my_app --database postgres
mix phx.new my_app --no-ecto --no-assets  # minimal

# Dependencies
mix deps.get         # fetch dependencies
mix deps.compile     # compile dependencies
mix deps.update --all

# Running
mix run              # run the application
iex -S mix           # start interactive shell with app loaded
mix phx.server       # start Phoenix server

# Database (Ecto)
mix ecto.create
mix ecto.migrate
mix ecto.rollback
mix ecto.gen.migration add_posts_table

# Code generation (Phoenix)
mix phx.gen.context Blog Post posts title:string body:text published:boolean
mix phx.gen.live Blog Post posts title:string body:text
mix phx.gen.json Blog Post posts title:string body:text
mix phx.gen.auth Accounts User users  # generates full auth system

# Tests
mix test
mix test test/my_app/accounts_test.exs
mix test test/my_app/accounts_test.exs:42  # specific line
mix test --stale     # only tests affected by recent changes
mix test --cover     # with coverage

# Custom tasks
mix my_app.backfill_data

Custom Mix tasks are straightforward:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
defmodule Mix.Tasks.MyApp.SeedData do
  use Mix.Task

  @shortdoc "Seed the database with initial data"

  @impl Mix.Task
  def run(_args) do
    Mix.Task.run("app.start")

    Mix.shell().info("Seeding users...")
    Enum.each(seed_users(), &MyApp.Accounts.create_user/1)

    Mix.shell().info("Seeding complete.")
  end

  defp seed_users do
    [
      %{email: "admin@example.com", name: "Admin", password: "securepassword123", role: :admin},
      %{email: "user@example.com",  name: "User",  password: "securepassword123", role: :user}
    ]
  end
end

IEx: The Interactive Shell

IEx is the Elixir REPL, and it is excellent for production debugging:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# Start with the app loaded
iex -S mix phx.server

# Useful IEx helpers
h String.split         # documentation for String.split
i "hello"              # inspect a value (type, protocol implementations, etc.)
v(3)                   # value of line 3 in history

# Debugging with dbg (Elixir 1.14+)
"hello world"
|> String.upcase()
|> dbg()               # prints each step of the pipeline with values
|> String.split()

# Recompile changed modules without restarting
recompile()

# In production, connect to a running node
# iex --remsh my_app@hostname --cookie mysecretcookie

# Check process info
Process.list() |> length()  # total process count
:observer.start()            # GUI process/memory/ETS inspector

# Inspect ETS tables
:ets.all()
:ets.info(:my_table)
:ets.tab2list(:my_table) |> Enum.take(5)

Hex: Package Manager

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# hex is built into Mix
mix hex.info jason        # package information
mix hex.search jason      # search packages

# Local development with path dependencies (mix.exs)
{:my_lib, path: "../my_lib"}

# Git dependency
{:my_lib, github: "username/my_lib", branch: "main"}

# Publish a package
mix hex.publish

ExUnit: Testing

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
defmodule MyApp.Accounts.UserTest do
  use MyApp.DataCase  # sets up database sandbox for each test

  alias MyApp.Accounts

  describe "create_user/1" do
    test "creates a user with valid attributes" do
      attrs = %{email: "test@example.com", name: "Test User", password: "securepassword123"}

      assert {:ok, user} = Accounts.create_user(attrs)
      assert user.email == "test@example.com"
      assert user.name == "Test User"
      refute is_nil(user.password_hash)
      assert is_nil(user.password)  # virtual field not persisted
    end

    test "fails with invalid email" do
      attrs = %{email: "not-an-email", name: "Test", password: "securepassword123"}

      assert {:error, changeset} = Accounts.create_user(attrs)
      assert %{email: ["must be a valid email address"]} = errors_on(changeset)
    end

    test "fails with short password" do
      attrs = %{email: "test@example.com", name: "Test", password: "short"}

      assert {:error, changeset} = Accounts.create_user(attrs)
      assert %{password: ["should be at least 12 character(s)"]} = errors_on(changeset)
    end

    test "fails with duplicate email" do
      attrs = %{email: "test@example.com", name: "Test", password: "securepassword123"}
      {:ok, _} = Accounts.create_user(attrs)

      assert {:error, changeset} = Accounts.create_user(attrs)
      assert %{email: ["has already been taken"]} = errors_on(changeset)
    end
  end

  # Doctest — examples in module docs are executed as tests
  doctest MyApp.Accounts
end

# GenServer test
defmodule MyApp.CacheTest do
  use ExUnit.Case, async: true  # runs concurrently with other async tests

  setup do
    {:ok, pid} = Cache.start_link([])
    %{cache: pid}
  end

  test "stores and retrieves values", %{cache: cache} do
    Cache.put(cache, :key, "value")
    assert {:ok, "value"} = Cache.get(cache, :key)
  end

  test "returns error for missing key", %{cache: cache} do
    assert {:error, :not_found} = Cache.get(cache, :missing)
  end

  test "respects TTL", %{cache: cache} do
    Cache.put(cache, :key, "value", 50)  # 50ms TTL
    assert {:ok, "value"} = Cache.get(cache, :key)
    Process.sleep(100)
    assert {:error, :expired} = Cache.get(cache, :key)
  end
end

Dialyxir: Type Checking with Dialyzer

Dialyzer performs success typing — it cannot prove a program is correct, but it can prove certain errors cannot occur and flag type inconsistencies:

1
2
mix dialyzer          # run type analysis
mix dialyzer --format short  # compact output
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# lib/my_app/accounts.ex
defmodule MyApp.Accounts do
  @spec get_user!(integer()) :: User.t()
  def get_user!(id) when is_integer(id) do
    Repo.get!(User, id)
  end

  @spec create_user(map()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
  def create_user(attrs) do
    %User{}
    |> User.registration_changeset(attrs)
    |> Repo.insert()
  end
end

Dialyzer is slower than Go’s type checker or Rust’s borrow checker (it analyses the entire call graph), but it catches real bugs and is essential for large codebases.

Credo: Code Style and Quality

1
2
3
mix credo             # run style checks
mix credo --strict    # stricter rules
mix credo explain     # explain each warning

Credo catches things like overly complex functions, unused variables, and consistent style issues. It is similar to RuboCop for Ruby but considerably faster.


11. Honest Assessment

Where the BEAM Dominates

Real-time applications with many concurrent connections. Chat, collaborative editing, live dashboards, presence systems, multiplayer games — anything that needs tens of thousands of simultaneous long-lived connections is exactly what the BEAM was designed for. Phoenix LiveView makes this accessible without writing complex distributed systems code by hand.

Telecommunications and IoT. The original use case. Managing many persistent connections to devices with per-device state is natural in Elixir. One process per device is idiomatic and scalable.

Stateful services with clear fault boundaries. If you are building a system where “each user/session/connection has associated state that needs to be managed over time,” the actor model maps perfectly. Process-per-user-session with automatic supervision and restart is a pattern that is difficult to implement cleanly in other runtimes.

Systems that need 99.99%+ availability. The combination of preemptive scheduling (no event loop blocking), per-process GC (no stop-the-world pauses), and OTP supervision trees makes it genuinely difficult to bring down a well-structured BEAM application. Rolling deployments with hot code upgrades are operationally simpler than they are in most runtimes.

Distributed systems where you own the cluster. When you control the nodes and can maintain network connectivity (not a public cloud with aggressive NAT), the BEAM’s built-in distribution eliminates substantial infrastructure complexity. No separate message broker, no distributed cache, no shared state server — it is all in the runtime.

Data pipelines with backpressure. GenStage and Broadway (built on GenStage) provide production-grade backpressure-aware data pipelines. Broadway integrates with SQS, Kafka, RabbitMQ, and others out of the box.

Where It Is Not the Right Tool

CPU-intensive computation. The BEAM’s scheduler is optimized for latency, not throughput. Long-running CPU-bound work blocks a scheduler thread for the duration of its reduction budget and triggers extra context switches. For number crunching, scientific computing, video encoding, or anything that needs to maximize single-core CPU throughput, Go, Rust, or C are better choices. Elixir handles this by delegating to native code via NIFs (Native Implemented Functions), but writing NIFs is complex and dangerous — a crashing NIF brings down the entire VM.

Machine learning and data science. Python’s ecosystem for ML (PyTorch, TensorFlow, scikit-learn, NumPy) is simply unmatched. Elixir’s Nx library (Numerical Elixir) and Axon are genuinely impressive and work for some use cases, but for training models or doing data science work, Python is the right tool by a significant margin.

Small scripts and CLI tools. The BEAM starts in ~500ms–1s. For quick scripts or CLI tools where startup time matters, Go (instant) or Python (fast enough) are better. Elixir 1.16’s Mix installs help here but it is still not a scripting language.

Teams without functional programming experience. The OTP mindset — process-per-entity, supervision trees, let it crash — is genuinely different from object-oriented or imperative programming. Teams moving from Java or Python typically need 2–4 months before they stop fighting the language and start leveraging it. This is a real cost that has to be justified by the problem domain.

Batch processing where throughput is the priority. If you are processing millions of records from a database as fast as possible with no concurrency requirements, Elixir is not meaningfully better than Python with multiprocessing, and may be worse because the BEAM’s scheduling overhead adds latency.

The Ecosystem and Hiring Reality

The Elixir ecosystem is smaller than Python, Go, or Node.js but it is mature and well-curated. The packages that exist tend to be high quality because the community is smaller and more experienced. Hex has around 15,000 packages (vs npm’s millions), but finding a well-maintained library for any common task is generally straightforward.

Hiring is a legitimate concern. There are far fewer Elixir engineers than Python, Go, or Java engineers. Senior Elixir engineers command premium salaries. The upside: engineers who choose Elixir tend to be experienced and self-selecting for quality. Several companies have reported lower headcount requirements for Elixir systems compared to equivalent Node.js systems, which can offset the hiring difficulty.

The OTP Learning Curve

The hardest part of Elixir is not the syntax — it is developing intuition for OTP design. Questions that take time to answer:

  • When do I use a GenServer versus just passing data through function calls?
  • When do I use ETS versus a GenServer for shared state?
  • How granular should my supervision tree be?
  • When should a process crash vs handle the error?
  • How do I test processes without introducing flakiness?

These questions have good answers, but they require internalizing a different model of computation. The payoff is real: once you have the OTP mindset, building fault-tolerant concurrent systems becomes straightforward. But the journey takes months, not days.


Getting Started

Installation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# macOS with Homebrew
brew install elixir

# Ubuntu/Debian via asdf (recommended for version management)
asdf plugin add erlang
asdf plugin add elixir
asdf install erlang 26.2.5
asdf install elixir 1.17.3-otp-26
asdf global erlang 26.2.5
asdf global elixir 1.17.3-otp-26

# Verify
elixir --version
# Erlang/OTP 26 [erts-14.2.5] ...
# Elixir 1.17.3 (compiled with Erlang/OTP 26)

Your First Phoenix App

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Install the phx_new archive
mix archive.install hex phx_new

# Create app with Postgres and LiveView (default)
mix phx.new my_app

cd my_app

# Create the database
mix ecto.create

# Start the server
mix phx.server
# Visit http://localhost:4000

Resources

  • Programming Elixir 1.6 by Dave Thomas — best introduction to the language
  • Programming Phoenix LiveView by Bruce Tate and Sophie DeBenedetto — authoritative LiveView guide
  • Designing Elixir Systems with OTP by James Edward Gray II — the OTP mindset book
  • The Little Elixir & OTP Guidebook by Benjamin Tan Wei Hao — concise OTP intro
  • elixirforum.com — active, friendly community
  • hexdocs.pm — comprehensive documentation for all Hex packages
  • hex.pm — package registry

Conclusion

The BEAM is not a trendy technology. It is 40 years of accumulated wisdom about building systems that stay up. The preemptive scheduler, per-process garbage collection, built-in distribution, and OTP’s supervision trees are not features added for marketing — they solve real problems that every engineer building distributed systems eventually encounters.

Elixir makes the BEAM accessible without sacrificing what makes it powerful. The syntax is approachable, the tooling is excellent, and the ecosystem has reached a maturity where you can build production systems without writing everything from scratch. Phoenix LiveView in particular represents something genuinely new: real-time interactive applications without the operational complexity of a separate JavaScript frontend and WebSocket server.

Is Elixir right for your next project? If you are building something that needs to handle many concurrent connections with low latency, that needs to stay up under partial failure, and that involves coordination or communication between users or devices — yes, seriously consider it. If you are building a batch processor, a data science pipeline, or a simple CRUD API for a small team that knows Python, probably not.

The engineers who invest in the BEAM tend to stay. The combination of a runtime that handles failure gracefully, a language that makes concurrent programming tractable, and a community that values correctness over novelty creates something rare: a technology stack that is both principled and practical.

The nine nines are not a guarantee. They are a target. The BEAM gives you the tools to get there.

Comments