Step 1: Connect to the Storage Node
Connecting is not enough to call the service: Malla.Node must first learn that StorageService runs on the other node. Discovery starts as soon as the nodes connect and takes a moment, so this cell waits for it (up to 10 seconds).
storage_node = :"storage@127.0.0.1"
wait_for_service = fn service ->
Enum.find_value(1..50, fn _ ->
case Malla.Node.get_nodes(service) do
[] ->
Process.sleep(200)
nil
nodes ->
nodes
end
end)
end
case Node.connect(storage_node) do
true ->
IO.puts("Connected to #{storage_node}")
case wait_for_service.(StorageService) do
nil -> IO.puts("StorageService not found: is it running in the Storage notebook?")
nodes -> IO.puts("StorageService found on #{inspect(nodes)}")
end
false ->
IO.puts("Failed to connect to #{storage_node}")
IO.puts("Make sure the Storage node IEx session is running")
endIf you call a service before it is discovered, the call is not lost: Malla.remote/4 logs a service_not_available warning, waits and retries (5 times, 1 second apart, by default).
Step 2: Make Remote Calls
Call the storage service running on the other node.
import Malla, only: [call: 1]
# Store data on the remote node
call StorageService.put(:from_client, %{message: "Hello from client node!"})
call StorageService.put(:timestamp, DateTime.utc_now())
IO.puts("Data stored on remote node")
# Retrieve data from the remote node
{:ok, data} = call StorageService.get(:from_client)
IO.puts("Retrieved from remote: #{inspect(data)}")Step 3: Understanding Remote Calls
The call macro is syntactic sugar for Malla.remote/4:
# These are equivalent:
# call StorageService.get(:user_1)
# Malla.remote(StorageService, :get, [:user_1])
# Let's see where the service is actually running
nodes = Malla.Node.get_nodes(StorageService)
IO.puts("StorageService is available on: #{inspect(nodes)}")
# Make an explicit remote call with options
result =
Malla.remote(
StorageService,
:get,
[:user_1],
timeout: 5000
)
IO.puts("Remote call result: #{inspect(result)}")Part 4: New service
First, run the "Add a Log Service" section in the Storage notebook. Service will be discovered immediately by this node. Then run this:
import Malla, only: [call: 1]
# Use the LogService running on the storage node
call LogService.log("Hello from client node!")
call LogService.log("Testing distributed logging")
# Retrieve logs
logs = call LogService.get_recent_logs(10)
IO.puts("\nRecent logs:")
Enum.each(logs, &IO.puts/1)Part 5: Virtual Modules
When Malla discovers a remote service, it creates a virtual module — a local proxy that forwards calls to the remote node automatically. This means you can call remote services directly, without using the call macro:
# No import, no `call` macro — just a regular function call.
# Malla's virtual module routes it to the storage node transparently.
logs = LogService.get_recent_logs(10)
IO.puts("Logs via virtual module:")
Enum.each(logs, &IO.puts/1)Virtual modules only have the callbacks (defcb) of the service: that is why LogService defines its functions with defcb. A plain def of the service, like StorageService.put/2, is not part of the virtual module: call it with call or Malla.remote/4. A virtual call also cannot take options such as :timeout. See the Remote Calls guide for these limits.
Summary
IO.puts("""
Congratulations! You've successfully:
1. Set up a distributed Erlang cluster with two nodes
2. Created a service on one node (StorageService)
3. Made remote calls from another node
4. Saw a service started later (LogService) found by the other node
5. Called a service through a virtual module
Key Concepts:
- global: true
Makes services discoverable across the cluster via :pg process groups
- Malla.call / Malla.remote
Route function calls to a node that runs the service
- Service Discovery
Malla.Node learns of a service when it changes status, and re-checks
every 10 seconds. The local node, if it runs the service, comes first;
the remote nodes are in a random order that is reshuffled at each check.
- Virtual Modules
When a remote service is detected and this node has no module of that
name, Malla creates a local proxy module with the service's callbacks
- Failover
A call goes to the next node only when a node answers that the
service is not available. Exceptions and timeouts are returned to the caller.
Next steps: start a third node that also runs StorageService, then look at
Malla.Node.get_nodes(StorageService), or use Malla.Node.call_cb_all/4 to
call every node of a service.
""")Troubleshooting
Connection Problems
IO.puts("This node: #{node()}")
IO.puts("Connected nodes: #{inspect(Node.list())}")
IO.puts("Cookie: #{Node.get_cookie()}")Service Discovery Issues
Process.sleep(11_000)
nodes_with_storage = Malla.Node.get_nodes(StorageService)
IO.puts("StorageService found on: #{inspect(nodes_with_storage)}")
members = :pg.get_members(Malla.Services2, StorageService)
IO.puts("Process group members: #{inspect(members)}")