Skip to content

5. Channel Messaging — Request · Send · Pub/Sub

Guide Home | Previous: 4. Backpressure — When Arrival Outpaces Processing | Next: 6. Spot

View in another language — C#/.NET · C++ · Java · Kotlin · Node/TypeScript

The document that owns this chapter's contract — Channel Messaging and ClientServer Channel own the behavior, and the per-language channel messaging public contract owns the surface. This chapter covers how to actually register and call that surface, focused on usage.

Channel messaging is the framework's most fundamental axis. It covers these interactions.

  • request/response — a 1:1 call that sends and waits for a response, e.g. a price lookup (DEALER → ROUTER)
  • one-way send — a fire-and-forget one-way command, e.g. a cache-invalidation notification (DEALER → ROUTER)
  • publish/subscribe — an event fan-out where every subscriber receives the message, e.g. propagating a domain event (PUB / SUB)

🔰 If terms like channel/handler/client/codec are unfamiliar, read the concept explanations in 03-concepts first. DEALER → ROUTER and PUB / SUB in parentheses are the underlying socket kinds — the application never handles these directly (the framework auto-maps them by channel kind).

↗ View larger

Terminators — The Last Piece That Actually Sends a Call

Every outbound call in this chapter starts with a builder and ends with exactly one terminator. The part that picks the target channel and the message only assembles the call; nothing has been sent yet. The runtime accepts the call at the moment you invoke the terminator. A builder left without one does nothing at all.

There are two kinds, and "finished" means something different for each.

Kind Used by What finishing means Result
One-way terminator send · publish Your own runtime accepted the submission None — it says nothing about whether the peer received it or the handler ran
Request terminator request The peer's reply arrived The reply payload. Finishing without a reply fails as a timeout or a route error

The reply type comes from the request terminator, not from the message you send. The same request payload can be answered with different reply types in different places, so the reply type is not tied to the request message.

// one-way — only until the submission is accepted. No return value.
await client.SendToChannel("profile", command).Async(ct);
await publisher.Publish("api.events", "profile.cache-refreshed", evt).Async(ct);

// request — until the reply arrives. The terminator names the reply type.
var reply = await client.RequestToChannel("price", query).Async<PriceReply>(ct);

A request terminator can be preceded by an optional terminator that changes how long the reply is awaited. The one-way kind never waits for a reply, so that surface does not exist there — §4 Outbound Calls covers its use.

Why even a one-way terminator is awaited — what happens when there is no room to send is owned by 04-backpressure §3.1. The exact terminator names, overloads, and exceptions are owned by the per-language public contract.

0. Use as a gRPC Replacement

Channel messaging is used in general web/microservice backends to replace gRPC between services. Instead of every service announcing a host:port or putting a gateway/load balancer in front, it ties calls together with a logical channel name and location-store auto-connect. Without a .proto IDL, HTTP/2-only infrastructure, or code generation, you get gRPC's four call shapes with just DTOs (records) and typed handlers.

gRPC pattern ZLink replacement This guide
Unary RPC request/response Writing a Handler · Outbound Calls
Unary Empty / fire-and-forget one-way send Writing a Handler · Outbound Calls
Server streaming / event feed pub/sub fan-out Outbound Calls
Client/Bidi streaming STREAM session 09-stream
Service location lookup (DNS/xDS) location-store auto-connect 10-location
Interceptor handler filter Filter — Common Processing
Deadline request timeout Outbound Calls

The call path diverges at this point: in gRPC, an L7 load balancer or service mesh sidecar takes a request created by a stub and sends it to one of the scaled-out servers. In ZLink, when the application calls by logical channel name, the framework runtime directly picks one of the connected server runtimes. So what's left in application code isn't endpoint or proxy configuration, but a channel name and a handler.

For example, for an order service, gRPC's rpc PlaceOrder(...) turns into this.

// Server: one handler (instead of a gRPC service implementation)
public sealed class PlaceOrderHandler
    : IZLinkRequestHandler<PlaceOrder, OrderPlaced>
{
    private readonly IOrderStore _orders;
    public PlaceOrderHandler(IOrderStore orders) => _orders = orders;

    public async ValueTask<OrderPlaced> HandleAsync(
        PlaceOrder request, IZLinkMessageContext context, CancellationToken ct)
    {
        await _orders.SaveAsync(request, ct);
        return new OrderPlaced(request.OrderId);
    }
}

// Client: inject IZLinkRouteClient instead of a gRPC stub
var placed = await client
    // The target is just one ChannelName. No address, no MeshName.
    .RequestToChannel("orders",
        new PlaceOrder("order-1042", "acct-77", 18742))
    .Async<OrderPlaced>(ct);

To compare deployment structure, call path, and infrastructure mapping side-by-side with a gRPC stack, 17-alternative covers that comparison. This chapter covers usage after that decision is already made.

1. Channel Kinds

A channel is a unit of connection between servers that picks a call target by a logical name like "orders" instead of an address. That name is called ChannelName, and one of the nodes that registered that name receives the request.

Below are the registrations that use the name "channel." All of them use ChannelName, but they differ in which messaging pattern they support and whether they share a socket. Here, a MeshNode is the basic unit of server-to-server connection that one process has, and a route mesh channel just adds a name on top of that socket.

Kind Registration Socket Connection pattern
Route mesh channel mesh.Channel(name).Server()/.Client() Shares an already-open MeshNode socket request/send via ChannelName select-one, publish between Spots (Logical Multicast) — Node direct, which specifies an RID directly, is separate (Calling a Managed Node Directly)
ClientServer channel AddClientServerChannel(name) Opens its own socket, separate from the MeshNode (.Listen(); connection is manual .Connect() or auto-discovery) Only request/send started by the Client — the Server can't send anything first except that reply
Fanout channel AddFanoutChannel(name) Opens an independent PUB/SUB socket publisher → many subscribers

A route mesh channel is a logical name that shares a MeshNode connection, and a ClientServer channel is an independent connection unit that opens its own transport.

1.1 Route Mesh Channel — One Connection, Channels Are Names on Top

You connect to the mesh with one MeshNode socket, and the channel name is the logical grouping on top of it that decides "who receives this request."

↗ View larger

The boxes are groups tied together by name, not sockets. Calling orders has select-one pick one of A1/A2 inside that box, and registering ten more channels doesn't add any sockets on node B.

1.2 ClientServer Channel — An Independent Runtime per Channel

A ClientServer channel doesn't share RouteMesh transport. Each channel gets its own independent runtime, and that runtime manages connections per Ready Server.

The client starts the connection. Since the server never connects out to the client, firewalls and security groups only need to open in one direction — client → server.

Registration info is also separate. ClientServer server registration info doesn't carry MeshName, RouteMesh membership, or Spot/Actor location. Conversely, MeshNode registration info isn't used for ClientServer discovery either — the two kinds never substitute for each other.

Using only manual endpoints means you don't need a location store. If auto-discovery is enabled and there's no store, startup fails before the listener binds.

↗ View larger

auth and report don't share connection targets or lifetimes. Even if the same process Z participates in both channels, each channel runtime manages its connection to Z separately.

A Server inside the same process is just as much a candidate as any other Server. It's neither picked first nor excluded for being in the same process. Even when selected, the handler isn't called directly — it's actually sent through the connection. There's no shortcut that skips codec, admission, caps, timeout, or reply handling. So don't assume a local call is faster.

Direction is also fixed, so a Server can only respond to a request the Client started. If the Server needs to send a notification first, use RouteMesh instead of ClientServer. This is why TicTacToe separates login authentication (the tictactoe.api ClientServer channel) from Game Spot creation (the MeshNode's Object role) (chapter 02. Getting Started §7).

1.3 Two Branches of Pub/Sub

A Spot is a state object found by id that lines up work addressed to it and processes it one by one. Exchanging events between Spots over a route mesh channel is called Logical Multicast. As in the earlier diagram, it reuses the already-connected mesh socket as-is, so there's no separate socket, and the receiving side is limited to Spots that subscribed to the same topic on that channel.

// Publishing -- inside the TicTacToeGame spot.
await Context.Outbound
    // The ChannelName that decides delivery scope.
    .Publish(SampleTopics.PlayerMilestoneChannel,
             // The topic that picks which Spots receive it within that scope.
             SampleTopics.PlayerMilestone,
             milestoneEvent)
    .Async(cancellationToken);

// Subscribing -- when PlayEntrySpot starts.
Context.Handlers.AddSubscribe<PlayerWinMilestoneEventHandler>(
    // Must match the publishing side's ChannelName/topic to receive it.
    SampleTopics.PlayerMilestoneChannel,
    SampleTopics.PlayerMilestone);

If you need to publish from outside a Spot, inject a spot publisher client and send the same way.

Conversely, a fanout channel (called Classic fanout in the spec) opens an independent pair of PUB/SUB sockets by itself. Regardless of Spot or MeshNode, one publisher delivers to every connected subscriber.

↗ View larger

For both, publish completing doesn't guarantee delivery. A publish call completing means the send was locally accepted for transport, not confirmation that a subscriber processed the event. Neither provides storage, retransmission, or ack.

The difference is target scope. Logical Multicast is limited to Spots that subscribed to the same channel/topic within that mesh, while Classic fanout delivers to every connected subscriber regardless of mesh composition.

The loss rule also differs. A fanout channel provides loss-tolerant delivery. If one subscriber falls behind and the publisher's send queue hits its cap, that subscriber's share is discarded and the publish still succeeds. Other subscribers aren't affected, and the publisher doesn't stall over one slow subscriber.

Logical Multicast doesn't use a PUB/SUB socket — it delivers to each node over the mesh connection, so this rule doesn't apply to it. Don't use a fanout channel for delivery that can't tolerate loss.

2. Writing a Handler

Kind Call What completion means
request Sends a request to a channel name and waits for the reply The peer's reply arrived
send Sends a message to a channel name The send was accepted -- not the peer's processing result
publish (fanout) Sends an event to a channel and topic The send was accepted for transport -- not subscriber receipt

Even if a request fails, it isn't auto-resent to a different server. If the connection drops or times out after picking and sending to a target, it just ends as a failure. This is because the first target may have already processed it and only the reply failed to come back. Resending is a new call from the application, and handling duplicate execution is its responsibility too.

The payload's content doesn't change the target kind. A message addressed to a node is handled by that node's handler — even if it carries a Spot ID or Actor ID inside, the Framework doesn't look inside and turn it into a Spot message. To send to a Spot or Actor, use that dedicated call from the start.

The call shape and matching handler interface per language are as follows.

Kind Call The handler that receives it
request RequestToChannel(name, req).Async<TReply>(ct) IZLinkRequestHandler<TRequest, TReply>
send SendToChannel(name, msg).Async(ct) IZLinkSendHandler<TMessage>
publish (fanout) Publish(name, topic, evt).Async(ct) IZLinkFanoutHandler<TEvent>

A channel handler is an independent class. Since different requests can run concurrently, don't put mutable domain state in a handler member. The handler instance and its scoped dependencies only live until that dispatch finishes.

The Framework doesn't process HTTP requests. A web framework's endpoints/middleware handle HTTP, and a channel handler is a separate server-to-server message dispatch path. The only similarity to a controller action is the authoring style — write a class, receive dependencies via DI, register it, and the runtime calls it.

A handler implements an interface and returns its result directly.

See it in a sample — TicTacToe. This is the request handler in which the API server receives an authentication request and returns player info. These snippets are actual code from the repository.

internal sealed class AuthenticatePlayerHandler(ILogger<AuthenticatePlayerHandler> logger)
    : IZLinkRequestHandler<AuthenticatePlayerReq, AuthenticatePlayerRes>
{
    public ValueTask<AuthenticatePlayerRes> HandleAsync(
        AuthenticatePlayerReq request,
        IZLinkMessageContext context,
        CancellationToken cancellationToken)
    {

        var actorId = request.AccessToken.Trim();
        if (string.IsNullOrWhiteSpace(actorId)) throw new InvalidOperationException("Authentication token is empty.");

        var player = CreatePlayer(actorId);

        logger.LogInformation(
            "play -> api: authenticate accepted. player={ActorId}, level={Level}, wins={Wins}",
            player.ActorId,
            player.Level,
            player.Wins);
        return ValueTask.FromResult(new AuthenticatePlayerRes(player));
    }

    private static PlayerInfo CreatePlayer(string actorId)
    {
        return actorId switch
        {
            "player-x" => new PlayerInfo(actorId, "Player X", 5, 99),
            "player-o" => new PlayerInfo(actorId, "Player O", 4, 12),
            "observer" => new PlayerInfo(actorId, "Observer", 1, 0),
            _ => new PlayerInfo(actorId, actorId, 3, 0)
        };
    }
}

The three branches in minimal form look like this.

// request-response
public sealed class GetProfileHandler
    : IZLinkRequestHandler<GetProfileRequest, GetProfileReply>
{
    private readonly IProfileStore _store;
    public GetProfileHandler(IProfileStore store) => _store = store;

    public async ValueTask<GetProfileReply> HandleAsync(
        GetProfileRequest request,
        IZLinkMessageContext context,
        CancellationToken cancellationToken)
    {
        var profile = await _store.LoadAsync(request.AccountId, cancellationToken);
        return new GetProfileReply(profile.AccountId, profile.Nickname);
    }
}

// one-way send (no response)
public sealed class RefreshCacheHandler
    : IZLinkSendHandler<RefreshCacheCommand>
{
    public ValueTask HandleAsync(
        RefreshCacheCommand message,
        IZLinkMessageContext context,
        CancellationToken cancellationToken)
    {
        // Cache invalidation, etc. The caller doesn't wait for a result.
        return ValueTask.CompletedTask;
    }
}

// Receiving a publish (subscriber side)
public sealed class CacheRefreshedEventHandler
    : IZLinkFanoutHandler<CacheRefreshedEvent>
{
    public ValueTask HandleAsync(
        CacheRefreshedEvent message,
        CancellationToken cancellationToken)
    {
        // A Classic fanout handler only receives the payload of the registered event type.
        return ValueTask.CompletedTask;
    }
}
  • A handler's dependencies come via constructor injection (like IProfileStore). No service-locator pattern for pulling a service from context.
  • Context is where you read that dispatch's message info (ChannelName, packet name, metadata, etc.). Cancellation is owned by a separate cancellation argument, not context. Per-path context types and the full field list are covered by the per-language channel messaging public contract.
  • A handler class is a code organization unit, not a dispatch key. Grouping methods topically in one class, or giving each packet its own class, both work the same.
  • An interface-based handler has the strongest compile-time type checking. If HandleAsync(...)'s payload, context, or return type doesn't match the interface contract, it fails to compile.

Attribute-Based Method Handlers

Instead of an interface, you can write the same handler as a method with an attribute. This is convenient when one class holds several handler methods.

// Groups this class's methods as the "api" group. Registration decides which channel exposes it.
[ZLinkHandlerGroup("api")]
public sealed class UserHandlers
{
    private readonly IZLinkFanoutClient _publisher;
    public UserHandlers(IZLinkFanoutClient publisher) => _publisher = publisher;

    // The method attribute decides the handler kind (doesn't take a channel name)
    [ZLinkRequest]
    public ValueTask<GetUserReply> GetUserAsync(
        // Argument order = (payload, context?, ct?) -- context/token can be omitted
        GetUserRequest request,
        IZLinkMessageContext context,
        CancellationToken cancellationToken)
        => ValueTask.FromResult(new GetUserReply(request.AccountId, "alice"));

    // A send handler -- returns ValueTask (no response). Contrast with request's ValueTask<TReply>.
    [ZLinkSend]
    public async ValueTask RefreshCacheAsync(
        RefreshUserCacheCommand command,
        IZLinkMessageContext context,
        CancellationToken cancellationToken)
    {
        await _publisher
            .Publish("api.events", "user.cache-refreshed",
                new UserCacheRefreshedEvent(command.AccountId))
            .Async(cancellationToken);
    }
}
  • The method signature order is (payload, context?, cancellation?), and context/token can be omitted.
  • An attribute-based handler makes it easy to group several request/send/publish methods in one class, but it doesn't lock down the handler contract at compile time as strongly as the interface-based approach. A wrong context type or return type may only surface at the framework's scan/validation step, or at runtime.
  • The attribute/annotation/decorator marking a handler kind doesn't take a channel name. Channel mapping is owned by registration.

Asynchronous Execution

Async values across the Framework are expressed as each language's standard async type. Send waits until the source runtime can submit the work, but doesn't wait for the target handler to complete. Request waits until the peer's reply arrives. There's one rule -- await on the runtime (handler) thread, blocking (.Result/.GetAwaiter().GetResult()) only in test/client scenarios.

public async ValueTask<CreateGameReply> HandleAsync(
    CreateGameRequest request, IZLinkMessageContext context, CancellationToken ct)
{
    // Runtime (handler) thread -- free it with await. Blocking (.Result/.GetAwaiter().GetResult()) is forbidden.
    var room = await _client
        .RequestToChannel("tictactoe.play", new CreateRoomRequest(request.GameName))
        // The cap on waiting for the reply.
        .Timeout(TimeSpan.FromSeconds(5))
        // Awaits until the reply arrives and receives it.
        .Async<CreateRoomReply>(ct);

    return new CreateGameReply(room.RoomId, room.GameName);
}

A channel handler runs in a per-channel asynchronous receive loop. When a handler reaches a wait point, only that flow of execution pauses -- the thread returns to the pool to handle other work.

↗ View larger

So without callbacks, code that reads top to bottom lets a handful of workers handle huge numbers of concurrent requests. Blocking with .Result keeps that thread occupied, so it's forbidden inside a handler. Failures surface as exceptions on the await path.

The same await rule applies to Spot/Actor handlers too, but there the turn is held until the handler completes, so the concurrency scope differs -- covered in 06-spot §2.1.

3. Exposing a Handler on a Channel

The framework doesn't automatically open every discovered handler on every channel. Discovery and exposure are separate steps.

AddHandlersFromAssemblyOf<...> discovers handler types, and the typed registration under Channel(name).Server() determines which channel on which MeshNode exposes it.

RouteMesh and Handler Registration

builder.Services.AddZLinkFramework(options =>
{
    // Discovers handler types
    options.AddHandlersFromAssemblyOf<Program>();
    var mesh = options.AddRouteMesh("services")
        .Listen("tcp://0.0.0.0:7101")
        .SetRoutingId(RoutingId.From("api-1"));
    // Server() is the role that receives handlers.
    mesh.Channel("api").Server()
        .AddRequestHandler<GetProfileHandler, GetProfileRequest, GetProfileReply>()
        .AddSendHandler<RefreshCacheHandler, RefreshCacheCommand>();
});

Registering Multiple Channels on One MeshNode

You can stack several channels on the same MeshNode, and each channel can have a different role.

var mesh = options.AddRouteMesh("services")
    .Listen("tcp://0.0.0.0:7101")
    .SetRoutingId(RoutingId.From("api-1"));

// A channel this node handles.
mesh.Channel("api").Server()
    // The handler already fixes the payload/reply types.
    .AddRequestHandler<GetProfileHandler>();
// A call-only channel is Client -- no handler registered.
mesh.Channel("billing").Client();

A fanout channel's subscription handler is registered with the fanout builder's AddHandler<...>().

See it in a sample — ZoneWorld. All three kinds appear in a single registration block. Control reports are received over a route mesh channel, and cluster-wide announcements are received over a fanout channel.

mesh.Channel(ZoneWorldNames.ZoneChannel).Server();    // A channel this node handles.
mesh.Channel(ZoneWorldNames.ReportChannel).Client();  // A channel used only to send reports.

options.AddFanoutChannel(ZoneWorldNames.BroadcastChannel)
    .Connect(shared.BroadcastEndpoint)      // Connects to the publisher endpoint.
    .AddHandler<WorldAnnounceSubscriber, WorldAnnounceEvent>()
    .AddHandler<NodeMaintenanceChangedSubscriber, NodeMaintenanceChangedEvent>();

Packet-name resolution order: (1) the packetName argument in handler registration → (2) a packet-name marker attached to the payload type → (3) if neither exists, the type name. The packet name is fixed once at registration -- there's no surface to respecify it per call.

Startup-Phase Validation of Registration Errors

The following are blocked as configuration errors immediately at host startup, not deferred until the first call.

  • Registering the same MeshName twice in the same process, or a MeshNode with no ChannelName registered at all.
  • Duplicate registration under the same key (MeshName + ChannelName + message kind + packet name) -- the same packet name can be reused across different MeshNames or ChannelNames.
  • A missing local endpoint or peer connection info.
  • A disallowed handler return type.

A fanout handler is registered on its own independent fanout channel builder and isn't mixed with RouteMesh handlers.

4. Outbound Calls

Request / Send -- Route Client

public sealed class PriceService(IZLinkRouteClient client)
{
    public async Task<decimal> GetAsync(string symbol, CancellationToken ct)
    {
        var reply = await client
            // The target is just one ChannelName.
            .RequestToChannel("price", new PriceRequest(symbol))
            // request: the reply type is specified on .Async<T>, not the payload
            .Async<PriceReply>(ct);
        return reply.Price;
    }

    public async ValueTask RefreshAsync(string accountId, CancellationToken ct)
        => await client
            .SendToChannel("profile", new RefreshCacheCommand(accountId))
            // send: only waits until my runtime accepts the submission
            .Async(ct);
}
  • The reply type is specified in .Async<TReply>(...), not the message.
  • Timeout(...) is a request-only optional terminal. The default reply-wait time is a global 30 seconds, and you attach it only when it needs to differ from that default (see the priority order in the example comments below). Send/Publish don't wait for a response, so they have no timeout surface at all.
  • The packet name can't be changed at call time. It's fixed once at registration.
  • A route client uses the RouteMesh registered at startup. It fails as a configuration error if the MeshName or ChannelName isn't registered.
  • The reason send is async is that it waits for an available send slot, not for a response. If the receiving side is backed up, it waits until the send queue drains before submitting, and if a slot never opens up, it ends with DeadlineExceeded. How this flow control (backpressure) works and what options affect it are covered by 04-backpressure.

Attach a terminal only when it needs to differ from the default.

await client
    .RequestToChannel("price", new PriceRequest(symbol))
    // Specify only when this call's reply-wait cap should differ from the default (30s).
    .Timeout(TimeSpan.FromSeconds(5))
    .Async<PriceReply>(ct);
// Order that decides the reply-wait cap (earlier wins):
//   1) Per-call .Timeout(...)
//   2) The MeshNode builder's SetDefaultRequestTimeout(...)
//   3) The global options.DefaultRequestTimeout (30 seconds by default)

Publish -- Fanout Client

public sealed class ProfileService(IZLinkFanoutClient publisher)
{
    public async ValueTask AnnounceAsync(string accountId, CancellationToken ct)
        => await publisher
            // Arguments = (channel, topic, message). The topic ("profile.cache-refreshed") is the fan-out routing key.
            .Publish("api.events", "profile.cache-refreshed",
                new ProfileCacheRefreshedEvent(accountId))
            .Async(ct);
}
  • The topic is optional. Sending with Publish(channelName, message) reaches every subscriber of that channel; Publish(channelName, topic, message) carries the topic along as a classification label.
  • A subscriber connects to the publisher endpoint with AddFanoutChannel(name).Connect(endpoint).
  • A Classic fanout handler only receives the registered typed event and a cancellation signal -- it doesn't expose the transport topic in the handler context. If you need business branching, split it by event type or by registered handler.
  • Completion of Async(...)/Async<T>(...) only guarantees delegation to transport -- it doesn't guarantee the remote handler completed or that a subscriber received it (see Two Branches of Pub/Sub).
  • Pub/sub has no replay. A message published before a subscriber connects, or one that passed while disconnected, doesn't arrive even after reconnecting. Fill that gap for events you can't afford to miss with a separate resync (e.g., a one-time request for current state after reconnecting).

See it in a sample — DeliveryDispatch. An order taken over HTTP is delegated to the dispatch server via a channel call, and delivery-status changes are propagated to control and customer-push subscribers via fanout publish. It is a representative example in which the request/send/publish surfaces are all used together in a single business flow.

5. Filter — Common Processing

A web framework's HTTP middleware is exclusive to the HTTP pipeline and doesn't apply to ZLink handlers. A handler filter gathers code repeated across many handlers -- logging, validation, permission checks, metrics -- in one place.

public sealed class AuditFilter(ILogger<AuditFilter> logger)
    : IZLinkHandlerFilter
{
    public async ValueTask InvokeAsync(
        // This dispatch's message info + which path it came through.
        IZLinkHandlerFilterContext context,
        // A no-argument delegate -- runs the next filter or handler.
        ZLinkHandlerFilterNext next,
        CancellationToken cancellationToken)
    {
        // Audit-logs only ops commands and lets regular business requests pass through.
        if (context.DispatchKind == ZLinkHandlerDispatchKind.NodeDirectRequest)
            logger.LogInformation("ops {Packet} on {Mesh}", context.PacketName, context.MeshName);

        // If not called, the handler doesn't run.
        await next();
    }
}

builder.Services.AddZLinkFramework(options =>
{
    // Registration order is execution order.
    options.UseFilter<AuditFilter>();
    options.UseFilter<ValidationFilter>();
});

Scope of Application

A filter applies to messages a node receives. It doesn't apply to handlers owned by a long-lived object like a Spot or Actor -- those use their own execution order and lifetime, and if you need common processing, do it inside that handler.

Dispatch Filter
Channel send/request (both route mesh channel and ClientServer channel) Runs
Fanout subscription handler Runs
Node direct route handler (Calling a Managed Node Directly) Runs
Spot handler, Actor handler Doesn't run
A Logical Multicast subscription a Spot registers Doesn't run
STREAM session handler Doesn't run

To handle it differently per path, check context.DispatchKind. ChannelSend/ ChannelRequest cover both route mesh channel and ClientServer channel, so if you need to tell them apart, also check context.MeshName -- route mesh channel and Node direct provide MeshName, but ClientServer channel and fanout don't.

Execution Order and Short-Circuiting

Execution passes through filters in registration order, then exits in reverse order once next finishes.

AuditFilter, before
  -> ValidationFilter, before
       -> handler
     ValidationFilter, after
AuditFilter, after

Each filter calls next at most once. If it isn't called, the handler doesn't run and that dispatch ends -- but what the caller sees as the result differs by path.

Dispatch What the caller sees
send Only that dispatch ends. The sender already received the send-accepted result, so nothing changes for them
request Receives a Rejected error reply. null never goes out as a normal response just because there's no value
Fanout subscription Only that one handler ends -- other subscription handlers that received the same event still run. Nothing is delivered to the publisher

There's no way for a filter to construct and return a response value directly. To block a request, don't call next; to change the response content, handle it in the handler. Calling next twice doesn't re-run the handler -- it's rejected as an error, classified as a code mistake.

Instances and Dependencies

A new scope opens for every dispatch that runs one handler. Filters and the handler are each created once within that scope and share the same Scoped service instance. The handler sees any value obtained by a filter unchanged, so you can carry per-request state through a scoped service. This rule doesn't change regardless of what lifetime the filter type is registered with in DI -- it's still created by DI, not new.

For fanout, a dispatch is created not per event but per matching subscription handler. So a filter runs that many times too, and a heavy filter's cost grows with the number of subscribers.

6. Connection Control

A manual connection is set on the MeshNode's peer list.

var mesh = options.AddRouteMesh("services")
    .Listen("tcp://0.0.0.0:7102")
    .SetRoutingId(RoutingId.From("profile-client-1"));
mesh.Channel("profile").Client();
mesh.PeerConnections.Connect("tcp://10.0.10.15:7101");
mesh.PeerConnections.Connect("tcp://10.0.10.16:7101");

The endpoint argument is a startup setting. It's not a handle for directly controlling a running socket after the host starts. The one exception: availability (drain/restore) can be changed at runtime -- see below.

In auto-connect mode, the location store owns the peer list. When a server restarts on a new endpoint, the store's descriptor row updates and client connections follow along, so no separate action is needed. A manual connection only takes effect after you change the config and restart the application.

Finding a target in the store doesn't mean the message is sent right away. A client gets the endpoint from registration info, then reconfirms identity and execution generation over the real connection before using that target. Manual connections go through the same confirmation. So a call can still end as target-not-found even though a row exists in the store -- at that point, check whether the connection was actually established, not the store.

Restarting a server changes its execution generation. Even with the same endpoint, a connection from a previous generation isn't used as the new target -- the client prepares the new generation first, then tears down the old connection. Generation values aren't ordered by numeric size.

A late-arriving reply becomes the result if the original request is still waiting -- even if it came from a previous generation. Conversely, if that request is already gone due to timeout, cancellation, or a client restart, it's discarded, and it's never used as the result of a different, later-started request.

A stopped store doesn't affect already-established connections or requests already received. During an outage, only target-list additions and removals stop being computed. However, if the server side fails to renew its authority and exceeds the grace period, it stops accepting new business messages. Once the store recovers, the list is realigned against the latest registration info.

Operational Drain / Restore (Runtime)

Right before maintenance, a rolling restart, or scale-in, sometimes you want to stop receiving new requests only, without shutting down the node or removing the store's descriptor row. Inject the RouteMesh runtime options and change weight by MeshName and ChannelName.

The Weight used here isn't a drain-only flag -- it's the peer weight that ChannelName membership consults when picking where to send a new message. If all connected servers' weights are equal, new requests are distributed evenly by round-robin. If weights differ, the server with the larger value is picked proportionally more often. 0 means "keep the connection but exclude it from new-request candidates," and 100 is the default, normal serving value.

↗ View larger

// An operational admin endpoint. "orders" is the registered ChannelName.
app.MapPost("/admin/channels/orders/drain",
    (IZLinkRouteMeshRuntimeOptions options) =>
    {
        // Excludes this ChannelName from new select-one targets
        options.Channel("orders").Weight = 0;
        return Results.Ok();
    });

app.MapPost("/admin/channels/orders/restore",
    (IZLinkRouteMeshRuntimeOptions options) =>
    {
        // Back to normal
        options.Channel("orders").Weight = 100;
        return Results.Ok();
    });
  • Weight = 0 (drain) doesn't close the serving socket. In-flight requests that already arrived are processed and replied to as normal, and peers only remove that node from new-request candidates. The store's descriptor row stays too (graceful drain).
  • The value range is 0..10000, and the default is 100. Weight = 100 restores normal serving.
  • Propagation of the drain signal is best-effort and eventual -- it only guarantees that "the drain signal was sent." Confirm that a peer has actually removed it from candidates by checking whether the peer's status is draining (chapter 11. Monitoring §2). The operational vocabulary drain/restore above is just the app's admin layer putting a name on Weight = 0/= 100.

The same Weight is also set as an initial value at registration time.

var mesh = options.AddRouteMesh("services")
    .Listen("tcp://0.0.0.0:7101")
    .SetRoutingId(RoutingId.From("orders-1"));
// This channel role's starting weight
mesh.Channel("orders").Server().SetWeight(30);

7. Serialization Codec

The payload serialization codec is enabled through framework registration.

options.Codecs.Use(ZLinkProtobufCodec.Default);
options.Codecs.Use(ZLinkMessagePackCodec.Default);

A payload has to be a DTO the codec can serialize. If the root/element type is abstract/interface, it's a configuration error without an explicit codec.

See it in a sample — when to specify a codec. Only Bingo, a real-time game that needs to cut packet size and encoding cost, registers the Protobuf codec and defines its DTOs with .proto. The rest of the samples don't register a codec and use the default -- explicit registration is an optional step you take only when you need it.

If you need a format beyond the default codecs (Avro, Thrift, etc.), implement a message serializer and register it by content type. A configuration error occurs if more than one matches a given payload type -- keep only one fallback serializer that accepts every type with no type condition, but you can have several type-conditioned serializers as long as they don't overlap.

public sealed class AvroOrderSerializer : IZLinkMessageSerializer
{
    private readonly Avro.Schema _schema = Avro.Schema.Parse(SchemaJson);

    // A serializer's only responsibility is converting business object <-> ZLinkEncodedPayload (byte payload).
    // Packet-name resolution and codec selection belong to the framework.
    public ZLinkEncodedPayload Serialize(object value, Type type)
    {
        using var buffer = new MemoryStream();
        var writer = new Avro.Generic.GenericWriter<object>(_schema);
        writer.Write(value, new Avro.IO.BinaryEncoder(buffer));
        return ZLinkEncodedPayload.From(buffer.ToArray());
    }

    public object? Deserialize(ZLinkEncodedPayload payload, Type type)
    {
        var reader = new Avro.Generic.GenericReader<object>(_schema, _schema);
        return reader.Read(null!, new Avro.IO.BinaryDecoder(new MemoryStream(payload.ToArray())));
    }
}

// Registers the Avro serializer once, inside the extension.
options.Codecs.Use(new AvroCodecExtension());

After registration, high-level calls still exchange business objects as-is, and serialization is handled by Avro. See the framework-api §9 table for registration surfaces in other languages.

8. Scaling a ChannelName Horizontally

To increase throughput, run several providers that own the same MeshName and ChannelName. The calling node registers provider endpoints via location-store auto-connect or PeerConnections.Connect(...).

See it in a sample — ShoppingMall. Two CommerceApi and two OrderWorkflow instances run at once to verify this section's scaling. The caller doesn't know how many providers there are and calls only by channel name -- no matter which instance receives a request, the same OrderId always arrives at the same owner spot.

// Processing node A -- a node registering the same ChannelName as Server becomes a candidate.
var mesh = options.AddRouteMesh("media")
    .Listen("tcp://0.0.0.0:5600")
    .SetRoutingIdPrefix("resize");
mesh.Channel("image.resize").Server()
    .AddRequestHandler<ResizeHandler, ResizeRequest, ResizeReply>();

The calling node registers the same ChannelName as Client and connects to the processing nodes.

The two paths treat the sending node itself differently. This is where they diverge, so check it before you lock in a scaling configuration.

Route mesh channel ClientServer channel
When the sending node itself is that channel's Server Not a candidate Just as much a candidate as any other Server
So calling from a node where only itself is Server Fails as target-not-found It can select itself
When there are no candidates yet at all Fails as target-not-found immediately Fails after waiting briefly

The reason route mesh excludes itself is structural -- channel registration doesn't create a new socket, it uses an already-existing peer connection, and a MeshNode never forms a peer connection with itself. If you want it handled in the same process, use the ClientServer path.

The wait exists for a reason too. In route mesh, no candidates means no peer has published that name -- waiting won't make one appear. In ClientServer, the configuration may already exist in the same process and just not have finished preparing. So it waits for whichever is shorter: the call's timeout or 5 seconds -- to prevent a call right after startup from failing as target-not-found even though the configuration is correct. This wait doesn't speed up preparation, it just waits for preparation already in progress to finish.

Selection ratio follows weight. After excluding targets with weight 0 and those draining, it picks by the remaining weight ratio. If two candidates are 100 and 300, that's roughly 1:3 over the long run -- it does not guarantee the order of any individual call.

An unregistered ChannelName isn't looked up anywhere else. Even if another MeshNode or ClientServer client exists in the same process, it isn't sent there instead. Conversely, registering the same ChannelName on two or more physical send routes also fails at host startup.

// Calling node -- a manual connection that writes the processing node endpoints directly.
var caller = options.AddRouteMesh("media")
    .Listen("tcp://0.0.0.0:5590")
    .SetRoutingIdPrefix("resize-client");
// Client, since it only calls.
caller.Channel("image.resize").Client();
caller.PeerConnections.Connect("tcp://10.30.1.10:5600");
caller.PeerConnections.Connect("tcp://10.30.1.10:5601");

// Or auto-discovery via the location store -- adding a node doesn't require restarting the caller.
options.AddRouteMesh("media")
    .Listen(0)
    .Channel("image.resize").Client();

↗ View larger

If a specific entity (order ID, user ID) always needs to be handled by the same execution unit, use a Spot or Actor instead of a channel (06-spot).

Give a provider a stable identifier. AddRouteMesh(...).SetRoutingId(...) gives the MeshNode a fixed logical id. Even if the provider stops and restarts a new process under the same RID, the location store connects to the new endpoint under the same logical id (same-rid failover) -- use this to carry which node processed a response (rid), or to keep routing continuous across a process replacement.

9. Route Mesh — Calling a Managed Node Directly

A Node direct call specifies one specific MeshNode by RoutingId. Use this path only when the node itself is the target -- like a health check or an ops command. Don't use it to pick where an Actor/Spot is created or to pin a business message to a specific server.

var mesh = options.AddRouteMesh("play")
    .Listen(playRouterEndpoint)
    .SetRoutingId(RoutingId.From(playRouterId));

// A handler that returns the node's own operational status.
mesh.AddRouteRequestHandler<NodeStatusHandler, GetNodeStatus, NodeStatus>(
    "ops.node.status");

The caller passes both the Node RID and MeshName obtained from the management system.

var target = RoutingId.From("play-node-1");

var status = await routeClient
    // Uses Node direct because it's asking about a specific node's operational status.
    .RequestToNode("play", target, new GetNodeStatus())
    .Async<NodeStatus>(ct);

public sealed class NodeStatusHandler
    : IZLinkRouteRequestHandler<GetNodeStatus, NodeStatus>
{
    public ValueTask<NodeStatus> HandleAsync(
        GetNodeStatus request,
        ZLinkRouteMessageContext context,
        CancellationToken cancellationToken)
        => ValueTask.FromResult(NodeStatus.Ready());
}

A business message uses the target's logical address.

  • Call an Actor with an actor client and ActorId.
  • Call a Spot with a spot client and SpotId.
  • To pick one member of a service, use SendToChannel(...) or RequestToChannel(...).

The Framework picks the current owner and an eligible node, so the application doesn't hold onto a Node RID.

↗ View larger

The relationship with SPOT continues in 06-spot.

10. A Combined Example — Server + Outbound + Pub/Sub

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddZLinkFramework(options =>
{
    options.Codecs.Use(ZLinkProtobufCodec.Default);
    // Discovery: finds handler types in the assembly.
    options.AddHandlersFromAssemblyOf<Program>();

    var mesh = options.AddRouteMesh("services")
        .Listen("tcp://0.0.0.0:7101")
        .SetRoutingId(RoutingId.From("api-1"));
    mesh.Channel("api").Server()
        // Exposure: ties the attribute handler group to this channel.
        .AddHandlerGroup("api");
    // A call-only channel.
    mesh.Channel("account").Client();

    options.AddFanoutChannel("api.events")
        // This process is the publisher.
        .EnablePublisher("tcp://0.0.0.0:7201")
        // Also subscribes to its own publish, as an example.
        .Connect("tcp://127.0.0.1:7201")
        .AddHandler<UserCacheRefreshedEventHandler, UserCacheRefreshedEvent>();
});

var app = builder.Build();

app.MapPost("/users/{id}", async (
    string id, IZLinkRouteClient client, CancellationToken ct) =>
{
    var account = await client
        .RequestToChannel("account", new GetAccountRequest(id))
        .Async<GetAccountReply>(ct);
    return Results.Ok(account);
});

app.Run();

[ZLinkHandlerGroup("api")]
public sealed class UserHandlers(IZLinkFanoutClient publisher)
{
    [ZLinkRequest]
    public ValueTask<GetUserReply> GetUserAsync(
        GetUserRequest request, IZLinkMessageContext context, CancellationToken ct)
        => ValueTask.FromResult(new GetUserReply(request.AccountId, "alice"));

    [ZLinkSend]
    public async ValueTask RefreshAsync(
        RefreshUserCacheCommand command, IZLinkMessageContext context, CancellationToken ct)
        => await publisher
            .Publish("api.events", "user.cache-refreshed",
                new UserCacheRefreshedEvent(command.AccountId))
            .Async(ct);
}

[ZLinkHandlerGroup("api.events")]
public sealed class UserCacheRefreshedEventHandler
    : IZLinkFanoutHandler<UserCacheRefreshedEvent>
{
    public ValueTask HandleAsync(
        UserCacheRefreshedEvent message, CancellationToken ct)
        => ValueTask.CompletedTask;
}

11. Common Problems

  • The handler never fires → AddHandlersFromAssemblyOf(...) alone doesn't expose it. It needs the typed registration under Channel(name).Server() (Exposing a Handler on a Channel).
  • A configuration error → the channel doesn't exist, or that role isn't registered on it. Check the registration.
  • An exception at startup → a duplicate channel name, a duplicate kind + packet name on the same channel, or a client with no connection route. It's fail-fast (Startup-Phase Validation of Registration Errors).
  • ZLink vs Zlink → every server framework type is ZLink (capital L).
  • Sending to a packet with no handler (at runtime) → request fails with an error reply (the client gets it as an exception), while send is silently dropped. Dropped means the caller gets no reply -- it doesn't mean there is no observable record. The application-configured logger/telemetry provider receives a formal structured dispatch-error record (no_handler / reply_error/drop) (chapter 11. Monitoring).