Skip to content

Messaging Local Bench Specification

This document is the standard for comparing the relative cost of gRPC, the ZLink raw binding, and the ZLink framework in the same format on a local development machine. The target languages are the five dotnet, node, java, kotlin, and cpp, and a reference bench written in C is used alongside them as the floor value. It doesn't represent a production environment's mesh, TLS, L7 load balancer, multi-node distribution, or network latency. The thing measured is messaging from server process A to server process B (server-to-server), and the load is generated inside A after it receives an HTTP trigger (§10). The HTTP call is the start signal, not a measured operation.

This bench is a service-level comparison. It measures the cost actually paid when the same task is implemented with each stack. It doesn't ask whether the two stacks use the same mechanism internally. A difference that arises because a capability exists in only one stack isn't hidden; it's left in the result as it is. The question this document answers is "implementing this task with gRPC costs this much, implementing it with ZLink costs this much." "Which side is faster when the two stacks transmit the same way" isn't this bench's question.

1. Comparison Targets

1.1 Implementation Names

Three implementations are compared per language. <lang> is one of dotnet, node, java, kotlin, cpp. The report table and the RESULT line use the same names.

Implementation Name Meaning
grpc-<lang> That language's gRPC unary RPC
zlink-<lang> The raw binding's ROUTER↔ROUTER TCP path, bypassing the framework
zlink-framework-<lang> Framework RouteMesh request/send addressed directly to a node RID

The default execution order is grpc-<lang>, zlink-<lang>, zlink-framework-<lang>. Since the three implementations run the same pattern at the same payload size and the same active duration, the table shows the three implementations side by side under one pattern.

1.2 The C Reference Bench

grpc-c and zlink-c from framework/bench/grpc/c are kept alongside as the floor reference. There's no framework layer in C, so only these two implementations exist there. The zlink-c request-backpressure value is the reference value for judging where each language's raw binding stands (§7.2). The C reference bench stays client-driven; it isn't converted to the server-driven model of §10, because what's borrowed from it is the per-request cost, not the place where load is generated. That difference is noted next to the denominator in §7.2.

Both ZLink rows use ROUTER↔ROUTER and address the peer node directly by RID. This condition isn't optional; it's the contract that makes the judgement formula hold. Every cell has one server (B).

Row Socket Configuration
zlink-framework-<lang> RouteMesh's direct node calls requestToNode / sendToNode, naming the server node's RID. RouteMesh connects as ROUTER↔ROUTER
zlink-<lang> The raw binding's ROUTER↔ROUTER. The client side also creates a ROUTER and requests/sends by specifying the peer ROUTER's routing id
zlink-c The same ROUTER↔ROUTER configuration as the raw binding row above
%%{init: {'flowchart': {'nodeSpacing': 32, 'rankSpacing': 40, 'padding': 8, 'wrappingWidth': 180}, 'themeVariables': {'fontSize': '18px'}}}%%
flowchart LR
    FC[framework RouteMesh ToNode client] <--> FS[framework node handler server]
    RC[raw binding ROUTER] <--> RS[raw binding ROUTER]

This contract is needed because of the judgement formula in §7.2. zlink-framework-<lang> / zlink-<lang> is the ratio for looking at the cost the framework layer additionally requires. Only when both rows use the same socket pattern and the same way of naming the target does this ratio isolate the framework layer. If the raw row is DEALER→ROUTER, or the framework row sends by channel name through requestToChannel/sendToChannel (which includes node selection), the division result contains the framework layer cost together with the socket pattern and target selection difference, and that value isn't the framework layer cost. The gRPC row is one channel attached 1:1 to one server, and the configuration above has the same shape.

Comparisons between ZLink models — DEALER→ROUTER versus ROUTER↔ROUTER, ToChannel (with node selection) versus ToNode, ClientServer versus RouteMesh — are not items of this bench; they belong to a separate ZLink model-comparison bench. This bench compares only the 1:1 request path that has the same shape as gRPC.

The current implementation state is recorded alongside. Java and the .NET, C++, and Node framework rows have moved to this configuration (Issue #13). Moving the raw paths and the C reference bench client is separate work. The table above prescribes the configuration to be used for measurement, and a value obtained before that change completes isn't a value that satisfies this specification.

2. Measurement Patterns

The initial scope uses only two payload sizes and three patterns. The default payload sizes are 1024 and 4096 bytes. The payload size is the full size of the protobuf bytes body or the raw ZLink message body. The first 29 bytes are used as the measurement header, and the rest is filled as the business payload area. The gRPC HTTP/2 frame, protobuf field overhead, ZLink envelope, and ZMP header aren't included in this size.

Pattern Name gRPC ZLink raw binding ZLink framework Interpretation
request-serial unary Echo RPC raw request send and reply receive node request call Sends one request and sends the next only after the reply completes
request-backpressure unary Echo RPC raw request send and reply receive node request call Puts no ceiling on incomplete requests. Submits continuously until admission backpressure
send-saturation unary Command RPC, replying Empty raw one-way send submission node send submission Compares the command path, which has no reply payload

The API names the three implementations use differ per language, but the contract is the same. In Java, the framework node request is requestToNode(mesh, rid, message) and the node send is sendToNode(mesh, rid, message) (§1.3). The per-language modules are in §8.

request-serial's throughput is decided by a single request's round-trip latency. This value is meant to show the cost of a "process one at a time" usage pattern.

request-backpressure puts no application ceiling on the number of incomplete requests. Without waiting for replies, the client submits requests continuously until it meets admission backpressure, and resumes submitting when it is signalled to. The gRPC side has the same shape: it submits until its own flow control stops it. Neither side gets an application ceiling.

In this pattern the incomplete depth is a measurement, not a setting. The depth a stack reaches is an outcome its design decides, so it is recorded as part of the result (§5.2).

This loop shape is not new; it is ported from an implementation already in the repository. The reference implementation is bindings/node/perf/multi/perf_multi_socket_reqrep.ts, which submits async requests continuously without waiting for a reply and then hands the turn to the completion pump. All six rows use the same shape, and the gRPC side is treated the same way: unary calls are submitted continuously rather than awaited one at a time.

This repository's perf convention already requires that inflight depth not be fixed artificially and that requests be submitted continuously until admission backpressure (the request/reply client items in doc/perf/PERF_MULTI_TEST_POLICY.md, and the fixed application window item in doc/perf/PERF_POLICY.md). The fixed-window pattern (request-window) diverges from that convention, so it is excluded from this bench. Behaviour at a fixed depth is an item of the ZLink model-comparison bench.

send-saturation isn't averaged together with request/reply. This pattern exists to look at the command family's relative cost separately.

2.1 The gRPC Counterpart for send-saturation

The gRPC side of send-saturation uses the current unary Command(BenchPayload) returns (Empty) as it is. No client-streaming RPC is added, and no RPC is added to the proto. The reasons this is the correct comparison for this bench are below.

  • In a real service, a call that needs no response is implemented as a unary call with an Empty reply. Client-streaming is a form meant for uploads and bulk ingestion, and isn't how a service sends a single command that needs no response.
  • gRPC has no one-way call primitive. So it pays one round trip when handling this task. In a service-level comparison that cost is a value that belongs in the result.
  • Inserting a different usage form on the gRPC side to even out the comparison would measure an implementation that real services don't use.

The reading rule for this cell is fixed alongside. This cell isn't described as a transport-speed difference. The correct sentence form is below.

When a command needs no response, gRPC pays a unary round trip while ZLink completes with a
one-way send. Under this condition the difference was N times.

3. Execution Conditions

  • For each implementation (grpc-<lang>, zlink-<lang>, zlink-framework-<lang>) run one source process A and one target process B. A has the HTTP trigger listener, a stats endpoint, and the client toward B (gRPC stub, raw ROUTER, framework node client). B has the echo (request) or receive counter (send) and a stats endpoint. §10 defines the roles, the trigger contract, and the cell order.
  • The local runner starts B then A for each cell and announces the phase start to A's trigger endpoint over HTTP. The runner itself generates no load.
  • Only a loopback address (127.0.0.1) is used. Ports use the per-language bands in §9.
  • Runs as a Release build.
  • Runs a fixed-duration measured active window after warmup. The warmup length is set per language and the value used is recorded in the result (§8.2).
  • The default payload size is 1024,4096 bytes.
  • The request-backpressure pattern has no incomplete-request ceiling setting. In that pattern depth is not a condition that is set but a result that is measured and recorded (§5.2). This bench has no request_window setting.
  • The default send concurrency is 8.
  • gRPC and ZLink framework use the same protobuf DTO. The ZLink raw binding skips the framework but puts the same shape on the wire: two parts, an envelope header part and a protobuf-encoded BenchPayload part, with the 29-byte measurement header inside that protobuf bytes body. This shape is not a choice; it is a precondition of the judgement. §7.2 formula 1 divides zlink-<lang> by zlink-c, so two rows with different wire shapes would divide two different experiments. The reference implementation is framework/bench/grpc/c, bench_zlink_client.cpp:14-16 and :130-140.
  • ZLink uses a manual endpoint connection with no location store.
  • Both ZLink rows use the ROUTER↔ROUTER configuration in §1.3.
  • The ZLink raw binding's request echo endpoint and command receive endpoint are separated. In the command measurement, if request echo replies mix into the same socket, the one-way receive volume with no reply can't be observed correctly.
  • TLS, compression, service mesh, gateway, and broker are not used.
  • Languages aren't measured at the same time. Only one language is measured at a time.

The two ZLink rows differ in how many endpoints they use. The raw binding row separates its request echo endpoint from its command endpoint and so uses two, while the framework row uses one RouteMesh connection that carries both a request handler and a send handler. This difference is intentional and contaminates no cell, because patterns are measured one at a time. While the send-saturation cell is measured, only send traffic occurs on either row; while a request-family cell is measured, only request traffic occurs on either row.

This reasoning depends on the premise that only one pattern is measured at a time. If a scenario that runs two patterns concurrently is added, the premise no longer holds and this endpoint configuration must be revisited.

The settle between cells follows this contract. After a cell completes, do not wait a fixed time. Poll the server's received count until it stops advancing, with an upper bound on that wait. If the count has not stopped advancing within the bound, record that fact and the observed drain time in the results, and mark the next cell that uses the same server as contaminated, excluding it from the tables and from every judgement. A contaminated cell is not measured and published. The observed drain time is recorded per cell in the results. This bound is not a formality; it is load-bearing. A bound set too small loses cells even on a healthy run, so it must be large enough that a healthy run never reaches it. The reference value is 30 seconds.

4. Output Format

The report table is grouped by pattern, with the implementation name shown on each row. The example follows the format below.

  > Benchmarking current for request-backpressure...
    Testing local:
      | Implementation          | Size     |       Throughput |    Bandwidth |  Lat.Mean(ms) |   Lat.P95(ms) |   Lat.P99(ms) | Client CPU | Client Mem | Server CPU | Server Mem |
      |-------------------------|----------|------------------|--------------|--------------|--------------|--------------|------------|------------|------------|------------|
      | grpc-dotnet             | 1024B    |       10.00 KOPS |   10.24 MB/s |     1.000 ms |     2.000 ms |     3.000 ms |      10.0% |  100.0 MB |      10.0% |  100.0 MB |
      | zlink-dotnet            | 1024B    |       30.00 KOPS |   30.72 MB/s |     0.300 ms |     0.600 ms |     0.900 ms |      10.0% |  100.0 MB |      10.0% |  100.0 MB |
      | zlink-framework-dotnet  | 1024B    |       20.00 KOPS |   20.48 MB/s |     0.500 ms |     0.900 ms |     1.200 ms |      10.0% |  100.0 MB |      10.0% |  100.0 MB |

The report and console output also leave a per-metric RESULT,current,... format alongside, for easier handling by the perf runner. The fields of one row are in the following order.

RESULT,current,<scenario>,local,<payload_size>,<metric>,<value>

An example follows.

RESULT,current,grpc-dotnet-request-backpressure,local,1024,throughput,10000.000
RESULT,current,zlink-dotnet-request-backpressure,local,1024,latency,0.300
RESULT,current,zlink-framework-dotnet-request-backpressure,local,1024,latency_p99,1.200

<scenario> is <implementation name>-<pattern name>, so the language is part of the name. For example, Node's equivalent cell is zlink-framework-node-request-backpressure.

The metric value uses throughput, bandwidth, latency, latency_p95, latency_p99, client_cpu_percent, client_memory_mb, server_cpu_percent, server_memory_mb. The raw value of throughput is completions per second, and the table displays it divided into KOPS or KMSG/s.

In the server-driven model (§10) client_* is the value of source process A and server_* is the value of target process B. The metric names are kept for the aggregator and for older raw files; the table header shows them as Source CPU and Target CPU. The per-cell raw JSON carries the following in addition.

Field Meaning
role source or target. A and B each write their own raw file and the runner merges them into one cell
trigger The trigger request A received, kept as is. Fields: runId, cellId, pattern, payloadBytes, durationMs, warmup (the warmup length, in the unit the language harness records per §8.2), endpoint (A's trigger URL, the companion information of §7.1) and receivedAtUnixMs (the wall-clock time A received the trigger). All eight are required; the aggregator accepts no alias or default
streams A's logical stream count and the per-stream in-flight ceiling (§10.3)
target_stats Receive count, error count, and drain time the runner read from B's stats endpoint after settle

5. Metrics

The required output is the metrics below. The throughput unit is separated to match the pattern's character.

Metric Meaning
Throughput Throughput during the measured window. The table shows the value and unit together in one cell, e.g. 10.000 KOPS, 183.618 KMSG/s
Bandwidth The transfer volume calculated from payload size and throughput. Shown as MB/s
Lat.Mean(ms) Mean latency. For request/reply this runs from just before A's outbound call until the reply completes; for send it's the receive latency B calculates from the header
Lat.P95(ms) p95 latency
Lat.P99(ms) p99 latency
Source CPU (client_cpu_percent) The CPU percentage source process A used during the active window
Source Mem (client_memory_mb) Source process A's working set
Target CPU (server_cpu_percent) The CPU percentage target process B used during the active window
Target Mem (server_memory_mb) Target process B's working set

request-serial and request-backpressure calculate KOPS based on the number of completions where an echo reply came back. Here, 1 KOPS means 1,000 request/reply completions per second.

send-saturation calculates KMSG/s based on the number of messages the server received during the active phase. Here, 1 KMSG/s means 1,000 messages per second. Since a ZLink send doesn't wait for a reply, throughput isn't calculated from the client's submission call count alone.

5.1 The Source Saturation Rule

"Client" in this section means the process that generates the load, which in the server-driven model is source process A. Source CPU is recorded for every cell. It isn't optional. A percentage alone cannot decide saturation, so the number of cores used is recorded alongside the percentage. The percentage is taken against all of the machine's logical cores. On a machine with 20 logical cores, a single-threaded client that fully occupies one core still reads 5%, and no fixed percentage threshold can catch that saturation.

Saturation is judged against the instrument and the ceiling that each language's harness declares. A cell whose declared instrument reaches 0.95 of the declared ceiling is marked as a saturated cell and is excluded from throughput ranking. In that cell the limit was set by the client runtime, not by the transport. A saturated cell's throughput is recorded only as "the value observed with this client configuration."

A harness declares what it measures, not only the ceiling, because the right instrument differs per language. The purpose of the rule is to catch "the client runtime, not the transport, set this limit", so the instrument has to measure the execution resource on which user code runs.

Language Declared instrument Ceiling
dotnet process cores used the parallelism the harness declares
node performance.eventLoopUtilization() from perf_hooks 1.0
java, kotlin CPU of the submit threads (ThreadMXBean, jvm_thread_cores) the submit parallelism the harness declares
cpp CPU of the application thread that runs submit and the completion drain (CLOCK_THREAD_CPUTIME_ID, submit_thread_cores) 1

dotnet is the only language that declares process cores used.

Measurement shows why process cores are the wrong instrument for Node. The ZLink binding runs native I/O threads, so process CPU divided by elapsed time reads 1.3-1.4 cores on that client, and those threads run no user code. Against a ceiling of 1 every cell would be marked saturated even with an idle JS thread, and the mark would carry no information. What actually limits this client is the one JS thread on which user code runs, and event loop utilization measures exactly that.

Process CPU is not a usable saturation instrument for a ZLink client. That is why this section makes the instrument a per-language declaration. Three languages reached the same conclusion for three different reasons: in Node process CPU also counted the binding's native I/O threads, in Java it counted GC and JIT threads, and in C++ it again counted the binding's I/O threads. Four of the five languages declare something other than process CPU. The problem is not the threshold but whether the two rows a judgement divides are commensurate. Measured in C++, zlink-cpp request-backpressure reads 0.950 on its declared instrument but 1.90 in process cores, while grpc-cpp in the same run reads 0.700 and 0.70 — effectively the same value. Judged on process cores, that difference reports which row links Core rather than which row was client-bound.

The declared instrument and ceiling differ per language, so both are recorded in the cell record and in the report. A result that declares no instrument or no ceiling cannot have its saturation judged, and the fact that it could not be judged is recorded in the result. An older result that declares no instrument is read as having declared process cores used.

5.2 Incomplete Depth Is Recorded Per Cell

Request-family cells record the three values below for every cell. The three are not optional.

Metric Meaning
peak_in_flight The maximum number of incomplete requests observed during the active window
Depth The mean incomplete count calculated as throughput x mean latency (Little's law)
abandoned The number of requests that did not complete by the drain bound

In request-backpressure these three are the result. Since no depth is configured, the depth a stack reaches is an outcome its design decides, and it goes into the table with the same standing as throughput. Quoting throughput without the depth reached makes values obtained at different depths read as values from the same condition.

Cells with a nonzero abandoned and cells with nonzero errors are not used for throughput comparison. What such a cell reports is not a speed but the fact that some requests did not complete. That fact is recorded as a correctness item, not a performance item.

6. The Measurement Payload Header

A 29-byte header with the same meaning as bindings/c/perf's metric header is placed at the front of the measured payload. This header is part of the payload body, going at the front of the protobuf bytes body or the raw ZLink message body.

Offset Size Value
0 4 magic 0x5A4C4E4B (ZLNK)
4 4 run id
8 1 phase (0 warmup, 1 active)
9 4 payload size
13 8 sequence
21 8 send timestamp ns

For request-serial and request-backpressure, the client validates the header that came back in the reply payload, and calculates KOPS from the active-phase reply count. send-saturation has the server read the header to calculate the active-phase message count and server-side receive latency. This approach puts the measurement value inside the payload so a one-way throughput isn't overstated by a client stopwatch alone.

This header layout is identical in every language. If a language used a different layout, its cells couldn't be placed side by side with the others.

7. Interpreting Results

This bench is a small local comparison tool.

7.1 Information Kept Alongside the Result

To put a performance-superiority statement into a result document or guide, keep the following information alongside it.

  • CPU, OS
  • That language's runtime version and gRPC library version
  • Commit hash
  • Payload size
  • Warmup and active duration configuration
  • gRPC and ZLink endpoint
  • The depth reached (request-backpressure pattern, §5.2)
  • A's logical stream count, per-stream in-flight ceiling, and trigger endpoint (§10)
  • Per-cell peak_in_flight, depth, and abandoned (§5.2)
  • The send concurrency value
  • Source (A) CPU and whether the cell was saturated (§5.1), target (B) CPU
  • The original result JSON

7.2 Judgement Between Layers

Performance judgment compares within the same ZLink layer, not against gRPC. The raw binding is judged against the zlink-c request-backpressure result under the same conditions. If the raw binding reaches 80% or more of the C result, the binding layer's baseline performance is judged as passing. The framework is judged against the same language's raw binding result. If the framework reaches 80% or more of the raw binding result, the framework's added cost is judged as passing.

at each of payload sizes 1024 and 4096:
zlink-<lang> / zlink-c                     >= 0.80   binding layer passes
zlink-framework-<lang> / zlink-<lang>      >= 0.80   framework added cost passes

Both formulas are calculated separately per payload size. A language passes only when it satisfies the criterion at both 1024 and 4096. A result that satisfies only one size isn't a pass, and the per-payload values are always recorded as they are. A stack that holds the criterion at 1024 but degrades at 4096 has a real problem, and since the report shows both sizes anyway, a per-payload gate exposes that problem at no added cost.

This criterion is applied to patterns where ZLink can send the next request without waiting for the reply. Such a pattern is the sound judgement cell because gRPC unary Echo and a ZLink request give the same guarantee, namely confirmation that the server processed the message. request-serial is kept as a supplementary metric for looking at the round-trip latency of a process-one-at-a-time usage pattern.

The reference pattern for judgement is request-backpressure. Whether a language passes is decided by that pattern satisfying the criterion at both payload sizes.

The grounds for choosing the reference pattern this way are below.

  • Both formulas are ratios of two rows measured under one condition, so that condition has to be one the numerator and the denominator can actually reach. A ratio computed under a condition that is not reached reports that condition rather than the layer cost.
  • A fixed window is a condition no service imposes on itself. A low ratio under it can mean "the per-request cost is high" or "that depth was never reached", and because both causes appear as the same number, the ratio alone does not separate them.
  • request-backpressure computes the ratio with each stack at the depth its own design allows, so it compares what a caller submitting normally actually gets. That is what "the binding layer's baseline performance" has to mean.

What this choice does not solve is recorded with it. request-backpressure removes the fixing of depth, not the difference in depth. A stack that stays at a low depth through its own backpressure still produces a low ratio in this pattern, and the ratio alone still fails to separate per-request cost from depth reached. So for either pattern, a ratio published without the three depth values of §5.2 beside it is not published at all.

There is no fixed-window pattern in this bench. How a stack behaves at an externally chosen depth is an item of the ZLink model-comparison bench, and loss and non-completion observed there are recorded separately as correctness items.

The second formula represents the framework layer cost only when the socket configuration in §1.3 is observed.

The denominator of the first formula, zlink-c, comes from the client-driven bench (§1.2) while the numerator comes from the server-driven model (§10). The two models generate load in different places, so read the ratio as "the same per-request cost measured under a different model" and say so in the report. The centre of the public comparison report isn't this ratio but the direct grpc · zlink · zlink-framework comparison within one language; the ratio goes in an appendix.

7.3 Cross-Language Reading Rules

Absolute throughput isn't compared across languages. Placing grpc-node and grpc-java side by side is a runtime comparison, not a ZLink comparison. If two languages' throughput differs in the table, the cause is mostly the runtime and the gRPC library, and this bench doesn't separate that cause out.

What can be read across languages is the ratio. zlink-framework-<lang> / zlink-<lang> is a value each language calculated against itself, so it can answer "which language's framework layer is the most expensive." A table holding a cross-language comparison places this ratio as its main entry.

7.4 Unit Alignment

Section 4 fixes the throughput unit as completions per second. Even when a runner records the value at a different scale, the shared aggregator (framework/bench/grpc/tools/) recovers that scale from bandwidth (which section 5 fixes as MB/s) and normalizes it, so comparison and judgement are always made from the aggregator's output. The table a language client prints for itself is a convenience for checking a single run directly, not a basis for judgement. Ratios are not calculated by reading a runner's report directly.

Results are interpreted only as "faster/slower under this condition." No claim of general production performance or superiority across every payload is made.

8. Per-Language Configuration

8.1 Modules Used Per Language

Language gRPC library and server Framework module Raw binding protobuf codec
dotnet ASP.NET Core gRPC framework/languages/dotnet/src/Zlink.Framework bindings/dotnet Zlink.Framework.Codecs.Protobuf
node @grpc/grpc-js framework/languages/node/packages/framework bindings/node packages/framework-codec-protobuf
java grpc-java zlink-framework-core bindings/java zlink-framework-codec-protobuf
kotlin grpc-kotlin coroutine stub zlink-framework-kotlin bindings/kotlin Uses the same codec as Java
cpp system libgrpc++ and grpc_cpp_plugin framework/languages/cpp/framework bindings/cpp zlink::framework_codec_protobuf

A's HTTP trigger listener uses each language's standard HTTP server (ASP.NET Core minimal API, Node http, the JDK HttpServer, C++ framework HTTP hosting). The listener sits outside the measured path and enters no cell's cost.

Kotlin is excluded from the full matrix and only supplementary cells are measured (§10.5). Kotlin uses a suspend interface on the ZLink side, so the gRPC side uses the grpc-kotlin coroutine stub as well. Only when the coroutine stub can't be used is the grpc-java blocking stub used, and that reason is recorded in the result.

C++ uses the libgrpc++ installed on the system. gRPC isn't built through vcpkg. This machine's version is 1.51.1, and since it's an old version it must be recorded in the result.

8.1.1 Per-message Payload Work

For every request/send submission, each supported client creates a body containing the 29-byte measurement header and a message object matching the BenchPayload schema, then serializes it with that language's protobuf runtime. Raw uses constant wire envelope header parts. Received payloads are parsed with protobuf; request replies serialize a typed payload and preserve the request's measurement header and timestamp. Sends have no payload reply; gRPC Command serializes its existing Empty response.

Language Raw request/reply serialization and receive parsing gRPC Framework
Java A fresh BenchPayload: toByteArray / parseFrom grpc-java serializes/parses the same generated type ZLinkProtobufCodec serializes/parses the same generated type
.NET A fresh BenchPayload: WriteTo / Parser.ParseFrom Google.Protobuf based gRPC serializes/parses the same generated type Protobuf codec serializes/parses the same generated type
C++ A fresh BenchPayload: protobuf serialization / ParseFromArray libgrpc++ serializes/parses the same generated type Protobuf codec serializes/parses the same generated type
Node A fresh BenchPayload DTO through the existing proto-loader serializer / deserializer Uses the same proto-loader protobuf serializer / deserializer Connects the same proto-loader serializer / deserializer to the schema envelope codec
C binding C++ driver creates a generated BenchPayload per message and uses existing libprotobuf Same generated type and libprotobuf No C framework row

The C binding bench uses .cpp drivers and already depends on protobuf. Raw shares that dependency; there is no manual protobuf framing exception. This table compares payload work, not transport calls, envelope processing, or buffer copy counts. Java raw and gRPC use ByteString.copyFrom, while Framework retains its existing UnsafeByteOperations.unsafeWrap path.

Run tools/test_raw_wire.sh to compare the frozen pre-change 29-byte payload dump with protobuf output, check length varint boundaries, and verify request reply body preservation.

Results including raw serialization cost form a new baseline. Do not compare performance trends with framework/raw ratios from raw drivers that bypassed serialization. A lower raw throughput can raise this ratio; that reflects corrected measurement conditions, not improved framework performance.

8.2 Values Set Per Language That Must Be Recorded

The three values below are set differently per language. The goal isn't to make them identical but to leave the values used in the result.

Item Reason
Warmup length The JVM reaches steady state only after JIT warmup finishes. Forcing the same warmup as .NET would measure an unwarmed runtime
gRPC server configuration The default server implementation differs per language. The gRPC side is left at each language's default configuration, and that configuration is recorded in the result
Runtime and gRPC library version The SDK version, runtime version, and gRPC library version are recorded in the cell's raw output and in the report

9. Port Bands

One implementation needs four ports (A trigger, A stats, B endpoint, B stats; the raw binding needs five because B's request and command endpoints are separate) and each language has three implementations, so every language gets a band of 20. The meaning of an offset within a band is the same in every language.

Language Band grpc A trigger/stats, B endpoint/stats zlink raw A trigger/stats, B request/command/stats framework A trigger/stats, B endpoint/stats
dotnet 5200-5219 5200/5201, 5202/5203 5205/5206, 5207/5208/5209 5212/5213, 5214/5215
node 5220-5239 5220/5221, 5222/5223 5225/5226, 5227/5228/5229 5232/5233, 5234/5235
java 5240-5259 5240/5241, 5242/5243 5245/5246, 5247/5248/5249 5252/5253, 5254/5255
kotlin (supplementary, §10.5) 5260-5279 5260/5261, B uses the java band 5242/5243 none 5272/5273, B uses the java band 5254/5255
cpp 5280-5299 5280/5281, 5282/5283 5285/5286, 5287/5288/5289 5292/5293, 5294/5295
C reference (client-driven) 6200-6219 6200/6201, 6202/6203 6205/6206, 6207/6208/6209 none

+16 to +19 of each band are reserved. The runner checks that its own band's ports are free before starting a measurement. If one is in use it stops instead of moving to another port. Moving a port would make the endpoint recorded in the result disagree with the endpoint actually used. The Kotlin supplementary cells reuse Java's B, so they never run at the same time as a Java measurement (one language at a time).

10. The Server-Driven Execution Model

10.1 Roles

Role Processes What it does
trigger client 1 (the runner) Sends HTTP POST /bench/start to A's trigger endpoint. Generates no load and counts no result
source A 1 per implementation On trigger, repeats request or send toward B over the logical streams of §10.3, aggregates completions, latency, and errors itself, and writes the cell's raw JSON. Exposes progress on its stats endpoint
target B 1 per implementation For request, echoes the same payload; for send, reads the header and counts received messages and receive latency. Exposes the values on its stats endpoint

The A·B pairs of the three implementations are independent process pairs. Even within one language only one pair is measured at a time.

10.2 Trigger Contract

The trigger and admin contract of the common perf specification (framework/doc/framework/common/perf/README.en.md §4.2, §5.1, §16) is used as is. Request and response are JSON and the five languages use the same fields.

POST http://127.0.0.1:<A trigger>/bench/start
{ "runId": "...", "cellId": "...", "pattern": "request-backpressure",
  "payloadBytes": 1024, "phase": "warmup" | "active",
  "durationMs": 5000, "requestWindow": 100, "sendConcurrency": 8 }
→ 200 { "accepted": true, "runId": "...", "cellId": "...", "phase": "active", "startedAt": <monotonic ns> }
  • A phase starts once. A duplicate trigger with the same runId, cellId, and phase returns the same start acknowledgement and generates no second load.
  • The trigger only carries pattern, payload, duration, window, and concurrency; it doesn't change the specified values. The defaults are the values §3 sets.
  • GET http://127.0.0.1:<A stats>/bench/stats and GET http://127.0.0.1:<B stats>/bench/stats return the phase, submitted/completed/error counts, received count, and current in-flight. They are what settle (§3) polls.
  • The HTTP round trips of trigger and stats are not measured operations and don't enter throughput or latency.

10.3 Logical Streams and the Pattern Mapping

A runs independent load flows toward B as logical streams. A pattern is expressed as a stream count and a per-stream in-flight ceiling.

Pattern Streams In-flight per stream Meaning
request-serial 1 1 Send one request, then the next after the reply
request-backpressure 1 No ceiling Submit until admission backpressure is met
send-saturation send_concurrency (default 8) 1 (until send completion notification) The command path with no reply

The stream count and per-stream in-flight are recorded in the cell's raw file (§4). How a language harness implements a stream (thread, task, coroutine, event loop) differs per language and is left in the result as in §8.2.

10.4 Cell Order

  1. The runner checks that its language band (§9) is free.
  2. Start the implementation's B and wait until its stats endpoint answers (30-second limit).
  3. Start the implementation's A and wait until A connects to B and reports route ready (30-second limit). If A's stats say ready=false, the cell isn't started and that fact is recorded.
  4. phase=warmup trigger → wait until A finishes warmup and reports phase=idle on stats.
  5. phase=active trigger → A closes the measured window after durationMs.
  6. Settle: poll B's (and A's) stats until the received and completed counts stop growing (30-second limit, the contamination rule of §3 unchanged).
  7. A writes the cell's raw JSON under log/<lang>/<stamp>/ and emits the RESULT lines. The runner merges B's stats into the same JSON as target_stats.
  8. Stop A and B. The next cell starts with a fresh process pair (a process is never reused across cells, so a previous cell's residual state can't enter the next one).

The gRPC implementation's A has the same trigger listener and runs as many unary stubs toward B as there are streams. The gRPC server configuration stays at the language default and is recorded (§8.2).

10.5 Kotlin Supplementary Cells

Kotlin shares the binding, server, and codec with Java, so it's excluded from the full matrix. Instead, two supplementary cells that show the cost of the Kotlin call layer — grpc-kotlin (coroutine stub) and zlink-framework-kotlin (suspend calls) at request-backpressure @1024 — are placed next to the Java rows. Only A is Kotlin; B is the Java binary on the Java band of §9, as is.