Chapter 24
Nitro Espresso Streamer Requirements
# Message Queue: The streamer must be able to fetch messages
from hotshot and add them to queue within less than 1 second of
the message being confirmed on Espresso.
Assumption: Espresso is
fully functional or
degraded, meaning it is fully functional except for
periods lasting no more than 24 hours. We also assume everything
sent to espresso will be finalized and that espresso will not fork or
reorg.
Acceptance: Send transactions to Espresso for the given
namespace using the Espresso Nitro batcher. Using Nitro Caff node
that uses the espresso streamer, determine if the block was produced
corresponding to that message within 5 seconds of the message being
confirmed on Espresso.
# Sequencer Sovereignty: The streamer must be able to reject
any batch which was not produced by the centralized sequencer.
Acceptance: Send otherwise-valid transactions to Espresso for the
given namespace, as the Nitro batcher would, but sign them with a
key other than the centralized sequencer’s key. Using Nitro Caff node
that uses the espresso streamer, query the RPC to check that none
of the transactions submitted in this way execute or get included in
blocks.
# Duplicate Message Detection: The streamer must be
able to detect duplicate messages and only add it to the queue if the
message is not a duplicate.
Assumption: Espresso is fully functional
or
degraded, meaning it is fully functional except for periods lasting
no more than 24 hours. We also assume everything sent to espresso
will be finalized and that espresso will not fork or reorg.
Acceptance:
Add unit tests to the streamer to test the duplicate message detection
because its hard to verify this requirement in production.
# Ordering: The streamer must be able to detect if the message
is in the correct order and only add it to the queue if the message
is in the correct order.
Assumption: Espresso is fully functional or
degraded, meaning it is fully functional except for periods lasting no
more than 24 hours. We also assume everything sent to espresso will
be finalized and that espresso will not fork or reorg.
Acceptance:
Add unit tests to the streamer to test the ordering of messages
because its hard to verify this requirement in production.
# Support for Multiple Query Node: The streamer must be
able to support multiple query nodes and not rely on a single node.
Assumption: Espresso is fully functional or
degraded, meaning it is
fully functional except for periods lasting no more than 24 hours.
We also assume everything sent to espresso will be finalized and that
espresso will not fork or reorg.
Acceptance: Run multiple instances
of query nodes and then run the streamer with multiple query nodes.
The streamer should be able to fetch messages from all the query
nodes and only process a response if majority of query nodes agree
on the response.
# Determinism: Any two Espresso streamers starting from
the same initial state will yield the same sequence of messages.
Acceptance: Run the rollup and an Espresso streamer. Record the
messages it outputs for some time. Start another Espresso streamer
with the same initial state and check that it outputs a sequence of
messages matching the recorded sequence.
Acceptance: Repeat this
test in the presence of L1 reorgs, sequencer, batcher, and builder
restarts.
# Checkpoints: The streamer allows
checkpoints to be created.
A fresh streamer can be created from a checkpoint, and the fresh
streamer will yield the same sequence of messages that the original
streamer yields from the point where the checkpoint was created.
Furthermore, checkpoints are
-
1.
- Small, meaning they are efficient and cost-effective to persist either in an L1 contract or in a database.
-
2.
- Recent, meaning replaying from a checkpoint requires a minimal
number of historical Espresso blocks to be scanned, and yields the
first new message within a short time (no more than a few seconds)
of creating a streamer from a checkpoint.
This feature allows both the Nitro batcher and Nitro caffeinated node to
persist their state periodically and resume where they left off in case
they need to restart. This also enables the Nitro batcher to seek to
various places in the block stream in case of L1 reorgs or Permissionless
Batching.
Acceptance: Run the rollup and connect an Espresso streamer. Force messages
to be committed to Espresso out of order. Create a checkpoint and continue
running the streamer for some messages, and record the messages. Create a
new streamer from the checkpoint and run it for the same number of
messages. Check that the two sequences of messages produced are equal.