Chapter 32
Nitro Espresso DA

This page documents the technical design for integrating Espresso DA into Nitro nodes. This design assumes two prerequisites:

• The chain is already using the Espresso Network as its fast confirmation layer.

• The batch poster operates in a trusted environment and will never post a sequencer message with invalid transaction hashes or batcher address.

Here is how DA providers integrate with Nitro:

1.

The batcher builds a batch and prepares to post it.

2.

If any DA provider is enabled, the batcher posts the batch to the DA.

3.

The batcher receives a sequencer message from the DA, which contains all the necessary information to recover the batch.

4.

The batcher writes this sequencer message to L1.

5.

Validators read the sequencer message from L1 and use it to recover the batch.

6.

Validators retrieve the batch data from the DA.

For Espresso DA, the key difference is that all messages are already available on Espresso DA once the batch is complete. Therefore, the batcher does not need to post the batch itself to Espresso DA. The workflow is as follows:

1.

The batcher builds a batch and prepares to post it.

2.

If Espresso DA is enabled, the batcher constructs a sequencer message according to submitted transactions.

3.

The batcher writes this sequencer message to L1.

4.

Validators read the sequencer message from L1 and use it to recover the batch.

5.

Validators retrieve the batch data from Espresso DA.

Therefore, our design focuses on construction of the sequencer message and how the batch can be recovered.

When Espresso DA is enabled, the batch poster posts new sequencer messages to L1, rather than the full batches. These sequencer messages allow the DA Reader to retrieve all messages included in the batches.

The sequencer message for Espresso DA is a byte array containing:

• Header byte: Indicates that the batch should be fetched via Espresso DA.

• Transaction hashes: Hashes of the HotShot transactions, each containing L2 messages.

1const ESPRESSO_DA_HEADER_BYTE = 0x36

This value is not chosen arbitrarily. In Nitro, sequencer messages can be stored in multiple DA providers, and specific bits are used to indicate which DA is in use. Examples of existing bytes:

• 0x80 (1000 0000): The first bit indicates DAS reader.

• 0x08 (0000 1000): The fifth bit indicates TreeDAS reader.

• 0x63 (0110 0011): Indicates a Celestia DA batch.

• 0x50 (0101 0000): Indicates a Blob header.

• 0x00 (0000 0000): Indicates brotli compressed data, implying no DA provider is used.

In practice, all other DA providers can be disabled when Espresso DA is enabled. To maintain flexibility and avoid conflicts, we use 0x36 (0011 0110) to indicate an Espresso DA batch.

Under normal circumstances, the batcher first submits data to DA providers and then obtains the sequencer message. However, when Espresso DA is enabled, this step is not required since the L2 messages are already accessible through Espresso DA. This is also why the batcher does not need to implement the DA provider interface.

1func (b *Batcher) maybePostSequencerBatch(ctx context.Context) error { 
2    /* Omitted code */ 
3    // In a loop, add messages to the building batch 
4    for addMessageLoop() { 
5        msgWithMetaDataAndPos := b.espressoStreamer.Next() 
6        // Record the transaction hash 
7        b.building.hotshotBlockHashes = append(b.building.hotshotBlockHashes, msgWithMetaDataAndPos.TxHash) 
8        /* Omitted code */ 
9    } 
10    /* Omitted code */ 
11    if !b.config().EspressoDAEnabled { 
12        /* Write batches to DA.*/ 
13        /* This is not necessary when Espresso DA is enabled because all messages are already available on Espresso DA. */ 
14    } else { 
15        // Build the sequencer message: header byte + transaction hashes 
16        // Hotshot transaction hashes are always 32 bytes, we can simply append them. 
17        sequencerMsgBytes := []byte{} 
18        sequencerMsgBytes = append(sequencerMsgBytes, ESPRESSO_DA_HEADER_BYTE) 
19        sequencerMsgBytes = append(sequencerMsgBytes, b.building.hotshotBlockHashes...) 
20        sequencerMsg := sequencerMsgBytes 
21    } 
22    /* Omitted code */ 
23}

Here is the DA reader interface:

1type Reader interface { 
2    // IsValidHeaderByte returns true if the given headerByte corresponds to this DA provider 
3    IsValidHeaderByte(headerByte byte) bool 
4 
5    // RecoverPayloadFromBatch fetches the underlying payload from the DA provider given the batch header information 
6    RecoverPayloadFromBatch( 
7        ctx context.Context, 
8        batchNum uint64, 
9        batchBlockHash common.Hash, 
10        sequencerMsg []byte, 
11        preimageRecorder PreimageRecorder, 
12        validateSeqMsg bool, 
13    ) ([]byte, error) 
14}

RecoverPayloadFromBatch is supposed to return the complete batch data, which is then later parsed to messages.

Ideally, we would construct a concise batch format, allowing Nitro’s DA logic to proceed without requiring changes to the existing codebase.

However, the batch construction logic resides in batch_poster.go, where the DA reader is invoked to verify the correctness of sequencer messages. As a result, it is not feasible to call the batch construction code at a lower level within the DA reader.

Two versions should be implemented: one for the replay binary and one for validators.

• Validator DA Reader: Responsible for fetching the underlying batch from the Espresso Network using the sequencer message, and for building a preimage map for the replay binary.

• Replay Binary Reader: Recovers batches using the preimage map; its logic is essentially the same as the Validator DA Reader.

These two versions are similar except for the way the Espresso transactions are fetched and the argument of preimageRecorder.

For the Validator DA Reader, preimageRecorder is not nil; it is used to record preimages for the replay binary, and transactions are fetched from the Espresso Network.

For the Replay Binary Reader, preimageRecorder is nil, and transactions are retrieved from the preimage map.

1 
2func RecoverPayload( 
3    sequencerMsg []byte, 
4    preimageRecorder PreimageRecorder, 
5    getTransactionByHash func(ctx, hashValue) ([]espressoTypes.Transaction, error) 
6) ([]byte, error) { 
7    txHashes, err := parseTransactionHashes(sequencerMsg) 
8    if err != nil { 
9        return nil, err 
10    } 
11 
12    messages := MessageWithMetadataAndPos{} 
13    for _, txHash := range txHashes { 
14        tag := taggedBase64.New(txHash) 
15        tx, err := getTransactionByHash(ctx, tag) 
16        if err != nil { 
17            return nil, err 
18        } 
19        // Record preimages for replay binary 
20        if preimageRecorder != nil { 
21            err := preimageRecorder.RecordPreimage(txHash, tx) 
22            if err != nil { 
23                return nil, err 
24            } 
25        } 
26        parsed, err := parseTransaction(tx) 
27        if err != nil { 
28            return nil, err 
29        } 
30        messages = append(messages, parsed...) 
31    } 
32 
33    // Sort stably and remove duplicates 
34    r.sortAndFilterMessages(messages) 
35 
36    // Create a batch-like structure: each transaction is prefixed with its length 
37    batch := []byte{} 
38    for _, msg := range messages { 
39        length := make([]byte, 8) 
40        binary.LittleEndian.PutUint64(length, uint64(len(msg.Payload))) 
41        batch = append(batch, length...) 
42        batch = append(batch, msg.Payload...) 
43    } 
44    return batch, nil 
45}

1type EspressoDAReader struct { 
2    client *EspressoClient 
3} 
4 
5func (r *EspressoDAReader) isValidSequencerMsg(b byte) (bool, error) { 
6    return b == ESPRESSO_DA_HEADER_BYTE, nil 
7} 
8 
9func (r *EspressoDAReader) RecoverPayloadFromBatch( 
10    ctx context.Context, 
11    batchNum uint64, 
12    batchBlockHash common.Hash, 
13    sequencerMsg []byte, 
14    preimageRecorder PreimageRecorder, 
15    validateSeqMsg bool, 
16) ([]byte, error) { 
17    return RecoverPayload(sequencerMsg, preimageRecorder, r.client.FetchTransactionByHash) 
18}

The replay binary loads data via wavmio.ReadPreImage, which reads data based on hash values and the specified preimage type (here, Keccak).

1type ReplayBinaryReader struct {} 
2 
3func (r *EspressoDAReader) isValidSequencerMsg(b byte) (bool, error) { 
4    return b == ESPRESSO_DA_HEADER_BYTE, nil 
5} 
6 
7func (r *ReplayBinaryReader) RecoverPayloadFromBatch( 
8    ctx context.Context, 
9    batchNum uint64, 
10    batchBlockHash common.Hash, 
11    sequencerMsg []byte, 
12    preimageRecorder PreimageRecorder, 
13    validateSeqMsg bool, 
14) ([]byte, error) { 
15    return RecoverPayload(sequencerMsg, preimageRecorder, wavmio.ReadPreImage) 
16}

ReadPreImage guarantees data correctness. If a challenge occurs due to incorrect data from a counterparty, a one-step proof is generated to check the hash value. However, ReadPreImage cannot guarantee the availability of the hash value. If a malicious sequencer message is posted, validators may not be able to retrieve the complete batch. Therefore, it is necessary to run the batcher within a Trusted Execution Environment (TEE).

As noted earlier, the DA reader does not construct a full batch format.

After obtaining the batch-like data, the original batch parsing process should be bypassed:

1func parseSequencerMessage() { 
2    if isEspressoDABatch(data) { 
3        // Store messages 
4        messages, err := parseEspressoPayload(data) 
5        if err == nil { 
6            parsedMessage.EspressoMessages = messages 
7            return parsedMessage, nil 
8        } 
9    } 
10}

1func (r *inboxMultiplexer) Pop(ctx context.Context) (*arbostypes.MessageWithMetadata, error) { 
2    /* Omitted code */ 
3    // If the next message is an Espresso message, return it directly 
4    msg, err := r.getNextEspressoMessage(ctx) 
5    if err != nil { 
6        msg = r.getNext() 
7    } 
8    /* Omitted code */ 
9}