Skip to main content
glamsterdam

Glamsterdam is a data monster

Glamsterdam has arrived on Sepolia and it's time to go through all the new data we're collecting

Glamsterdam is a data monster

Introโ€‹

Sepolia forked into Glamsterdam on 6 October at 13:53 UTC, epoch 353024. The upgrade has two headline changes: enshrined proposer-builder separation (EIP-7732, ePBS) and block-level access lists (EIP-7928, BALs).

ePBS is the big one for us on the data side. Since the Merge, a block has been a single object. The consensus part and the execution payload were built and gossiped together, and most of our tables read execution data straight out of the beacon block. With ePBS they've been split apart which was... fun for our data stack!

We've been building for this via Glamsterdam devnets since March. This post walks through the data we now collect for Glamsterdam and where you can see it.

Xatu: the raw eventsโ€‹

Xatu gained a staggering 25 new raw data tables for Glamsterdam. This is a significant step up to our last fork, Fusaka, which only included 3! They come from four places throughout the network:

Sentriesโ€‹

Xatu sentries run next to beacon nodes and subscribe to the beacon API event stream. Glamsterdam added seven event topics, and we're capturing all of them:

TableWhat it captures
beacon_api_eth_v1_events_proposer_preferencesa proposer's fee recipient and target gas limit
beacon_api_eth_v1_events_execution_payload_bidbuilder bids, with value, execution_payment and blob count
beacon_api_eth_v1_events_execution_payload_gossipthe payload envelope, first seen on the wire
beacon_api_eth_v1_events_execution_payloadthe payload imported into fork choice
beacon_api_eth_v1_events_execution_payload_availablepayload and blobs verified locally, the point where a PTC member can vote
beacon_api_eth_v1_events_payload_attestationindividual PTC votes
beacon_api_eth_v1_events_head_v2the new head event, which says whether the head block's payload is full or empty

Xatu Contributors via Contributoor will also pick up these new events when we release the next version of Contributoor!

Gossipโ€‹

Internally, we have two instrumented consensus clients: Dimhouse, a Lighthouse build with Xatu Sidecar, and tysm, our instrumented Prysm. These clients record the four new gossip topics straight off the wire.

TableWhat it captures
libp2p_gossipsub_execution_payload_bidbuilder bids
libp2p_gossipsub_execution_payload_envelopepayload envelopes
libp2p_gossipsub_payload_attestation_messageindividual PTC votes
libp2p_gossipsub_proposer_preferencesproposer preferences

Cannonโ€‹

Cannon walks the finalized chain and writes one deduplicated row for everything it finds, and it now supports the Glamsterdam parts of the block.

TableWhat it captures
canonical_beacon_block_execution_payload_bidthe winning bid in each block
canonical_beacon_block_payload_attestationthe aggregated PTC votes included in each block
canonical_beacon_block_execution_request_builder_depositbuilders joining through the builder deposit contract (EIP-8282)
canonical_beacon_block_execution_request_builder_exitbuilders leaving
canonical_beacon_block_access_listevery entry in the block-level access list
canonical_beacon_block_access_list_summaryone row per block with change counts and the encoded size
canonical_beacon_state_builderthe builder registry, snapshotted each epoch
canonical_beacon_state_builder_pending_paymentbuilder payments waiting to settle
canonical_beacon_state_builder_pending_withdrawalbuilder withdrawals in the queue
canonical_beacon_state_ptc_memberthe PTC for every slot, one row per seat
canonical_beacon_state_execution_payload_availabilitywhether each slot's payload made it into the state

Synthetic eventsโ€‹

We've added a few synthetic events within tysm that we think may be useful. These aren't exposed by a beacon node normally:

TableWhat it captures
beacon_synthetic_payload_status_resolvedevery change to a payload's status (PENDING, FULL, EMPTY, INVALID), with the PTC tally at that moment
beacon_synthetic_payload_attestation_processedevery PTC vote the node processed, with when it arrived, which peer sent it and how long it took
beacon_synthetic_builder_pending_payment_settlementthe settle-or-drop decision for each builder payment, with the weight it reached against the quorum

CBT: Pre-Aggregated Dataโ€‹

xatu-cbt now builds 11 Glamsterdam tables on top of the raw tables defined above. We are still building these out, so expect to see more pop up as we charge towards Mainnet.

TableWhat it captures
fct_block_payloadthe winning bid joined with what the revealed payload actually held
fct_block_payload_bidthe winning bid in each canonical block
fct_block_payload_first_seen_by_nodewhen each node first saw the envelope, across the event stream and gossip
fct_block_payload_available_by_nodewhen each node had the payload and its blobs verified
fct_block_payload_ptc_votethe PTC verdict per block from on-chain aggregates, plus orphaned blocks only seen live
fct_block_payload_ptc_vote_headPTC votes per block from the live event stream, available before finality
int_block_payload_ptc_vote_canonicalPTC votes per block from the aggregates included in canonical blocks
fct_block_payload_status_hourlydelivered and absent payloads per hour
fct_payload_bid_highest_value_chunked_50msthe auction frontier: the best bid in every 50 ms window around the slot
fct_payload_bid_highest_value_by_builder_chunked_50msthe best bid from each builder in every 50 ms window around the slot
fct_payload_attestation_first_seen_chunked_50msPTC vote arrivals in 50 ms windows

Notes:

  • fct_block_mev and the proposer reward tables used to come from MEV relay data. On Glamsterdam networks they come from the winning bid in each block instead.
  • The transaction columns on canonical_beacon_block, like execution_payload_transactions_count, are zero after the fork, since the payload isn't in the beacon block anymore. Use fct_block_payload for these.

In the wildโ€‹

Labโ€‹

Lab slot pages on Sepolia have a Payload card. It shows the winning bid, whether the proposer built the block itself, when sentries saw and verified the envelope, and the PTC verdict (delivered, late or withheld).

Doraโ€‹

We've integrated some of this data into the slot pages on Dora. The Propagation tab shows when Xatu nodes saw the block and its payload, next to the attestation and PTC vote waves.

Pandaโ€‹

Everything above is queryable through panda, and remains our recommended way to access the data for Core Devs.

panda search examples "ePBS payload"
panda schema clickhouse-raw default canonical_beacon_state_ptc_member

Thanksโ€‹

Sepolia's successful fork of Glamsterdam is the culmination of a lot of hard work from a large group of smart, dedicated people. While we aren't live on Mainnet yet, it's only a matter of time.

Thanks to all of those who we work with to make forks like this possible โค๏ธ


Love,

ethPandaOps Team

Next Post
Panda: one tool for the whole data stack