Error when parsing assethub block event - "polkadotxcm (FeesPaid)"

[OfirModai]

Hi folks,
I run into some issues when trying to parse assethub block 9520865 (and some other ones as well),
When iterating the block events, to figure out fee paid for example, the crate subxt fails to parse some events and stop parsing when fails at one.

async fn get_block_events() {
    let client = OnlineClient::<AssetHubConfig>::from_url("wss://polkadot-asset-hub-rpc.polkadot.io")
        .await
        .expect("Failed to connect to node");
    let bytes = hex::decode(&"8cec6507f89ccb7e156075f167b979d69d79ed6f2cd453f8f221319484da8960").unwrap();
    let hash = <<SubstrateConfig as Config>::Hasher as Hasher>::Output::decode(&mut &bytes[..]).unwrap();
    let events = client.events().at(hash).await.unwrap();
    for (index, event) in events.iter().enumerate() {
        println!("parsing event #{}", index);
        if let Ok(details) = event.as_ref() {
            println!("Event {}: {:?}", details.pallet_name(), details.variant_name());
        }
        else {
            print!("Error parsing event");
        }
    }
}

I built repository to demonstrate what I done: https://github.com/OfirModai/Parsing-assethub-events/blob/example/src/main.rs
Also attached image of the event of the block found in the explorer, there it has 14 events, during my execution it parse only 8 - as it fails the 9th.
The block in Subscan: https://assethub-polkadot.subscan.io/event?block=9520865

parsing event #0
Event Balances: "Transfer"
parsing event #1
Event System: "ExtrinsicSuccess"
parsing event #2
Event System: "ExtrinsicSuccess"
parsing event #3
Event Balances: "Withdraw"
parsing event #4
Event Balances: "Transfer"
parsing event #5
Event PolkadotXcm: "Attempted"
parsing event #6
Event Balances: "Burned"
parsing event #7
Event Balances: "Minted"
parsing event #8
Error parsing event

[jsdw]

I suspect that a runtime upgrade happened after that block (would have to check) which led to the shape of some events changing.

Subxt is focused on head of chain interactions, and so is not really supposed to be used to decode historic blocks/state (yet). There are two paths that you couldtake though:

1. Download the metadata for the block/runtime you're decoding and then OnlineClient::set_metadata to manually tell Subxt to use that instead of the latest metadata. (this may break Subxt's ability to submit transactions and decode current blocks as a tradeoff
2. Have a look at the (still very much a work in progress) subxt-historic library and the examples for it, which is being built in order to answer the question of how to decode "historic" blocks/state/etc. (In the future this is likely to be integrated much more closely with subxt itself so that it's easy to decode historic blocks/state in Subxt, but for now it's separate while it is iterated on). Events aren't specifically added but they are just storage entries that exist at a specific block :)

[OfirModai]

The end goal is to parse current blocks to present for some addresses they're transactions history, including fee for tx.
I did update to the latest matadata (I need always to parse the latest blocks).
your examples didn't show me how to parse fee, as it not shown directly in the extrinsic but only in the block events - if you have somewhere else to look for them I will be happy to know.
The block I am parsing is not an old one, it's two weeks ago, also same thing happen in recent blocks.
So how do you suggest me to get the fee of polkadotxcm tx's?

[jsdw]

Ah I see; apologies, I focused too much on the example block being old; it's a common source of this sort of issue!

I made the following example to reproduce (lots of hackery and not-ideal things here...):

#![allow(missing_docs)]
use subxt::{OnlineClient, PolkadotConfig};
use subxt_rpcs::methods::legacy::NumberOrHex;

#[subxt::subxt(runtime_metadata_insecure_url = "wss://polkadot-asset-hub-rpc.polkadot.io")]
pub mod polkadot {}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let rpc_client = subxt_rpcs::RpcClient::from_url("wss://polkadot-asset-hub-rpc.polkadot.io").await?;
    
    // Use rpc methods as a bit of a hack to get an old block
    let rpc_methods = subxt_rpcs::LegacyRpcMethods::<PolkadotConfig>::new(rpc_client.clone());
    let block_hash = rpc_methods.chain_get_block_hash(Some(NumberOrHex::Number(9520865))).await?.unwrap();

    let api = OnlineClient::<PolkadotConfig>::from_rpc_client(rpc_client).await?;

    // Fetch the old block.
    let block = api.blocks().at(block_hash).await?;
    let events = block.events().await?;

    println!("Block {}", block.number());
    for event in events.iter() {
        match event {
            Ok(details) => {
                println!("  {}.{}", details.pallet_name(), details.variant_name());
            },
            Err(e) => {
                print!("  Error parsing event: {e}");
            }
        }
    }

    Ok(())
}

And the output I got back was:

  Balances.Transfer
  System.ExtrinsicSuccess
  System.ExtrinsicSuccess
  Balances.Withdraw
  Balances.Transfer
  PolkadotXcm.Attempted
  Balances.Burned
  Balances.Minted
  Error parsing event: Scale codec error: out of range decoding Compact<u32>

I am using current metadata and need to configrm that this is still the case with metadata at that same block (as the error still appears to be related to this offhand), and then can look into what exactly the issue is if ensuring the correct metadata doesn't fix it!

[OfirModai]

Great, looking forward for your update:)

[jsdw]

I believe I've gotten to the bottom of this issue.

Steps I took to fix:

1. Download metadata at the failing block number. (for this, I made a tweak to the subxt CLI tool so that it can download old metadata here, and called it with subxt metadata --url wss://polkadot-asset-hub-rpc.polkadot.io --at-block 0x8cec6507f89ccb7e156075f167b979d69d79ed6f2cd453f8f221319484da8960 > OLD_AH.scale
2. Run the below code, telling subxt to explicitly use the old metadata (by default it uses head of chain metadata):

#![allow(missing_docs)]
use codec::Decode;
use subxt::{OnlineClient, PolkadotConfig};
use subxt_rpcs::methods::legacy::NumberOrHex;


#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let rpc_client = subxt_rpcs::RpcClient::from_url("wss://polkadot-asset-hub-rpc.polkadot.io").await?;
    
    // Use rpc methods as a bit of a hack to get an old block
    let rpc_methods = subxt_rpcs::LegacyRpcMethods::<PolkadotConfig>::new(rpc_client.clone());
    let block_hash = rpc_methods.chain_get_block_hash(Some(NumberOrHex::Number(9520865))).await?.unwrap();

    let api = OnlineClient::<PolkadotConfig>::from_rpc_client(rpc_client).await?;

    // Use old metadata
    let metadata_bytes = std::fs::read("./OLD_AH.scale").unwrap();
    let metadata = subxt::Metadata::decode(&mut &*metadata_bytes).unwrap();
    api.set_metadata(metadata);

    // Fetch the old block.
    let block = api.blocks().at(block_hash).await?;
    let events = block.events().await?;

    println!("Block {}", block.number());
    for event in events.iter() {
        match event {
            Ok(details) => {
                println!("  {}.{}", details.pallet_name(), details.variant_name());
            },
            Err(e) => {
                print!("  Error parsing event: {e}");
            }
        }
    }

    Ok(())
}

Now, all events decode successfully.

(Note: I use PolkadotConfig instead of AssetHubConfig for simplicity which is not ideal but works in this case since we don't touch any of the areas that are different)

@OfirModai I suspect that your subxt instance was running for a while, and at some point during this, a runtime update happened which led to theshape of this event changing and thus subxt did not know how to decode it any more. One way to avoid this is to keep subxt uptodate w.r.t the metadata by running this updater. This attempts to keep Subxt uptodate with the latest metadata in the event of runtime updates happening :)

For the most part, you can get away without doing this, because Subxt can more gracefully skip over things that it cannot decode. In the case of events though, they are encoded in such a way that we must be able to decode them fully else we'll be unable to decode any subsequent events!

[OfirModai]

Thank you, works for me now