eth_getFilterChanges returns empty output

[fgimenez]

System information

Geth version:

Geth
Version: 1.13.5-stable
Git Commit: 916d6a441a866cb618ae826c220866de118899f7
Architecture: amd64
Go Version: go1.21.4
Operating System: linux
GOPATH=
GOROOT=

CL client & version: none
OS & Version: Linux 6.6.11-200.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jan 10 19:25:59 UTC 2024 x86_64 GNU/Linux

Expected behaviour

A local Clique node returns old logs when queried with eth_newFilter + eth_getFilterChanges

Actual behaviour

When the logs are queried right after emitted they are available, a bit after they are no longer available, meaning, creating the filter and requesting filter changes ~60s after the events were emitted.

Steps to reproduce the behaviour

Run a local geth node with this genesis file:

{
  "config": {
    "chainId": 31337,
    "homesteadBlock": 0,
    "eip150Block": 0,
    "eip155Block": 0,
    "eip158Block": 0,
    "byzantiumBlock": 0,
    "constantinopleBlock": 0,
    "petersburgBlock": 0,
    "istanbulBlock": 0,
    "berlinBlock": 0,
    "londonBlock": 0,
    "clique": {
      "period": 2,
      "epoch": 30000
    }
  },
  "alloc": {
    "0x7d4454490AfA139d89042247DE811c2B0d7aCDF2": { "balance": "100000000000000000000000000" },
    "0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266": { "balance": "100000000000000000000000000" },
    "0x70997970C51812dc3A010C7d01b50e0d17dc79C8": { "balance": "100000000000000000000000000" },
    "0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC": { "balance": "100000000000000000000000000" }
  },
  "coinbase": "0x0000000000000000000000000000000000000000",
  "difficulty": "0x20000",
  "extraData": "0x00000000000000000000000000000000000000000000000000000000000000007d4454490AfA139d89042247DE811c2B0d7aCDF20000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
  "gasLimit": "0x1fffffffffffff",
  "nonce": "0x0000000000000042",
  "mixhash": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "timestamp": "0x00"
}

and start command:

exec geth \
     --datadir $DATA_DIR \
     --networkid $CHAIN_ID \
     --mine \
     --miner.etherbase "0x7d4454490AfA139d89042247DE811c2B0d7aCDF2" \
     --unlock "0x7d4454490AfA139d89042247DE811c2B0d7aCDF2" \
     --password $PASSWORD_FILE \
     --allow-insecure-unlock \
     --nodiscover \
     --http \
     --http.addr 0.0.0.0 \
     --http.vhosts=* \
     --http.port 8545 \
     --http.api eth,web3,net"

After deploying a smart contract that emits events I create a filter with:

curl -X POST \                                                                                          
  -H "Content-Type: application/json" \
  --data '{"jsonrpc":"2.0","method":"eth_newFilter","params":[{"fromBlock": "earliest", "toBlock": "latest", "address": "0xa513E6E4b8f2a923D98304ec87F64353C4D5C853","topics": ["0x8955a20de0ce0688e9ee8f31e787dfa365d9f8420c8565c945af9c18695469e0"]}],"id":2}' \
  http://localhost:8545

then I query the updates

curl -X POST \                                    
  -H "Content-Type: application/json" \
  --data '{"jsonrpc":"2.0","method":"eth_getFilterChanges","params":["0xb05d7da336bc4581e060a834f321eb54"],"id":1}' http://localhost:8545

If these actions are done right after the event is emitted all works fine and the logs are returned, if they are done a bit time later eth_getFilterChanges returns an empty array.

Note: this works fine with anvil.

[mmsqe]

May I clarify that you can getFilterChanges after events emitted when filter is created afterwards? From our test, logs are returned if filter is created before events emitted which is opposite to your description.

[fgimenez]

Thanks for your response @mmsqe sorry if I haven't been clear, you are correct, if the filter is created before the events are emitted all works fine. But if the filter (with fromBlock: earliest) is created after the events are emitted, then the logs are not always returned, only if the filter is created right after the events were emitted.

I would expect that, if the filter is created with fromBlock: earliest, when the filter is created after the events are emitted it should also return the events, maybe this is not the case?

In the docs I see that earliest is commonly used for the earliest/genesis block, but in eth_newFilter and eth_getLogs it says "pending", "earliest" for not yet mined transactions. https://ethereum.org/developers/docs/apis/json-rpc#eth_newfilter So if earliest in this case means not yet mined transactions, how can I query old logs? FWIW eth_getLogs actually returns old logs if fromBlock: earliest is specified.

[akaladarshi]

Hey @fgimenez,

The documentation mentioned that eth_newFilter notifies when the state is changed. So I guess if you create the filter before the block is produced (state changes) then only it will notify(return logs) and eth_getFilterChanges is basically the way to get those new logs.

Also, To get the logs eth_getLogs seems the best option. It allows for retrieving logs for a specific contract right from the genesis block (using fromBlock: earliest). You can continually update the fromBlock parameter to fetch new logs as new blocks are generated.