Go-ethereum rpc server have a request limit, how to expand the limit?

[VinceXie]

I have done a test for ethereum about ethereum rpc handle ability. I found in the case when post many requests in short time . Ethereum cant accept this request because rpc server not work.

After a while , rpc server was back to good status . But repeat many post , rpc server change to not work again.

Please help me.

System information

Geth version: All version
OS & Version: Windows/Linux/OSX
Commit hash : (if develop)

Expected behaviour

sendTransaction is good.

2018-06-25T07:24:12.689Z-------5396
0xfab1899742f3610598f147bbd0c6be14ad7af5094ef3915b8abbad0c349003fd
2018-06-25T07:24:12.690Z-------5397
0x78264d3e5da581cc97b2d0664a3004d58d74ada499dc3da4c19f53243c0d5b23
2018-06-25T07:24:12.691Z-------5398
0xb2f358478c51b164e729ae2e71cdafac0930d074880d0da559e0a60ef514f0df
2018-06-25T07:24:12.699Z-------5399
0x09bc6cdc0f8857ff09fdecb52c056cc6ce6f39d2f90b2912ede320ee6e9cd39e

Actual behaviour

2018-06-25T07:24:12.705Z-------5401

Error: Invalid JSON RPC response: ""
    at Object.InvalidResponse (/Users/vincexie/WebstormProjects/server/node_modules/web3-core-helpers/src/errors.js:42:16)
    at XMLHttpRequest.request.onreadystatechange (/Users/vincexie/WebstormProjects/server/node_modules/web3-providers-http/src/index.js:73:32)
    at XMLHttpRequestEventTarget.dispatchEvent (/Users/vincexie/WebstormProjects/server/node_modules/xhr2/lib/xhr2.js:64:18)
    at XMLHttpRequest._setReadyState (/Users/vincexie/WebstormProjects/server/node_modules/xhr2/lib/xhr2.js:354:12)
    at XMLHttpRequest._onHttpRequestError (/Users/vincexie/WebstormProjects/server/node_modules/xhr2/lib/xhr2.js:544:12)
    at ClientRequest.<anonymous> (/Users/vincexie/WebstormProjects/server/node_modules/xhr2/lib/xhr2.js:414:24)
    at ClientRequest.emit (events.js:180:13)
    at Socket.socketErrorListener (_http_client.js:395:9)
    at Socket.emit (events.js:180:13)
    at emitErrorNT (internal/streams/destroy.js:64:8)
    at process._tickCallback (internal/process/next_tick.js:178:19)

Steps to reproduce the behaviour

Here my test code , good in start , but fail when transaction to 6000.

var Web3 = require('web3');
var web3 = new Web3('http://127.0.0.1:20001');

//直接转账 必须解锁
var accounts = [];
web3.eth.getAccounts().then(function (res) {
    accounts = res
    console.log(accounts)

    sendTxRecur(0)
    sendTxRecur(0)
    sendTxRecur(0)
    sendTxRecur(0)
    sendTxRecur(0)
    sendTxRecur(0)
    sendTxRecur(0)
    sendTxRecur(0)
    sendTxRecur(0)
    sendTxRecur(0)

}).catch(function (err) {
    console.error(err)

})


var j = 0;
//用while并发会出现堵塞，用递归控制并发
var sendTxRecur = function (i) {
    web3.eth.sendTransaction({from:accounts[0], to:"0x810E626827eAFF977Fa1b00E19389220672aEB34", value:1},function (err,hash) {
        console.log(new Date().toISOString()+"-------" + j)
        if(err){
            console.error(err)
        }
        else{
            console.log(hash)
        }
        i = i + 1;
        j++;
        if(i>=10000){
            return
        }
        else{
            sendTxRecur(i)
        }
    })
}

Backtrace

[backtrace]

[karalabe]

Make sure you're reusing HTTP connections and not creating a new one for every request. If you make a new TCP connection for each request, and you concurrently do tons of requests, the OS will run out of sockets/file descriptors and will deny connecting.

It would help diagnose if you reported the error too. The empty JSON response probably also entails a socket level error that web3 is hiding.

[VinceXie]

@karalabe How to reuse HTTP connections ? I'm using the web3.js. I'm a noob and can't find this part in web3.js doc.

[karalabe]

Not really sure tbh, I'd suggest asking in their repo https://github.com/ethereum/web3.js/

[CryptoKiddies]

@VinceXie you can use the batch transactions web3 method, although it's more efficient to send individual transactions. Under normal (even unusual) use, you won't send txns to the network at the rate a loop would, so I would avoid that pattern.

[jmooo]

@karalabe using the go-ethereum library I also run out of ports on my local (~30k ports avail) when making calls out to geth via goroutines for the purposes of pulling all txn data into postgres. does every call like client.TransactionReceipt() or transaction.Hash() make a fresh JSON RPC call into the geth client? Pulling a single block from geth into a database seems to open hundreds of local ports all to 8545 on geth

EDIT: so it looks like the initial BlockByNumber is 2 calls, then for every transaction TransactionReceipt adds 1 more call. So for a 1000 txn block I'm looking at a total of 1002 ports opened/closed? I believe that gets me all uncles/log data as well without further RPC calls & port opening

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[noprom]

@jmooo Have you resolved this issue?