Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added caching to the async session in request.py and AsyncHTTPProvider #2254

Merged
merged 27 commits into from
Feb 24, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
38f7e84
Added caching to the async session in request.py and AsyncHTTPProvider
dbfreem Dec 15, 2021
92ceac1
reverted pytest-mock upgrade
dbfreem Dec 16, 2021
ee778ae
Merge branch 'ethereum:master' into feature/async_session_cache
dbfreem Dec 16, 2021
229533f
removed pytest-mock upgrade from news
dbfreem Dec 16, 2021
801638b
Merge branch 'feature/async_session_cache' of https://github.com/dbfr…
dbfreem Dec 16, 2021
3cff06f
made small suggested changes
dbfreem Dec 18, 2021
00f2c67
typo
dbfreem Dec 18, 2021
ff12510
added thread lock to the async session cache
dbfreem Dec 18, 2021
b9dcffb
Merge branch 'ethereum:master' into feature/async_session_cache
dbfreem Jan 12, 2022
e7bbec5
removed lock around close since it was causing deadlocks
dbfreem Jan 14, 2022
097fa1b
accidently commited test that I was working on
dbfreem Jan 14, 2022
3d9e77b
Added SessionCache class
dbfreem Jan 16, 2022
eae163d
rearranged the cache code
dbfreem Jan 16, 2022
65b12f2
added one more test on new SessionCache
dbfreem Jan 16, 2022
f713776
lint
dbfreem Jan 16, 2022
9656ae8
Had the wrong import not sure how that worked locally
dbfreem Jan 16, 2022
88ee5ac
updated docs
dbfreem Jan 16, 2022
1720df3
fixed updating of existing cache items
dbfreem Jan 16, 2022
e307c8b
lint problem again
dbfreem Jan 16, 2022
4289e64
slight changes to docs
dbfreem Jan 16, 2022
8bdcbcd
Moved caching of session in AsynHTTPProvider to a method since it is …
dbfreem Jan 16, 2022
503146d
Merge branch 'ethereum:master' into feature/async_session_cache
dbfreem Jan 26, 2022
92a2b84
Merge branch 'ethereum:master' into feature/async_session_cache
dbfreem Feb 1, 2022
e1ff8fe
Merge branch 'ethereum:master' into feature/async_session_cache
dbfreem Feb 8, 2022
40ca5f9
Update docs/providers.rst
dbfreem Feb 24, 2022
cf29678
Update web3/_utils/request.py
dbfreem Feb 24, 2022
2d3a275
add _ for confusion reduction
Feb 24, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Moved caching of session in AsynHTTPProvider to a method since it is …
…async
  • Loading branch information
dbfreem committed Jan 16, 2022
commit 8bdcbcdd224710507b4e669e8f93cca0fb5f0d06
10 changes: 5 additions & 5 deletions docs/providers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,7 @@ AsyncHTTPProvider
be omitted from the URI.
* ``request_kwargs`` should be a dictionary of keyword arguments which
will be passed onto each http/https POST request made to your node.
* ``session`` allows you to pass in your own ``aiohttp.ClientSession`` object.
* the ``cache_async_session()`` method allows you to use your own ``aiohttp.ClientSession`` object. This is and async method and not part of the consturctor
dbfreem marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: python

Expand All @@ -388,17 +388,17 @@ AsyncHTTPProvider
>>> from web3.eth import AsyncEth
>>> from web3.net import AsyncNet

>>> custom_session = ClientSession() # If you want to pass in your own session
>>> w3 = Web3(
... AsyncHTTPProvider(endpoint_uri),
... modules={'eth': (AsyncEth,),
... 'net': (AsyncNet,),
... 'geth': (Geth,
... {'txpool': (AsyncGethTxPool,)})
... },
... middlewares=[], # See supported middleware section below for middleware options
... session = custom_session
)
... middlewares=[] # See supported middleware section below for middleware options
... )
>>> custom_session = ClientSession() # If you want to pass in your own session
>>> await w3.provider.cache_async_session(custom_session) # This method is an async method so it needs to be handled accordingly

Under the hood, the ``AsyncHTTPProvider`` uses the python
`aiohttp <https://docs.aiohttp.org/en/stable/>`_ library for making requests.
Expand Down
22 changes: 22 additions & 0 deletions tests/core/providers/test_async_http_provider.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@

import pytest

from aiohttp import (
ClientSession,
)

from web3._utils import (
request,
)
from web3.providers.async_rpc import (
AsyncHTTPProvider,
)


@pytest.mark.asyncio
async def test_user_provided_session() -> None:

session = ClientSession()
provider = AsyncHTTPProvider(endpoint_uri="http://mynode.local:8545")
await provider.cache_async_session(session)
assert len(request._async_session_cache) == 1
12 changes: 7 additions & 5 deletions web3/providers/async_rpc.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@
Union,
)

from aiohttp import (
ClientSession,
)
from eth_typing import (
URI,
)
Expand Down Expand Up @@ -40,8 +43,7 @@ class AsyncHTTPProvider(AsyncJSONBaseProvider):

def __init__(
self, endpoint_uri: Optional[Union[URI, str]] = None,
request_kwargs: Optional[Any] = None,
session: Optional[Any] = None
request_kwargs: Optional[Any] = None
) -> None:
if endpoint_uri is None:
self.endpoint_uri = get_default_http_endpoint()
Expand All @@ -50,11 +52,11 @@ def __init__(

self._request_kwargs = request_kwargs or {}

if session is not None:
cache_async_session(self.endpoint_uri, session)

super().__init__()

async def cache_async_session(self, session: ClientSession) -> None:
await cache_async_session(self.endpoint_uri, session)

def __str__(self) -> str:
return "RPC connection {0}".format(self.endpoint_uri)

Expand Down