Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Expose Setting for GRPC Channel-Level Compression at Client Side #480

Merged
merged 14 commits into from
Mar 1, 2024

Conversation

geetu040
Copy link
Contributor

@geetu040 geetu040 commented Feb 4, 2024

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

  1. Does your submission pass tests?
  2. Have you installed pre-commit with pip3 install pre-commit and set up hooks with pre-commit install?

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

Issue

This PR refers to this issue #87 . It enables client to set GZip, grpc compression algorithm. By default no algorithm is applied.

Following code was modified

  • 1st commit: functions in qdrant_client/connection.py
    • get_channel
    • get_async_channel
  • 2nd commit: remote classes
    • class QdrantRemote in qdrant_client/qdrant_remote.py
    • class AsyncQdrantRemote in qdrant_client/async_qdrant_remote.py
  • 3rd commit: client classes and grpc compression enum
    • class QdrantClient in qdrant_client/qdrant_client.py
    • class AsyncQdrantClient in qdrant_client/async_qdrant_client.py
    • qdrant_client/grpc/__init__.py

Usage

Gzip compression algorithm

from qdrant_client import QdrantClient
from qdrant_client.grpc import Compression

client = QdrantClient(
	prefer_grpc=True,
	grpc_compression=Compression.Gzip
)

No compression algorithm

from qdrant_client import QdrantClient
from qdrant_client.grpc import Compression

# client = QdrantClient(prefer_grpc=True)
# client = QdrantClient(prefer_grpc=True, grpc_compression=None)
client = QdrantClient(prefer_grpc=True, grpc_compression=Compression.NoCompression)

Copy link

netlify bot commented Feb 4, 2024

Deploy Preview for poetic-froyo-8baba7 ready!

Name Link
🔨 Latest commit 35b67fd
🔍 Latest deploy log https://app.netlify.com/sites/poetic-froyo-8baba7/deploys/65e20eaf4cd093000879913e
😎 Deploy Preview https://deploy-preview-480--poetic-froyo-8baba7.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@geetu040 geetu040 changed the title Grpc compress Expose Setting for GRPC Channel-Level Compression Algorithm at Client Side Feb 4, 2024
@geetu040 geetu040 changed the title Expose Setting for GRPC Channel-Level Compression Algorithm at Client Side Expose Setting for GRPC Channel-Level Compression at Client Side Feb 4, 2024
@joein
Copy link
Member

joein commented Feb 4, 2024

Hi @geetu040

Thanks for the contribution!

Do you think it would be possible to add some tests?

@geetu040
Copy link
Contributor Author

geetu040 commented Feb 4, 2024

@joein Thanks for seeing, I have added some test cases

@geetu040 geetu040 changed the title Expose Setting for GRPC Channel-Level Compression at Client Side feat: Expose Setting for GRPC Channel-Level Compression at Client Side Feb 5, 2024
@geetu040
Copy link
Contributor Author

geetu040 commented Feb 5, 2024

This PR Closes #87

@geetu040
Copy link
Contributor Author

geetu040 commented Feb 7, 2024

Hi @joein
If you think the changes are all good to go, can you merge them?

@joein
Copy link
Member

joein commented Feb 15, 2024

Hey

Thanks for the contribution and sorry for the late response

Actually we would not like to extend client's signature with this parameter, since the signature is already large.
We haven't seen a big demand for this option either.
Thus, we would like to keep it in **kwargs

@joein joein self-requested a review February 15, 2024 11:37
@geetu040
Copy link
Contributor Author

Hey @joein
I have moved grpc_compression parameter from client's signature to **kwargs

qdrant_client/grpc/__init__.py Outdated Show resolved Hide resolved
@joein
Copy link
Member

joein commented Mar 1, 2024

eh, found a discrepancy between pre-commit setup and async-client-generator, should be fixed by now

@joein
Copy link
Member

joein commented Mar 1, 2024

@geetu040 thanks for the contribution!

@joein joein merged commit 3ad05b3 into qdrant:dev Mar 1, 2024
9 checks passed
joein added a commit that referenced this pull request Mar 5, 2024
#480)

* expose grpc channel-level compression settings in base functions

* expose grpc channel-level compression settings in remote classes

* expose grpc channel-level compression settings in client

* raise TypeError for compression

* added test cases for grcp channel-level compression

* move grpc_compression parameter from client's signature to **kwargs

* use grpc.Compression instead of creating new enum qdrant.grpc.Compression in qdrant/grpc/__init__.py

* refactor grpc_compression type hint

* fix: Compression instead of grpc.Compression in type hint

* tests: move and update tests

* chore: remove magic method

* fix: fix async client generator, update precommit dependencies

* fix: update isort options

* fix: update dev dependencies

---------

Co-authored-by: George Panchuk <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants