Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: The get_compaction_plans() and get_compaction_state() interfaces cannot be used independently #2593

Open
1 task done
zhuwenxing opened this issue Jan 23, 2025 · 1 comment
Assignees
Labels
kind/bug Something isn't working

Comments

@zhuwenxing
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

The get_compaction_plans() and get_compaction_state() interfaces cannot be used independently


testcases/test_collection_operations.py:1614 (TestCollectionMaintenance.test_collection_compact)
self = <test_collection_operations.TestCollectionMaintenance object at 0x1284695d0>

    def test_collection_compact(self):
        """
        target: test collection compact
        method: create collection, insert data, flush multiple times, then compact
        expected: compact successfully
        """
        # Create collection
        name = gen_collection_name()
        client = self.collection_client
        vector_client = self.vector_client
        payload = {
            "collectionName": name,
            "schema": {
                "fields": [
                    {"fieldName": "book_id", "dataType": "Int64", "isPrimary": True, "elementTypeParams": {}},
                    {"fieldName": "my_vector", "dataType": "FloatVector", "elementTypeParams": {"dim": 128}}
                ]
            }
        }
        client.collection_create(payload)
    
        # Insert and flush multiple times
        for i in range(3):
            # Insert data
            vectors = [gen_vector(dim=128) for _ in range(10)]
            insert_data = {
                "collectionName": name,
                "data": [
                    {
                        "book_id": i * 10 + j,
                        "my_vector": vector
                    }
                    for i, vector in enumerate(vectors)
                    for j in range(10)
                ]
            }
            response = vector_client.vector_insert(insert_data)
            assert response["code"] == 0
    
            # Flush after each insert
            c = Collection(name)
            c.flush()
    
        # Compact collection
        response = client.compact(name)
        assert response["code"] == 0
    
        # Get compaction state
        response = client.get_compaction_state(name)
        assert response["code"] == 0
        assert "state" in response["data"]
        assert "compactionID" in response["data"]
        # verify by pymilvus
        c = Collection(name)
>       res = c.get_compaction_plans()

test_collection_operations.py:1669: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Collection>:
-------------
<name>: test_collection_2025_01_23_16_38_52_591233pzxRGbBb
<description>: 
<schema>: {'aut...or', 'description': '', 'type': <DataType.FLOAT_VECTOR: 101>, 'params': {'dim': 128}}], 'enable_dynamic_field': False}

timeout = None, kwargs = {}
conn = <pymilvus.client.grpc_handler.GrpcHandler object at 0x12846b190>

    def get_compaction_plans(self, timeout: Optional[float] = None, **kwargs) -> CompactionPlans:
        """Get the current compaction plans
    
        Args:
            timeout (``float``, optional): An optional duration of time in seconds to allow
                for the RPC. When timeout is set to None, client waits until server response
                or error occur.
        Returns:
            CompactionPlans: All the plans' states of this compaction.
        """
        conn = self._get_connection()
>       return conn.get_compaction_plans(self.compaction_id, timeout=timeout, **kwargs)
E       AttributeError: 'Collection' object has no attribute 'compaction_id'

While verifying the compact restful API, I needed to use pymilvus's get_compaction_plans() and get_compaction_state() for auxiliary checks, but encountered errors. The compaction_id variable is set in the compact() method, and get_compaction_plans() and get_compaction_state() cannot be executed without first running this method.

Also, I think there might be a missing interface for listing compact job IDs. If you have executed multiple compactions, how can you view the information of previous compact IDs?

Expected Behavior

Add an interface for listing compact jobs, and add compact ID as an input parameter for get_compaction_plans() and get_compaction_state() methods.

Steps/Code To Reproduce behavior

Environment details

- Hardware/Softward conditions (OS, CPU, GPU, Memory):
- Method of installation (Docker, or from source):
- Milvus version (v0.3.1, or v0.4.0):
- Milvus configuration (Settings you made in `server_config.yaml`):

Anything else?

No response

@zhuwenxing zhuwenxing added the kind/bug Something isn't working label Jan 23, 2025
@zhuwenxing
Copy link
Contributor Author

/assign @XuanYang-cn

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants