Add a new column from already computed arrow Table / Arrow

[eddyxu]

Problem Statement

Bring back the functionality added in #320 in Lance

    def merge(self,
      right: pyarrow.Table,
      left_on: str,
      right_on: str,
      metadata: Optional[Dict[str, str]] = None,
    )

To use it,

import lance

dataset = lance.dataset("/tmp/dataset")
new_table = pyarrow.Table.from_pandas(df)
dataset.merge(new_table, left_on="id", right_on="id")

Once the merge is successfully done, a new version of dataset is generated.

References:

• Format description: https://eto-ai.github.io/lance/format.html#dataset-update-and-schema-evolution
• https://github.com/eto-ai/lance/blob/78e5af7dec1a9602823797ba18eafc632ba727c3/python/lance/_lib.pyx#L406-L432
• https://github.com/eto-ai/lance/blob/78e5af7dec1a9602823797ba18eafc632ba727c3/cpp/include/lance/arrow/dataset.h#L221-L226
• https://github.com/eto-ai/lance/blob/78e5af7dec1a9602823797ba18eafc632ba727c3/cpp/src/lance/arrow/hash_merger.h#L38-L42