Remove unneeded partitioning

[Fokko]

Speeds up the integration tests

[kevinjqliu]

Looks like some tests are failing due to number of partitions being hardcoded in tests

=========================== short test summary info ============================
FAILED tests/integration/test_reads.py::test_pyarrow_deletes[session_catalog_hive] - assert [4, 5, 6] == [1, 2, 3]
  At index 0 diff: 4 != 1
  Full diff:
  - [1, 2, 3]
  + [4, 5, 6]
FAILED tests/integration/test_reads.py::test_pyarrow_deletes[session_catalog] - assert [4, 5, 6] == [1, 2, 3]
  At index 0 diff: 4 != 1
  Full diff:
  - [1, 2, 3]
  + [4, 5, 6]
FAILED tests/integration/test_reads.py::test_pyarrow_deletes_double[session_catalog_hive] - assert [10] == [5]
  At index 0 diff: 10 != 5
  Full diff:
  - [5]
  + [10]
FAILED tests/integration/test_reads.py::test_pyarrow_deletes_double[session_catalog] - assert [10] == [5]
  At index 0 diff: 10 != 5
  Full diff:
  - [5]
  + [10]
= 4 failed, 812 passed, 8 skipped, 2859 deselected, 2 warnings in 286.99s (0:04:46) =

[kevinjqliu]

Looks like modifying provision.py to include this part is what we need. I double checked by running integration tests locally.

spark = (
    SparkSession
        .builder
        .config("spark.sql.shuffle.partitions", "1")
        .config("spark.default.parallelism", "1")
        .getOrCreate()
)

from https://github.com/apache/iceberg-rust/pull/826/files#r1892947712

[Fokko]

@kevinjqliu Nice one! The partitioning would shuffle all the records based on year, so they would fall into the same bucket. I think the config is much cleaner, thanks @kevinjqliu! 🙌

[sungwy]

Thank you for cleaning this up, and thank you @kevinjqliu for the review!