GH-38341: [Python] Remove usage of pandas internals DatetimeTZBlock

[jorisvandenbossche]

Rationale for this change

This usage probably stems from a long time ago that it was required to specify the Block type, but nowadays it's good enough to just specify the dtype, and thus cutting down on our usage of internal pandas objects.

We only need to ensure the data passed is 2D, because it is always for a Block in a dataframe (not series), and otherwise it complains the placement doesn't match.

Part of #35081

• Closes: [Python] pandas internals: avoid using DatetimeTZBlock  #38341

[github-actions]

⚠️ GitHub issue #35081 has been automatically assigned in GitHub to PR creator.

[jorisvandenbossche]

So it seems that pandas doesn't preserve the unit in the DatetimeArray constructor:

In [4]: arr = np.array([1, 2, 3], dtype="datetime64[s]")

In [5]: dtype = pd.DatetimeTZDtype("s", tz="Europe/Brussels")

In [6]: pd.arrays.DatetimeArray(arr, dtype)
Out[6]: 
<DatetimeArray>
['1970-01-01 01:00:01+01:00', '1970-01-01 01:00:02+01:00',
 '1970-01-01 01:00:03+01:00']
Length: 3, dtype: datetime64[ns, Europe/Brussels]

This seems fixed on the last pandas release 2.1, so will have to keep the older code using internals for the older pandas versions.

[jorisvandenbossche]

@github-actions crossbow submit -g python

[github-actions]

Revision: a133014

Submitted crossbow builds: ursacomputing/crossbow @ actions-eda4ccd5da

Task	Status
test-conda-python-3.10
test-conda-python-3.10-cython2
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-latest
test-conda-python-3.10-pandas-nightly
test-conda-python-3.10-spark-v3.5.0
test-conda-python-3.10-substrait
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-upstream_devel
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.8
test-conda-python-3.8-pandas-1.0
test-conda-python-3.8-spark-v3.5.0
test-conda-python-3.9
test-conda-python-3.9-pandas-latest
test-cuda-python
test-debian-11-python-3
test-fedora-35-python-3
test-ubuntu-20.04-python-3
test-ubuntu-22.04-python-3

[github-actions]

⚠️ GitHub issue #38341 has been automatically assigned in GitHub to PR creator.

[jorisvandenbossche]

@github-actions crossbow submit -g python

[github-actions]

Revision: dfcfa22

Submitted crossbow builds: ursacomputing/crossbow @ actions-6fbf28b83d

Task	Status
test-conda-python-3.10
test-conda-python-3.10-cython2
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-latest
test-conda-python-3.10-pandas-nightly
test-conda-python-3.10-spark-v3.5.0
test-conda-python-3.10-substrait
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-upstream_devel
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.8
test-conda-python-3.8-pandas-1.0
test-conda-python-3.8-spark-v3.5.0
test-conda-python-3.9
test-conda-python-3.9-pandas-latest
test-cuda-python
test-debian-11-python-3
test-fedora-38-python-3
test-ubuntu-20.04-python-3
test-ubuntu-22.04-python-3

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 6b93c4a.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.