Support spark 3.4 (#81)

.github/CODEOWNERS

@@ -4,4 +4,4 @@
 # These owners will be the default owners for everything in
 # the repo. Unless a later match takes precedence, these people will be requests a review
 # review when someone opens a pull request.
-*       @jrbentzon @george-zubrienko
+* @SneaksAndData/platform-engineering

.github/workflows/build.yaml

@@ -13,7 +13,7 @@ jobs:
     if: ${{ github.ref != 'refs/heads/main' }}
 
     container:
-      image: esdcrdevelopment.azurecr.io/spark:v3.1.0-2-gcaf5773-bitnami-a0a12a1d-python-3.9.15
+      image: esdcrdevelopment.azurecr.io/spark:v3.1.1-5-g28008d0-bitnami-c31d3b46-python-3.9.16
       credentials:
         username: ${{ secrets.AZCR_DEV_USER }}
         password: ${{ secrets.AZCR_DEV_TOKEN }}

.github/workflows/prepare_release.yaml

@@ -16,4 +16,4 @@ jobs:
         uses: SneaksAndData/github-actions/[email protected]
         with:
           major_v: 0
-          minor_v: 5
+          minor_v: 6

README.md

@@ -11,10 +11,11 @@ pip install hadoop-fs-wrapper
 
 Select a version that matches hadoop version you are using:
 
-| Hadoop Version | Compatible hadoop-fs-wrapper version |
-|----------------|:------------------------------------:|
-| 3.2.x          |                0.4.x                 |
-| 3.3.x          |             0.4.x, 0.5.x             |
+| Hadoop Version / Spark version | Compatible hadoop-fs-wrapper version |
+|--------------------------------|:------------------------------------:|
+| 3.2.x / 3.2.x                  |                0.4.x                 |
+| 3.3.x / 3.3.x                  |             0.4.x, 0.5.x             |
+| 3.3.x / 3.4.x                  |                0.6.x                 |
 
 ## Usage
 Common use case is accessing Hadoop FileSystem from Spark session object:
@@ -44,4 +45,4 @@ def is_valid_source_path(file_system: FileSystem, path: str) -> bool:
 Currently basic filesystem operations (listing, deleting, search, iterative listing etc.) are supported. If an operation you require is not yet wrapped,
 please open an issue or create a PR.
 
-All changes are tested against Spark 3.2/3.3 running in local mode.
+All changes are tested against Spark 3.4 running in local mode.