diff --git a/README.md b/README.md index ae051460c..a56e9e5de 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ 2.4* - v2.1.0 + v2.1.1 3.0 diff --git a/benchmark/scala/pom.xml b/benchmark/scala/pom.xml index 44ddd8d22..4dd4207b0 100644 --- a/benchmark/scala/pom.xml +++ b/benchmark/scala/pom.xml @@ -3,7 +3,7 @@ 4.0.0 com.microsoft.spark microsoft-spark-benchmark - 2.1.0 + 2.1.1 2019 UTF-8 diff --git a/docs/release-notes/2.1.1/release-2.1.1.md b/docs/release-notes/2.1.1/release-2.1.1.md new file mode 100644 index 000000000..8ef69dd5a --- /dev/null +++ b/docs/release-notes/2.1.1/release-2.1.1.md @@ -0,0 +1,203 @@ +# .NET for Apache Spark 2.1.1 Release Notes + +### New Features/Improvements + +* Add base classes for ML and refine code base ([#1031](https://github.com/dotnet/spark/pull/1031), [#1049](https://github.com/dotnet/spark/pull/1049)) +* Capture dotnet application error stack trace ([#1047](https://github.com/dotnet/spark/pull/1047)) +* JvmBridge semaphore fix ([#1061](https://github.com/dotnet/spark/pull/1061)) + +### Bug Fixes + +* None + +### Infrastructure / Documentation / Etc. + +* Switch to 1ES hosted pools on main ([#1021](https://github.com/dotnet/spark/pull/1021)) + +### Breaking Changes + +* None + +### Known Issues + +* Broadcast variables do not work with [dotnet-interactive](https://github.com/dotnet/interactive) ([#561](https://github.com/dotnet/spark/pull/561)) +* UDFs defined using class objects with closures does not work with [dotnet-interactive](https://github.com/dotnet/interactive) ([#619](https://github.com/dotnet/spark/pull/619)) +* In [dotnet-interactive](https://github.com/dotnet/interactive) blocking Spark methods that require external threads to unblock them does not work. ie `StreamingQuery.AwaitTermination` requires `StreamingQuery.Stop` to unblock ([#736](https://github.com/dotnet/spark/pull/736)) + +### Compatibility + +#### Backward compatibility + +The following table describes the oldest version of the worker that the current version is compatible with, along with new features that are incompatible with the worker. + + + + + + + + + + + + +
Oldest compatible Microsoft.Spark.Worker version
v2.0.0
+ +#### Forward compatibility + +The following table describes the oldest version of .NET for Apache Spark release that the current worker is compatible with. + + + + + + + + + + + + +
Oldest compatible .NET for Apache Spark release version
v2.0.0
+ +### Supported Spark Versions + +The following table outlines the supported Spark versions along with the microsoft-spark JAR to use with: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Spark Versionmicrosoft-spark JAR
2.4.0microsoft-spark-2-4_2.11-2.1.1.jar
2.4.1
2.4.3
2.4.4
2.4.5
2.4.6
2.4.7
2.4.2Not supported
3.0.0microsoft-spark-3-0_2.12-2.1.1.jar
3.0.1
3.0.2
3.1.1microsoft-spark-3-1_2.12-2.1.1.jar
3.1.2
3.2.0microsoft-spark-3-2_2.12-2.1.1.jar
3.2.1
+ +### Supported Delta Versions + +The following table outlines the supported Delta versions along with the Microsoft.Spark.Extensions version to use with: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Delta VersionMicrosoft.Spark.Extensions.Delta
0.1.02.1.1
0.2.0
0.3.0
0.4.0
0.5.0
0.6.0
0.6.1
0.7.0
0.8.0
1.0.0
1.1.0
+ +### Supported Hyperspace Versions + +The following table outlines the supported Hyperspace versions along with the Microsoft.Spark.Extensions version to use with: + + + + + + + + + + + + + + + + + + + + + + + +
Hyperspace VersionMicrosoft.Spark.Extensions.Hyperspace
0.1.02.1.1
0.2.0
0.3.0
0.4.0
diff --git a/eng/Versions.props b/eng/Versions.props index 941719d67..e3a8cf7f9 100644 --- a/eng/Versions.props +++ b/eng/Versions.props @@ -1,7 +1,7 @@ - 2.1.0 + 2.1.1 prerelease diff --git a/src/scala/pom.xml b/src/scala/pom.xml index 50c618ba8..890af92b8 100644 --- a/src/scala/pom.xml +++ b/src/scala/pom.xml @@ -7,7 +7,7 @@ ${microsoft-spark.version} UTF-8 - 2.1.0 + 2.1.1