Skip to content

Commit

Permalink
[SPARK-51061][CORE] Hide Jetty info in REST Submission API
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR aims to hide `Jetty` info in REST Submission API like SPARK-46239.
- #44158

### Why are the changes needed?

Start `Spark Master` server.
```
$ SPARK_MASTER_OPTS='-Dspark.master.rest.enabled=true' sbin/start-master.sh
```

**BEFORE:**
```
$ curl -vv http://0.0.0.0:6066/v1/submissions/readyz
*   Trying 0.0.0.0:6066...
* Connected to 0.0.0.0 (0.0.0.0) port 6066
> GET /v1/submissions/readyz HTTP/1.1
> Host: 0.0.0.0:6066
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< Date: Sun, 02 Feb 2025 05:14:04 GMT
< Content-Type: application/json;charset=utf-8
< Content-Length: 114
< Server: Jetty(11.0.24)
<
{
  "action" : "ReadyzResponse",
  "message" : "",
  "serverSparkVersion" : "4.1.0-SNAPSHOT",
  "success" : true
* Connection #0 to host 0.0.0.0 left intact
}
```

**AFTER**
```
$ curl -vv http://0.0.0.0:6066/v1/submissions/readyz
*   Trying 0.0.0.0:6066...
* Connected to 0.0.0.0 (0.0.0.0) port 6066
> GET /v1/submissions/readyz HTTP/1.1
> Host: 0.0.0.0:6066
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< Date: Sun, 02 Feb 2025 05:12:41 GMT
< Content-Type: application/json;charset=utf-8
< Content-Length: 114
<
{
  "action" : "ReadyzResponse",
  "message" : "",
  "serverSparkVersion" : "4.1.0-SNAPSHOT",
  "success" : true
* Connection #0 to host 0.0.0.0 left intact
}
```

### Does this PR introduce _any_ user-facing change?

No, there is no technical behavior change because this is a meta-data.

### How was this patch tested?

Manual test with the above procedure.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #49762 from dongjoon-hyun/SPARK-51061.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
dongjoon-hyun committed Feb 3, 2025
1 parent d29648d commit ac8f8b0
Showing 1 changed file with 9 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import scala.io.Source
import com.fasterxml.jackson.core.JsonProcessingException
import jakarta.servlet.DispatcherType
import jakarta.servlet.http.{HttpServlet, HttpServletRequest, HttpServletResponse}
import org.eclipse.jetty.server.{HttpConnectionFactory, Server, ServerConnector}
import org.eclipse.jetty.server.{HttpConfiguration, HttpConnectionFactory, Server, ServerConnector}
import org.eclipse.jetty.servlet.{FilterHolder, ServletContextHandler, ServletHolder}
import org.eclipse.jetty.util.thread.{QueuedThreadPool, ScheduledExecutorScheduler}
import org.json4s._
Expand Down Expand Up @@ -104,6 +104,13 @@ private[spark] abstract class RestSubmissionServer(
threadPool.setDaemon(true)
val server = new Server(threadPool)

// Hide information.
val httpConfig = new HttpConfiguration()
logDebug("Using setSendServerVersion: false")
httpConfig.setSendServerVersion(false)
logDebug("Using setSendXPoweredBy: false")
httpConfig.setSendXPoweredBy(false)

val connector = new ServerConnector(
server,
null,
Expand All @@ -112,7 +119,7 @@ private[spark] abstract class RestSubmissionServer(
null,
-1,
-1,
new HttpConnectionFactory())
new HttpConnectionFactory(httpConfig))
connector.setHost(host)
connector.setPort(startPort)
connector.setReuseAddress(!Utils.isWindows)
Expand Down

0 comments on commit ac8f8b0

Please sign in to comment.