Support for listening on Unix sockets #545
Comments
|
@SergioBenitez I would be interested in taking this issue on. I will try to get this done by the end of the reading week. |
|
@piperRyan Sure, that would be great! Before you start writing code, please post a detailed description of your design plans. In particular, you should address how you're going to tackle this problem internally and how the new functionality will be exposed to the users. The former can be somewhat vague but the latter should be crystal clear. You should include example code and/or configuration files. This also gives you an opportunity to post any questions you might have about how to approach this. I'll then review the plan and suggest changes, if any, that will make it more likely for your contribution to be accepted without any major changes. |
|
@SergioBenitez unfortunately, some stuff came up so I may not be able to hit my desired target. However, this is a early formation of my plan. I still want to research some other implementation to see how they are handling this case. That being said I would still appreciate some feedback. Files Expected Change
General Design Plan
How this will exposed to the end user
will generate the socket file, but pointing any traffic should be left to the user as it depend fairly widely on how the end user wants to do so be it with nginx, netstat, socat or whatever.
#[derive(Clone)]
pub struct ConfigBuilder {
/// The environment that this configuration corresponds to.
pub environment: Environment,
/// The path to create the socket.
pub socket_path: Path,
/// The number of workers to run in parallel.
pub workers: u16,
/// How much information to log.
pub log_level: LoggingLevel,
/// Size limits.
pub limits: Limits,
/// Any extra parameters that aren't part of Rocket's config.
pub extras: HashMap<String, Value>,
/// The root directory of this config.
pub root: PathBuf,
}
impl ConfigBuilder {
pub fn new(environment: Environment) -> ConfigBuilder { // ... }
pub fn socket_path<P: <AsRef<Path>>>(mut self, socket_path: P) -> Self { //... }
pub fn workers(mut self, workers: u16) -> Self { // ... }
pub fn log_level(mut self, log_level: LoggingLevel) -> Self { // ... }
pub fn limits(mut self, limits: Limits) -> Self { // ... }
pub fn environment(mut self, env: Environment) -> Self { // ... }
pub fn root<P: AsRef<Path>>(mut self, path: P) -> Self { // ... }
pub fn extra<V: Into<Value>>(mut self, name: &str, value: V) -> Self { // ... }
pub fn finalize(self) -> Result<Config> { // ... }
pub fn unwrap(self) -> Config { // ... }
pub fn expect(self, msg: &str) -> Config { // ... }
}The same thought process for the Config setup: #[derive(Clone)]
pub struct Config {
/// The environment that this configuration corresponds to.
pub environment: Environment,
/// The path to the socket.
pub socket_path: Path,
/// The number of workers to run concurrently.
pub workers: u16,
/// How much information to log.
pub log_level: LoggingLevel,
/// Streaming read size limits.
pub limits: Limits,
/// Extra parameters that aren't part of Rocket's core config.
pub extras: HashMap<String, Value>,
/// The path to the configuration file this config belongs to.
pub config_path: PathBuf,
}
impl Config {
pub fn build(env: Environment) -> ConfigBuilder { // ... }
pub fn new(env: Environment) -> Result<Config> { // ... }
pub fn development() -> Result<Config> { // ... }
pub fn staging() -> Result<Config> { // ... }
pub fn production() -> Result<Config> { // ... }
pub(crate) fn default<P>(env: Environment, path: P) -> Result<Config> { // ... }
pub(crate) fn bad_type(&self,name: &str, actual: &'static str, expect: &'static str) -> ConfigError { // ... }
pub(crate) fn set_raw(&mut self, name: &str, val: &Value) -> Result<()> {// .... }
pub fn set_root<P: AsRef<Path>>(&mut self, path: P) { // .... }
pub fn set_socket_path<P: AsRef<Path>>(&mut self, socket_path: P) -> Result<()> { // ... }
pub fn set_workers(&mut self, workers: u16) { // ... }
pub fn set_log_level(&mut self, log_level: LoggingLevel) { // ... }
pub fn set_limits(&mut self, limits: Limits) { // ... }
pub fn set_extras(&mut self, extras: HashMap<String, Value>) { // ... }
pub fn extras<'a>(&'a self) -> impl Iterator<Item=(&'a str, &'a Value)> { // ... }
pub fn get_str<'a>(&'a self, name: &str) -> Result<&'a str> {}
pub fn get_string(&self, name: &str) -> Result<String> {// ... }
pub fn get_int(&self, name: &str) -> Result<i64> { // ... }
pub fn get_bool(&self, name: &str) -> Result<bool> { // ... }
pub fn get_float(&self, name: &str) -> Result<f64> { // ... }
pub fn get_slice(&self, name: &str) -> Result<&Array> {// ... }
pub fn get_table(&self, name: &str) -> Result<&Table> { // ... }
pub fn get_datetime(&self, name: &str) -> Result<&Datetime> { //... }
pub fn root(&self) -> &Path { // ... }
pub fn root_relative<P: AsRef<Path>>(&self, path: P) -> PathBuf {// ... }
impl fmt::Debug for Config {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { // ... }
}
impl PartialEq for Config {
fn eq(&self, other: &Config) -> bool { // ... }
}So my intention here is to have the syntax/flow as similar as possible. However, it might change a bit depending how socket creation/cleanup is done.
|
|
@piperRyan Thanks for the write-up!
For the most part, using
My take is that it would be easier and more maintainable to do what Also, I don't think any of the logging stuff should be an issue. What potential issues do you see there?
This is a great point, and I'm happy to see you're thinking about it. I think the most natural thing for Rocket to do is to fail on start-up if a socket already exists, create a socket on start-up if it doesn't, and delete the socket on shut-down. This depends on #180, of course, but I would love to see that solved as well. I think leaving unused socket files around is a bad idea, and we shouldn't roll this out unless we can clean up after ourselves. As a proof-of-concept, I'd be okay leaving this out. But it would need to be present for a PR to be merged.
I don't think sockets should be behind a feature flag. If you're compiling on a platform that supports them (by the way, presumably none of this will work on Windows, right?), you should have the option. The code cost to support them shouldn't be high, so a feature flag just adds friction.
You've proposed adding a [development]
socket_path = "/my/socket"
address = "0.0.0.0"That is, specify both an [development]
address = "unix:///my/socket"If we do make this change, we probably want to also allow: [development]
address = "0.0.0.0" # serve over TCP by default
address = "tcp://0.0.0.0" # serve over TCP
address = "udp://0.0.0.0" # serve over UDP...if a user wants itThen specifying [development]
address = "unix:///my/socket"
port = 8000 # error: address type 'unix' does not accept a portAlso, how will using a unix domain socket interact with TLS? My intuition is that if TLS is enabled, we should do TLS over the unix domain socket. This should be easy to handle. Finally, the big change in Rocket is likely to be in the |
Dang, that is right I forgot about that.
I don't see a potential issue per se, but rust doesn't really have any test hooks so being able to assess the internal state of the function is a powerful tool to have for debugging. However, I could lean either way on this issue as it is very rare that debugging needs to be done from a stream/socket level if is done right.
To be honest I am not sure Windows did not to long ago announce support for unix sockets. However, I haven't put much research into that blog post. Maybe this needs to be tested? It would be a interesting development to say the least. Regardless, I would say it is safe to assume that a vast majority of individuals will not have access to unix sockets on a windows machine.
Yea, that is a way better idea! Perhaps a simplified Enum similar to hyper's Uri struct or the rust url's url struct would be valuable here, but I will have to think on that a bit to see if there is any value in doing so.
I agree, this is one the things I wanted to check out in different projects since a couple of them that I have looked at are suggesting using socat or using Apache or nginx in front which should handle the TLS related logic. A decision I don't really agree with, but I wanted to understand the rationale behind before committing any support to TLS. However, if you want it then I will ensure it works correctly. I will probably start thinking the types of testing that needs to be done in addition to what is present before I start coding. I will update the post with testing matrix when I get some time. |
|
@piperRyan How is your implementation going? I have just put together a proof of concept implementation in #594 |
|
@messense unfortunately, very slowly I had a few people drop from a coarse which had group project in it so accordingly I have been super busy trying to handle that. However, I will have a look at that proof concept you submitted! Regardless, thank you for picking up my slack :P. |
|
I would very much like to see this issue resolved soon. My new rust project requires that we write "external plugins" that are essentially an HTTP server that listens on a Unix domain socket for HTTP requests. |
|
I came across this issue when trying to find a way to use catflap with Rocket in order to get Before this issue / PR is merged, is there another good way to achieve what I want? I suppose I could bind the server on port zero to bind on an unused high port, although that adds a little bit of cognitive overhead in the form of remembering to change the port numbers in my curls / browser visits. |
|
Oh, actually, got it working like this: as suggested here: #163 (comment) |
|
Possibly (I've not looked at the Rocket internals at all!), a way to resolve this (and also provide additional functionality) would be to allow Rocket to accept an already bound socket listener to be passed to it from its parent OS process in the form of an open file descriptor? This was the jist of the "catflap" comment above, but to expand on why this might be a good idea... This would allow for easy integration with e.g. inetd, Apple launchd, and systemd (socket activation mode), and allow for:
Some references: |
|
Supporting UNIX domain sockets would be really awesome -- they are needed to use Rocket for writing Docker plugins. Without them, we are forced to use actix-web for now. |
|
The PR (#594) seems to have gotten derailed in large part due to supporting Unix sockets on Windows.. why is that a priority?? I could try to make an MVP with support for Unix sockets on Unix only, if there's still interest in this feature. |
|
Is UDP listening important to anyone? If not I'll drop it for the MVP |
|
@mattfbacon I have a lot of intereset in this feature. |
|
I want to tackle this as part of #1070 wherein the default listener knows about Unix domain sockets, for both Windows and *nix, as well as TCP. Closing this as a result. |
|
How do you measure how much overhead is gone( ping, memory foot print, and network traffic wise ) by using unix sockets opposed to loading the entire network stack? My typical use case would be to "mount" the unix socket on a network path in a reverse proxy such as nginx. As pr example: https://www.youtube.com/watch?v=m8ogrogKjXo&t=145s |
|
There's a benchmark to measure the difference between TCP and Unix socket on the same simple server |
I would like to utilize my existing infrastructure on atthost.pl. They are running nginx proxy and allowing me to bind it to any service exposing Unix socket (so I can run servers written in Nodejs, RoR, Golang etc.). The proxy handles SSL/TLS and communicates with servers through socket file. I cannot f.e. open a random port and use
socatas proxy between TCP and Unix sockets (due to missingnetstatcommand).Rust
stdhas UnixListener, buthyper0.10.x does not. Should anyone find time to implement this feature, here is some relevant code:hyperlocalcrate for hyper 0.10.x: https://github.com/softprops/hyperlocalThe text was updated successfully, but these errors were encountered: