Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graceful shutdown #11

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/build.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Given no arguments, builds all packages. Otherwise, builds only the specified pa
// TODO: Parallelism.
for _, pkg := range packages {
buildOpts.Package = pkg
if err := internal.Build(repo, buildOpts); err != nil {
if err := internal.Build(cmd.Context(), repo, buildOpts); err != nil {
return err
}
}
Expand Down
14 changes: 13 additions & 1 deletion cmd/run.go
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
package cmd

import (
"context"
"errors"
"fmt"
"os"
"os/exec"
"os/signal"
"path/filepath"
"syscall"

"github.com/deref/uni/internal"
"github.com/spf13/cobra"
Expand Down Expand Up @@ -61,7 +64,16 @@ export const main = async (...args: string[]) => {

runOpts.Args = args[1:]

err = internal.Run(repo, runOpts)
ctx, cancel := context.WithCancel(cmd.Context())
shutdown := make(chan os.Signal, 1)
signal.Notify(shutdown, syscall.SIGINT)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to handle SIGKILL too?

go func() {
_ = <-shutdown
fmt.Fprintf(os.Stderr, "Received shutdown signal. Waiting for process to finish.\n")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is kinda a lie, if we've got a timeout and then kill without waiting. Also, if we're logging this, we probably should log when that timeout fires. If we have both of those logging things, maybe they belong close together in the code?

cancel()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two things going on here that are not "library-safe", but I think it's OK because this is "cmd" code and not "internal" code.

  1. I believe you're supposed to defer cancel() right after you call WithCancel, so that you can cleanup any channel resources that are allocated in order to make cancelling possible.
  2. This go routine may block on shutdown forever.

Not sure there is any action for this comment, just thinking aloud as I review this code.

}()

err = internal.Run(ctx, repo, runOpts)
var exitErr *exec.ExitError
if errors.As(err, &exitErr) {
os.Exit(exitErr.ExitCode())
Expand Down
7 changes: 4 additions & 3 deletions internal/build.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package internal

import (
"context"
"fmt"
"io/ioutil"
"os"
Expand All @@ -17,7 +18,7 @@ type BuildOptions struct {
Watch bool
}

func Build(repo *Repository, opts BuildOptions) error {
func Build(ctx context.Context, repo *Repository, opts BuildOptions) error {
pkg := opts.Package

packageDir := path.Join(repo.OutDir, "dist", pkg.Name)
Expand Down Expand Up @@ -161,7 +162,7 @@ void (async () => {
},
}
},
}.Run()
}.Run(ctx)
}

type funcProcess struct {
Expand All @@ -172,7 +173,7 @@ func (proc *funcProcess) Start() error {
return proc.start()
}

func (proc *funcProcess) Kill() error {
func (proc *funcProcess) Stop() error {
return nil
}

Expand Down
31 changes: 23 additions & 8 deletions internal/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,27 +18,30 @@
package internal

import (
"context"
"fmt"
"io/ioutil"
"os"
"os/exec"
"path"
"runtime"
"syscall"
"time"

"github.com/evanw/esbuild/pkg/api"
)

var maxProcStopWait = 5 * time.Second

type RunOptions struct {
Watch bool
Entrypoint string
Args []string
BuildOnly bool
}

// TODO: Need to handle interrupts in order to have a higher chance
// of cleaning up temporary files.

// Status code may be returend within an exec.ExitError return value.
func Run(repo *Repository, opts RunOptions) error {
// Status code may be returned within an exec.ExitError return value.
func Run(ctx context.Context, repo *Repository, opts RunOptions) error {
if err := EnsureTmp(repo); err != nil {
return err
}
Expand Down Expand Up @@ -118,10 +121,11 @@ if (typeof main === 'function') {
node.Stdin = os.Stdin
node.Stdout = os.Stdout
node.Stderr = os.Stderr
node.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with this incantation. A comment would help.


return &cmdProcess{cmd: node}
},
}.Run()
}.Run(ctx)
}

type cmdProcess struct {
Expand All @@ -132,11 +136,22 @@ func (proc *cmdProcess) Start() error {
return proc.cmd.Start()
}

func (proc *cmdProcess) Kill() error {
func (proc *cmdProcess) Stop() error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b/c of the timeout, this really StopAndThenKill(), instead of either Stop or Kill. Maybe it's Kill(gracePeriod)?

if proc.cmd.Process == nil {
return nil
}
return proc.cmd.Process.Kill()

if runtime.GOOS == "windows" {
return proc.cmd.Process.Kill()
}

go func() {
// TODO: Make the wait time configurable.
time.Sleep(maxProcStopWait)
_ = syscall.Kill(-proc.cmd.Process.Pid, syscall.SIGKILL)
}()

return syscall.Kill(-proc.cmd.Process.Pid, syscall.SIGTERM)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is strange to me. We async start a kill after a sleep, then we attempt a term and return that error code. Then the calling code needs the waitDone logic b/c this Stop function doesn't actually wait for the kill. If instead, this function is Kill(gracePeriod), and you do the term first, then race between proc.cmd.Wait() and time.Sleep(), now the call site can just be Kill(gracePeriod) and the waitDone thing is not necessary. Right?

}

func (proc *cmdProcess) Wait() error {
Expand Down
30 changes: 24 additions & 6 deletions internal/watch.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package internal

import (
"context"
"fmt"
"log"
"os"
Expand All @@ -22,10 +23,10 @@ type buildAndWatch struct {
type process interface {
Start() error
Wait() error
Kill() error
Stop() error
}

func (opts buildAndWatch) Run() error {
func (opts buildAndWatch) Run(ctx context.Context) error {
repo := opts.Repository

plugins := append([]api.Plugin{}, opts.Esbuild.Plugins...)
Expand Down Expand Up @@ -86,6 +87,11 @@ func (opts buildAndWatch) Run() error {
for {
proc := opts.CreateProcess()
done := make(chan error, 1)
waitDone := func() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is not necessary. See elsewhere.

if err := <-done; err != nil {
fmt.Fprintf(os.Stderr, "could not wait for process to finish: %v\n", err)
}
}

buildOK := len(result.Errors) == 0
shouldStart := buildOK && !waitForChange
Expand All @@ -104,8 +110,10 @@ func (opts buildAndWatch) Run() error {
}
select {
case <-abort:
if err := proc.Kill(); err != nil {
fmt.Fprintf(os.Stderr, "could not kill: %v\n", err)
if err := proc.Stop(); err != nil {
fmt.Fprintf(os.Stderr, "could not stop: %v\n", err)
} else {
waitDone()
}
return nil
case <-restart:
Expand All @@ -119,8 +127,10 @@ func (opts buildAndWatch) Run() error {
break loop
}
}
if err := proc.Kill(); err != nil {
fmt.Fprintf(os.Stderr, "could not kill: %v\n", err)
if err := proc.Stop(); err != nil {
fmt.Fprintf(os.Stderr, "could not stop: %v\n", err)
} else {
waitDone()
}
result = result.Rebuild()
waitForChange = false
Expand Down Expand Up @@ -152,9 +162,17 @@ func (opts buildAndWatch) Run() error {
close(abort)
return err
}
case <-ctx.Done():
close(abort)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewer note: this looks right b/c of the following return, so you can't unintentionally close the abort channel twice 👍🏻

return nil
}
}
})
} else {
go func() {
<-ctx.Done()
close(abort)
}()
}

return g.Wait()
Expand Down