Skip to content

Commit

Permalink
feat: 增加 verbose 參數
Browse files Browse the repository at this point in the history
  • Loading branch information
shihyuho committed Jan 15, 2024
1 parent cabf395 commit eb598b0
Show file tree
Hide file tree
Showing 4 changed files with 65 additions and 44 deletions.
11 changes: 11 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
FROM golang:1.20-alpine AS builder
ENV GO111MODULE=on CGO_ENABLED=0
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY main.go .
RUN go vet && gofmt -s -w main.go && go build -o memory-calculator

FROM eclipse-temurin:17-jre-alpine
COPY --from=builder /app/memory-calculator /usr/local/bin/memory-calculator
COPY entrypoint.sh /tmp/entrypoint.sh
19 changes: 12 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 記憶體計算工具(Memory Calculator)

**記憶體計算工具**是為了協助 Java
** 記憶體計算工具 ** 是為了協助 Java
虛擬機(JVM)在運行時計算記憶體設定而開發的工具,基於 [paketo-buildpacks/libjvm](https://github.com/paketo-buildpacks/libjvm/)

## Calculation Algorithm
Expand Down Expand Up @@ -43,29 +43,34 @@ Non-Heap = Direct Memory + Metaspace + Reserved Code Cache + (Thread Stack * Thr

## Input Variables

在計算過程中, 需要依照特定的順序和邏輯使用多個參數, 以下是參數的取得順序及其說明:
在計算過程中需要依照特定的順序和邏輯使用多個參數以下是參數的取得順序及其說明:

| 參數說明 | 優先判斷 args 傳入 | 其次判斷 OS Variable | 最後的預設值或行為 |
|---|---|---|---|
| 記憶體計算工具分配的預留空間百分比 | `--head-room` | `$BPL_JVM_HEAD_ROOM` | `0` |
| 運行時將加載的 class 數量 | `--loaded-class-count ` | `$BPL_JVM_LOADED_CLASS_COUNT ` | 動態計算 App 目錄下 class 總數量的 35% |
| 運行時將加載的 class 數量 | `--loaded-class-count` | `$BPL_JVM_LOADED_CLASS_COUNT` | 動態計算 App 目錄下 class 總數量的 35% |
| 運行時的用戶線程數 | `--thread-count` | `$BPL_JVM_THREAD_COUNT` | `200` |
| App 目錄 | `--application-path` | | `/app` |
| VM 建立參數 | `--jvm-options` | `$JAVA_TOOL_OPTIONS` | |
| Java啟動參數 | | `$JAVA_OPTS ` | |
| Java 啟動參數 | | `$JAVA_OPTS` | |
| Java Home | | `$JAVA_HOME ` | |

執行以下指令以查看完整的 args 參數說明:

```sh
memory-calculator -h
```

## Entrypoint

[`entrypoint.sh`](./entrypoint.sh) 是一個專為使用 [Jib](https://github.com/GoogleContainerTools/jib) 打包的 Image 而設計的進入點,它在執行時會根據前面提到的[計算演算法](#calculation-algorithm)來計算出建議的記憶體配置,然後啟動 Java 應用程式。
[`entrypoint.sh`](./entrypoint.sh) 是一個專為使用 [Jib](https://github.com/GoogleContainerTools/jib) 打包的 Image 而設計的進入點,它在執行時會根據前面提到的 [計算演算法](#calculation-algorithm) 來計算出建議的記憶體配置,然後啟動 Java 應用程式。

使用 `entrypoint.sh` 的步驟如下:

1.`entrypoint.sh` 放入 Jib 所使用的 Base Image 中,例如放在 `/tmp` 資料夾下。
2. 在 Jib 的配置中,將 entrypoint 設定為 `/tmp/entrypoint.sh`

在 Jib 中若[自定義了 entrypoint](https://github.com/GoogleContainerTools/jib/tree/master/jib-maven-plugin#custom-container-entrypoint)`<jvmFlags>` 參數將無法被直接引用。因此,`entrypoint.sh` 還整合了公司開發的 [jib-jvm-flags-extension-maven](https://github.com/softleader/jib-jvm-flags-extension-maven)。藉由這個 Jib Extension,我們就可以繼續使用 `<jvmFlags>`

在 Jib 中若 [自定義了 entrypoint](https://github.com/GoogleContainerTools/jib/tree/master/jib-maven-plugin#custom-container-entrypoint)`<jvmFlags>` 參數將無法被直接引用。因此,`entrypoint.sh` 還整合了公司開發的 [jib-jvm-flags-extension-maven](https://github.com/softleader/jib-jvm-flags-extension-maven)。藉由這個 Jib Extension,我們就可以繼續使用 `<jvmFlags>`

### 支援的參數

Expand Down
6 changes: 5 additions & 1 deletion jib-demo/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,11 @@
<name>jib-demo</name>
<description>jib-demo</description>
<properties>
<container-base-image>harbor.softleader.com.tw/library/eclipse-temurin-alpine:17-jre-taipei</container-base-image>
<!--
1. 需要先到上一層目錄中,執行: docker build -t jib-demo .
2. 再回到這層目錄, 執行: mvn jib:dockerBuild
-->
<container-base-image>docker://jib-demo</container-base-image>
<container-image-project>demo</container-image-project>
<container-image-repository>harbor.softleader.com.tw</container-image-repository>
<java.version>17</java.version>
Expand Down
73 changes: 37 additions & 36 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,60 +7,57 @@ import (
"github.com/paketo-buildpacks/libpak/bard"
"github.com/paketo-buildpacks/libpak/sherpa"
"github.com/spf13/cobra"
"log"
"os"
"strconv"
"strings"
)

const (
envJavaHome = "JAVA_HOME"
envJavaToolOptions = "JAVA_TOOL_OPTIONS"
envBplJvmHeadRoom = "BPL_JVM_HEAD_ROOM"
envBplJvmThreadCount = "BPL_JVM_THREAD_COUNT"
envBpiApplicationPath = "BPI_APPLICATION_PATH"
envBpiJvmClassCount = "BPI_JVM_CLASS_COUNT"
envBpiMemoryLimitPathV2 = "BPI_MEMORY_LIMIT_PATH_V2"
defaultMemoryLimitPathV2Fix = "/sys/fs/cgroup/memory/memory.max_usage_in_bytes"
defaultJvmOptions = ""
defaultHeadRoom = 0
defaultThreadCount = 200
defaultApplicationPath = "/app"
desc = `This command calculate the JVM memory for applications to run smoothly and stay within the memory limits of the container.
During the computation process, numerous parameters are required, which must be obtained in a specific order and logic.
envJavaHome = "JAVA_HOME"
envJavaToolOptions = "JAVA_TOOL_OPTIONS"
envBplJvmHeadRoom = "BPL_JVM_HEAD_ROOM"
envBplJvmThreadCount = "BPL_JVM_THREAD_COUNT"
envBpiApplicationPath = "BPI_APPLICATION_PATH"
envBpiJvmClassCount = "BPI_JVM_CLASS_COUNT"
defaultJvmOptions = ""
defaultHeadRoom = helper.DefaultHeadroom
defaultThreadCount = 200
defaultApplicationPath = "/app"
desc = `This command calculate the JVM memory for applications to run smoothly and stay within the memory limits of the container.
During the computation process, numerous parameters are required, which must be obtained in a specific order and logic.
The sequence and explanations of these parameters are as follows:
1. Percentage of reserved space allocated by Memory Calculation tool:
- First, determine if '--head-room' is passed through args.
- If not, check the OS environment variable $BPL_JVM_HEAD_ROOM.
- If neither is available, the default value is 0.
2. Number of classes loaded at runtime:
- First, determine if '--loaded-class-count' is passed through args.
- If not, check the OS environment variable $BPL_JVM_LOADED_CLASS_COUNT.
- If neither is available, dynamically calculate 35% of the total number of classes in the App directory.
3. Number of user threads at runtime:
- First, determine if '--thread-count' is passed through args.
- If not, check the OS environment variable $BPL_JVM_THREAD_COUNT.
- If neither is available, the default value is 200.
4. App directory:
- First, determine if '--application-path' is passed through args.
- If not, the default directory is /app.
5. VM creation parameters:
- First, determine if '--jvm-options' is passed through args.
- If not, check the OS environment variable $JAVA_TOOL_OPTIONS.
6. Java startup parameters:
- Only check the OS environment variable $JAVA_OPTS.
7. Java home:
- Only check the OS environment variable $JAVA_HOME.
Examples:
# Use ZGC and output to /tmp/.env
# Use ZGC and output to /tmp/.env
memory-calculator --jvm-options '-XX:+UseZGC' -o '/tmp/.env'
# Print the version and exit
Expand All @@ -79,6 +76,7 @@ type Config struct {
memoryLimitPathV2 string
output string
version bool
verbose bool
}

func main() {
Expand All @@ -104,18 +102,18 @@ func main() {
flags.StringVar(&c.applicationPath, "application-path", c.applicationPath, "the directory on the container where the app's contents are placed")
flags.StringVarP(&c.output, "output", "o", c.output, "write to a file, instead of STDOUT")
flags.BoolVar(&c.version, "version", c.version, "print version and exit")
flags.BoolVarP(&c.verbose, "verbose", "v", c.verbose, "enable verbose output")
if err := cmd.Execute(); err != nil {
os.Exit(1)
}
}

func newConfig() Config {
c := Config{
jvmOptions: defaultJvmOptions,
headRoom: defaultHeadRoom,
threadCount: defaultThreadCount,
applicationPath: defaultApplicationPath,
memoryLimitPathV2: defaultMemoryLimitPathV2Fix,
jvmOptions: defaultJvmOptions,
headRoom: defaultHeadRoom,
threadCount: defaultThreadCount,
applicationPath: defaultApplicationPath,
}
if val, ok := os.LookupEnv(envJavaToolOptions); ok {
c.jvmOptions = val
Expand All @@ -132,13 +130,16 @@ func newConfig() Config {
if val, ok := os.LookupEnv(envBpiApplicationPath); ok {
c.applicationPath = val
}
// 修正部分記憶體限制檔案位置不一致問題
if val, ok := os.LookupEnv(envBpiMemoryLimitPathV2); ok {
c.memoryLimitPathV2 = val
}
return c
}

func (c *Config) newLogger() bard.Logger {
if c.verbose {
return bard.NewLoggerWithOptions(os.Stdout, bard.WithDebug(os.Stdout))
}
return bard.NewLogger(os.Stdout)
}

func (c *Config) prepareLibJvmEnv() (err error) {
if err = os.Setenv(envBplJvmThreadCount, c.jvmOptions); err != nil {
return err
Expand Down Expand Up @@ -175,13 +176,13 @@ func run(c Config) (err error) {
return err
}
var (
l = bard.NewLogger(os.Stdout)
l = c.newLogger()
a = helper.ActiveProcessorCount{Logger: l}
j = helper.JavaOpts{Logger: l}
m = helper.MemoryCalculator{
Logger: l,
MemoryLimitPathV1: helper.DefaultMemoryLimitPathV1,
MemoryLimitPathV2: c.memoryLimitPathV2,
MemoryLimitPathV1: helper.DefaultMemoryLimitPathV1, // cgroup v1 的記憶體上限路徑
MemoryLimitPathV2: helper.DefaultMemoryLimitPathV2, // cgroup v2 的記憶體上限路徑
MemoryInfoPath: helper.DefaultMemoryInfoPath,
}
)
Expand Down Expand Up @@ -209,7 +210,7 @@ func run(c Config) (err error) {
var javaToolOptions = os.Getenv(envJavaToolOptions)

if c.output == "" {
log.Printf("%v: %v\n", envJavaToolOptions, javaToolOptions)
l.Infof("%v: %v\n", envJavaToolOptions, javaToolOptions)
return nil
}

Expand All @@ -220,7 +221,7 @@ func run(c Config) (err error) {
defer func(file *os.File) {
err := file.Close()
if err != nil {
log.Printf("failed to close file %v: %v\n", file.Name(), err)
l.Infof("WARNING: failed to close file %v: %v\n", file.Name(), err)
}
}(file)
_, err = file.WriteString(fmt.Sprintf("export %v='%s'\n", envJavaToolOptions, javaToolOptions))
Expand Down

0 comments on commit eb598b0

Please sign in to comment.