Add a new `StressChaos` to generate plenty of stresses #332

AngleNet · 2020-03-11T14:19:32Z

What problem does this PR solve?

Add a new StressChaos to generate plenty of stresses over a collection of pods. We can burn cpus #144 , eat memory #145 and so on.

What is changed and how does it work?

Inject stresses into container's cgroup

Check List

Tests

Manul tests

Code changes

Has CI related scripts change

Side effects

NO

Related changes

Need to update the documentation

Does this PR introduce a user-facing change?:

Add a new `StressChaos` to generate plenty of stresses

…eature/stress

zhouqiang-cl · 2020-03-12T02:57:00Z

well done

codecov-io · 2020-03-14T06:05:16Z

Codecov Report

Merging #332 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #332   +/-   ##
=======================================
  Coverage   59.02%   59.02%           
=======================================
  Files          68       68           
  Lines        4369     4369           
=======================================
  Hits         2579     2579           
  Misses       1598     1598           
  Partials      192      192

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2d82eb6...2d82eb6. Read the comment docs.

…eature/stress

AngleNet · 2020-03-14T08:49:57Z

I'm very pleased to say that it's ready to be reviewed now ! Please help me to take a look. @zhouqiang-cl @ethercflow ❤️

AngleNet · 2020-03-14T08:52:45Z

Still missing unit tests for it since I'm not familiar with the current testing framework. Maybe I could add them later in another PR ?

cwen0 · 2020-03-14T08:53:32Z

@AngleNet Good job!

api/v1alpha1/stresschaos_webhook.go

yeya24 · 2020-04-25T14:36:48Z

Have you ever tested this feature in contained environment, such as Kind? Seems it doesn't work, both CPU stress and Memory stress.

➜  examples git:(feature/stress) ✗ kubectl top pods -l app=etcd
NAME                    CPU(cores)   MEMORY(bytes)
etcd-7db9d6f6f9-2tt6m   6m           4Mi

➜  examples git:(feature/stress) ✗ cat burn-memory.yaml
apiVersion: pingcap.com/v1alpha1
kind: StressChaos
metadata:
  name: burn-memory
  namespace: chaos-testing
spec:
  mode: one
  selector:
    labelSelectors:
      app: etcd
  stressors:
    memory:
      workers: 1
      size: 100MB
  duration: "20m"
  scheduler:
    cron: "@every 30m"

➜  examples git:(feature/stress) ✗ klo chaos-daemon-ht9st
Chaos-daemon Version: version.Info{GitVersion:"v0.0.0-master+$Format:%h$", GitCommit:"d893703a7602ebbf8a5bc3dba0aab0de3b7e8cea", GitTreeState:"dirty", BuildDate:"2020-04-24T21:35:38Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
2020-04-25T14:24:04.132Z	INFO	chaos-daemon-server	Starting http endpoint	{"address": "0.0.0.0:31766"}
2020-04-25T14:24:04.134Z	INFO	chaos-daemon-server	Starting grpc endpoint	{"address": "0.0.0.0:31767", "runtime": "containerd"}
2020-04-25T14:31:13.881Z	INFO	chaos-daemon-server	Executing stressors	{"request": "scope:POD target:\"containerd://236bb108ef4070579b843562b3999d4dea188cce86a17233b3fb7bf9d322a4aa\" stressors:\" --vm 1 --vm-keep --vm-bytes 100000000\" "}

AngleNet · 2020-04-25T14:41:59Z

What did you mean it didn’t work? I see the stressors are injected

yeya24 · 2020-04-25T14:46:21Z

Yes, it is injected. But the memory/CPU is not increasing at all. They remain the same. Is it the expected behavior? I guess no?

Or maybe the measuring method is not correct. kubectl top pods cannot get the correct result. So is there any other ways to validate there is a CPU/memory load. How did you do that? It would be helpful if you could provide some guides in the document.

cwen0 · 2020-04-26T02:39:35Z

Yes, it is injected. But the memory/CPU is not increasing at all. They remain the same. Is it the expected behavior? I guess no?

Or maybe the measuring method is not correct. kubectl top pods cannot get the correct result. So is there any other ways to validate there is a CPU/memory load. How did you do that? It would be helpful if you could provide some guides in the document.

I also run stressChaos in my environment, this is work for me, but we can't find this stress process in containers, I guess that we should run this stress-ng in the target namespace.

Gallardot · 2020-04-26T02:58:33Z

api/v1alpha1/stresschaos_webhook.go

+}
+
+// Validate validates whether the Stressors are all well defined
+func (in *Stressors) Validate(parent *field.Path, errs field.ErrorList) field.ErrorList {


Suggested change

func (in *Stressors) Validate(parent *field.Path, errs field.ErrorList) field.ErrorList {

func (in *Stressors) Validate(parent *field.Path) field.ErrorList {

keep the same style

Gallardot · 2020-04-26T02:58:57Z

api/v1alpha1/stresschaos_webhook.go

+}
+
+// Validate validates whether the Stressor is well defined
+func (in *Stressor) Validate(parent *field.Path, errs field.ErrorList) field.ErrorList {


Suggested change

func (in *Stressor) Validate(parent *field.Path, errs field.ErrorList) field.ErrorList {

func (in *Stressor) Validate(parent *field.Path) field.ErrorList {

Gallardot · 2020-04-26T02:59:15Z

api/v1alpha1/stresschaos_webhook.go

+}
+
+// Validate validates whether the MemoryStressor is well defined
+func (in *MemoryStressor) Validate(parent *field.Path, errs field.ErrorList) field.ErrorList {


Suggested change

func (in *MemoryStressor) Validate(parent *field.Path, errs field.ErrorList) field.ErrorList {

func (in *MemoryStressor) Validate(parent *field.Path) field.ErrorList {

api/v1alpha1/stresschaos_webhook.go

AngleNet · 2020-04-26T04:39:35Z

/run-e2e-test

cwen0 · 2020-04-26T05:43:04Z

api/v1alpha1/stresschaos_types.go

+		stressors += fmt.Sprintf(" --vm %d --vm-keep", in.MemoryStressor.Workers)
+		if len(in.MemoryStressor.Size) != 0 {
+			if in.MemoryStressor.Size[len(in.MemoryStressor.Size)-1] != '%' {
+				size, _ := units.FromHumanSize(in.MemoryStressor.Size)


Handle this error

pkg/chaosdaemon/stress_server.go

cwen0 · 2020-04-26T10:17:37Z

/merge

sre-bot · 2020-04-26T10:19:54Z

/run-all-tests

dismiss

sre-bot · 2020-04-26T10:54:27Z

cherry pick to release-0.8 failed

* Add stress chaos type

* Add stress chaos type Co-authored-by: AngleNet <[email protected]>

* Add stress chaos type

AngleNet added 2 commits March 10, 2020 20:33

Add stress chaos type

be15e8b

Update

43fed52

sre-bot added the contribution label Mar 11, 2020

AngleNet added 5 commits March 11, 2020 22:22

Merge branch 'master' of https://github.com/pingcap/chaos-mesh into f…

8a491a3

…eature/stress

Fix CI

f6c091b

Fix hound reviews

d829171

Fix CI

65c87f8

Fix CI

0c5d6a0

AngleNet changed the title ~~Add a new StressChaos to generate plenty of stresses~~ [WIP] Add a new StressChaos to generate plenty of stresses Mar 11, 2020

AngleNet added 6 commits March 13, 2020 15:55

Update

f4d5db2

Update

eb7f0aa

Update

8e62163

Update

e9051f9

Add doc

890fffe

Update

0ce66e7

AngleNet added 3 commits March 14, 2020 14:09

Merge branch 'master' of https://github.com/pingcap/chaos-mesh into f…

5a59d0a

…eature/stress

Update

9cd4bdb

Update

4fa8914

AngleNet changed the title ~~[WIP] Add a new StressChaos to generate plenty of stresses~~ [PTAL] Add a new StressChaos to generate plenty of stresses Mar 14, 2020

AngleNet added 2 commits March 14, 2020 15:26

Add webhook

da424ba

Update

74b6293

zhouqiang-cl added the status/PTAL label Mar 14, 2020

cwen0 added the component/operator label Mar 14, 2020

zhouqiang-cl changed the title ~~[PTAL] Add a new StressChaos to generate plenty of stresses~~ Add a new StressChaos to generate plenty of stresses Mar 14, 2020

zhouqiang-cl reviewed Mar 14, 2020

View reviewed changes

api/v1alpha1/stresschaos_webhook.go Outdated Show resolved Hide resolved

Gallardot previously requested changes Apr 26, 2020

View reviewed changes

cwen0 previously approved these changes Apr 26, 2020

View reviewed changes

Update

de24230

AngleNet dismissed cwen0’s stale review via de24230 April 26, 2020 04:23

cwen0 previously approved these changes Apr 26, 2020

View reviewed changes

cwen0 reviewed Apr 26, 2020

View reviewed changes

Yisaer requested changes Apr 26, 2020

View reviewed changes

pkg/chaosdaemon/stress_server.go Outdated Show resolved Hide resolved

Update

f82db5d

AngleNet dismissed cwen0’s stale review via f82db5d April 26, 2020 10:03

Yisaer approved these changes Apr 26, 2020

View reviewed changes

cwen0 approved these changes Apr 26, 2020

View reviewed changes

sre-bot added the status/can-merge label Apr 26, 2020

Merge branch 'master' into feature/stress

718e29c

zhouqiang-cl merged commit 9753087 into chaos-mesh:master Apr 26, 2020

cwen0 pushed a commit to cwen0/chaos-mesh that referenced this pull request Apr 26, 2020

Add a new StressChaos to generate plenty of stresses (chaos-mesh#332)

6124641

* Add stress chaos type

cwen0 added a commit that referenced this pull request Apr 26, 2020

Add a new StressChaos to generate plenty of stresses (#332) (#454)

0d10282

* Add stress chaos type Co-authored-by: AngleNet <[email protected]>

YangKeao mentioned this pull request Jul 30, 2020

The stress-burn-cpu seems not working #755

Closed

sjwsl pushed a commit to sjwsl/chaos-mesh that referenced this pull request May 6, 2021

Add a new StressChaos to generate plenty of stresses (chaos-mesh#332)

1753982

* Add stress chaos type

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a new `StressChaos` to generate plenty of stresses #332

Add a new `StressChaos` to generate plenty of stresses #332

AngleNet commented Mar 11, 2020 •

edited

Loading

zhouqiang-cl commented Mar 12, 2020

codecov-io commented Mar 14, 2020 •

edited

Loading

AngleNet commented Mar 14, 2020 •

edited

Loading

AngleNet commented Mar 14, 2020

cwen0 commented Mar 14, 2020

yeya24 commented Apr 25, 2020

AngleNet commented Apr 25, 2020

yeya24 commented Apr 25, 2020 •

edited

Loading

cwen0 commented Apr 26, 2020

Gallardot Apr 26, 2020

Gallardot Apr 26, 2020

Gallardot Apr 26, 2020

AngleNet commented Apr 26, 2020

cwen0 Apr 26, 2020

cwen0 commented Apr 26, 2020

sre-bot commented Apr 26, 2020

sre-bot commented Apr 26, 2020

	func (in Stressors) Validate(parent field.Path, errs field.ErrorList) field.ErrorList {
	func (in Stressors) Validate(parent field.Path) field.ErrorList {

	func (in Stressor) Validate(parent field.Path, errs field.ErrorList) field.ErrorList {
	func (in Stressor) Validate(parent field.Path) field.ErrorList {

	func (in MemoryStressor) Validate(parent field.Path, errs field.ErrorList) field.ErrorList {
	func (in MemoryStressor) Validate(parent field.Path) field.ErrorList {

Add a new StressChaos to generate plenty of stresses #332

Add a new StressChaos to generate plenty of stresses #332

Conversation

AngleNet commented Mar 11, 2020 • edited Loading

What problem does this PR solve?

What is changed and how does it work?

Check List

Does this PR introduce a user-facing change?:

zhouqiang-cl commented Mar 12, 2020

codecov-io commented Mar 14, 2020 • edited Loading

Codecov Report

AngleNet commented Mar 14, 2020 • edited Loading

AngleNet commented Mar 14, 2020

cwen0 commented Mar 14, 2020

yeya24 commented Apr 25, 2020

AngleNet commented Apr 25, 2020

yeya24 commented Apr 25, 2020 • edited Loading

cwen0 commented Apr 26, 2020

Gallardot Apr 26, 2020

Choose a reason for hiding this comment

Gallardot Apr 26, 2020

Choose a reason for hiding this comment

Gallardot Apr 26, 2020

Choose a reason for hiding this comment

AngleNet commented Apr 26, 2020

cwen0 Apr 26, 2020

Choose a reason for hiding this comment

cwen0 commented Apr 26, 2020

sre-bot commented Apr 26, 2020

sre-bot commented Apr 26, 2020

Add a new `StressChaos` to generate plenty of stresses #332

Add a new `StressChaos` to generate plenty of stresses #332

AngleNet commented Mar 11, 2020 •

edited

Loading

codecov-io commented Mar 14, 2020 •

edited

Loading

AngleNet commented Mar 14, 2020 •

edited

Loading

yeya24 commented Apr 25, 2020 •

edited

Loading