Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v3 decoding can allocate 3-6x memory and take 2-3x long as v2 #520

Closed
liggitt opened this issue Oct 10, 2019 · 3 comments
Closed

v3 decoding can allocate 3-6x memory and take 2-3x long as v2 #520

liggitt opened this issue Oct 10, 2019 · 3 comments

Comments

@liggitt
Copy link
Contributor

liggitt commented Oct 10, 2019 •
edited
Loading

Using benchcmp and the dense/deep object benchmarks added recently:

git checkout v2
go test . -run None -bench . -benchmem > v2.txt

Benchmark1000KB100Aliases-8                    1        1118968506 ns/op        393118512 B/op   5832427 allocs/op
Benchmark1000KBDeeplyNestedSlices-8            5         228028088 ns/op         4690166 B/op       9069 allocs/op
Benchmark1000KBDeeplyNestedMaps-8              5         229030847 ns/op         4690419 B/op       9073 allocs/op
Benchmark1000KBDeeplyNestedIndents-8         500           2855677 ns/op         2970180 B/op      10083 allocs/op
Benchmark1000KB1000IndentLines-8               3         415356292 ns/op        143718512 B/op   4093516 allocs/op
Benchmark1KBMaps-8                          3000            476491 ns/op          218985 B/op       3126 allocs/op
Benchmark10KBMaps-8                          300           4799206 ns/op         2178156 B/op      30780 allocs/op
Benchmark100KBMaps-8                          30          48915174 ns/op        22002808 B/op     307269 allocs/op
Benchmark1000KBMaps-8                          2         525698256 ns/op        220560592 B/op   3072080 allocs/op

git checkout v3
go test . -run None -bench . -benchmem > v3.txt

Benchmark1000KB100Aliases-8                    1        2090412170 ns/op        1470212864 B/op  6344284 allocs/op
Benchmark1000KBDeeplyNestedSlices-8            5         222505744 ns/op        13930344 B/op       9091 allocs/op
Benchmark1000KBDeeplyNestedMaps-8              5         220789034 ns/op        13930736 B/op       9097 allocs/op
Benchmark1000KBDeeplyNestedIndents-8         200           9294121 ns/op        12260665 B/op      10109 allocs/op
Benchmark1000KB1000IndentLines-8               2         714826480 ns/op        697303904 B/op   4093555 allocs/op
Benchmark1KBMaps-8                          2000            746970 ns/op          649890 B/op       3651 allocs/op
Benchmark10KBMaps-8                          200           9184349 ns/op        11417516 B/op      35923 allocs/op
Benchmark100KBMaps-8                          20         101826019 ns/op        114370504 B/op    358502 allocs/op
Benchmark1000KBMaps-8                          1        1075424573 ns/op        1297663912 B/op  3584123 allocs/op

go get golang.org/x/tools/cmd/benchcmp
benchcmp v2.txt v3.txt

benchmark                                old ns/op      new ns/op      delta
Benchmark1000KB100Aliases-8              1118968506     2090412170     +86.82%
Benchmark1000KBDeeplyNestedSlices-8      228028088      222505744      -2.42%
Benchmark1000KBDeeplyNestedMaps-8        229030847      220789034      -3.60%
Benchmark1000KBDeeplyNestedIndents-8     2855677        9294121        +225.46%
Benchmark1000KB1000IndentLines-8         415356292      714826480      +72.10%
Benchmark1KBMaps-8                       476491         746970         +56.76%
Benchmark10KBMaps-8                      4799206        9184349        +91.37%
Benchmark100KBMaps-8                     48915174       101826019      +108.17%
Benchmark1000KBMaps-8                    525698256      1075424573     +104.57%

benchmark                                old allocs     new allocs     delta
Benchmark1000KB100Aliases-8              5832427        6344284        +8.78%
Benchmark1000KBDeeplyNestedSlices-8      9069           9091           +0.24%
Benchmark1000KBDeeplyNestedMaps-8        9073           9097           +0.26%
Benchmark1000KBDeeplyNestedIndents-8     10083          10109          +0.26%
Benchmark1000KB1000IndentLines-8         4093516        4093555        +0.00%
Benchmark1KBMaps-8                       3126           3651           +16.79%
Benchmark10KBMaps-8                      30780          35923          +16.71%
Benchmark100KBMaps-8                     307269         358502         +16.67%
Benchmark1000KBMaps-8                    3072080        3584123        +16.67%

benchmark                                old bytes     new bytes      delta
Benchmark1000KB100Aliases-8              393118512     1470212864     +273.99%
Benchmark1000KBDeeplyNestedSlices-8      4690166       13930344       +197.01%
Benchmark1000KBDeeplyNestedMaps-8        4690419       13930736       +197.00%
Benchmark1000KBDeeplyNestedIndents-8     2970180       12260665       +312.79%
Benchmark1000KB1000IndentLines-8         143718512     697303904      +385.19%
Benchmark1KBMaps-8                       218985        649890         +196.77%
Benchmark10KBMaps-8                      2178156       11417516       +424.18%
Benchmark100KBMaps-8                     22002808      114370504      +419.80%
Benchmark1000KBMaps-8                    220560592     1297663912     +488.35%
@liggitt liggitt changed the title v3 branch decoding allocates up to 5x as much memory as v2 branch v3 decoding can allocate 3-6x memory and take 2-3x long as v2 Oct 10, 2019
@niemeyer
Copy link
Contributor

I've just run these same commands in my local tree, and this is the outcome:

% benchcmp v2.txt v3.txt                                                                                                               
benchmark                                old ns/op      new ns/op      delta
Benchmark1000KB100Aliases-4              1178494993     1237510843     +5.01%
Benchmark1000KBDeeplyNestedSlices-4      248467998      244807407      -1.47%
Benchmark1000KBDeeplyNestedMaps-4        246630219      244783401      -0.75%
Benchmark1000KBDeeplyNestedIndents-4     3255168        3318219        +1.94%
Benchmark1000KB1000IndentLines-4         430011188      438619991      +2.00%
Benchmark1KBMaps-4                       509103         508197         -0.18%
Benchmark10KBMaps-4                      5062608        5072763        +0.20%
Benchmark100KBMaps-4                     53291550       52497158       -1.49%
Benchmark1000KBMaps-4                    562670959      542666084      -3.56%

benchmark                                old allocs     new allocs     delta                                                                                             
Benchmark1000KB100Aliases-4              5832428        5832425        -0.00%                                                                                            
Benchmark1000KBDeeplyNestedSlices-4      9069           9069           +0.00%                                                                                            
Benchmark1000KBDeeplyNestedMaps-4        9073           9073           +0.00%                                                                                            
Benchmark1000KBDeeplyNestedIndents-4     10083          10083          +0.00%                                                                                            
Benchmark1000KB1000IndentLines-4         4093516        4093516        +0.00%                                                                                            
Benchmark1KBMaps-4                       3126           3126           +0.00%                                                                                            
Benchmark10KBMaps-4                      30780          30780          +0.00%                                                                                            
Benchmark100KBMaps-4                     307269         307269         +0.00%                                                                                            
Benchmark1000KBMaps-4                    3072079        3072081        +0.00%                                                                                            

benchmark                                old bytes     new bytes     delta                                                                                               
Benchmark1000KB100Aliases-4              393118096     393117840     -0.00%                                                                                              
Benchmark1000KBDeeplyNestedSlices-4      4689718       4689718       +0.00%                                                                                              
Benchmark1000KBDeeplyNestedMaps-4        4689984       4690022       +0.00%
Benchmark1000KBDeeplyNestedIndents-4     2969970       2969967       -0.00%                                                                                              
Benchmark1000KB1000IndentLines-4         143718544     143718512     -0.00%                                                                                              
Benchmark1KBMaps-4                       218984        218985        +0.00%                                                                                              
Benchmark10KBMaps-4                      2178154       2178154       +0.00%
Benchmark100KBMaps-4                     22002798      22002802      +0.00%                                                                                              
Benchmark1000KBMaps-4                    220560496     220560656     +0.00%

What could I be missing?

@liggitt
Copy link
Contributor Author

liggitt commented Oct 10, 2019

apologies, I didn't notice the test package was referencing the fully-qualified gopkg.in import path. I had a stale yaml.v3 sitting in my $GOPATH missing the safeguards.

With updated deps for yaml.v2 and yaml.v3 in my GOPATH, I get:

benchmark                                old ns/op      new ns/op      delta
Benchmark1000KB100Aliases-8              1107407250     1101121894     -0.57%
Benchmark1000KBDeeplyNestedSlices-8      232696702      208512287      -10.39%
Benchmark1000KBDeeplyNestedMaps-8        217480119      212591945      -2.25%
Benchmark1000KBDeeplyNestedIndents-8     2851039        2734317        -4.09%
Benchmark1000KB1000IndentLines-8         408085503      383460138      -6.03%
Benchmark1KBMaps-8                       441535         448312         +1.53%
Benchmark10KBMaps-8                      4755982        4424239        -6.98%
Benchmark100KBMaps-8                     48365821       45379562       -6.17%
Benchmark1000KBMaps-8                    514252520      472024177      -8.21%

benchmark                                old allocs     new allocs     delta
Benchmark1000KB100Aliases-8              5832428        5832429        +0.00%
Benchmark1000KBDeeplyNestedSlices-8      9069           9069           +0.00%
Benchmark1000KBDeeplyNestedMaps-8        9073           9073           +0.00%
Benchmark1000KBDeeplyNestedIndents-8     10083          10083          +0.00%
Benchmark1000KB1000IndentLines-8         4093516        4093516        +0.00%
Benchmark1KBMaps-8                       3126           3126           +0.00%
Benchmark10KBMaps-8                      30780          30780          +0.00%
Benchmark100KBMaps-8                     307269         307269         +0.00%
Benchmark1000KBMaps-8                    3072079        3072080        +0.00%

benchmark                                old bytes     new bytes     delta
Benchmark1000KB100Aliases-8              393118640     393118672     +0.00%
Benchmark1000KBDeeplyNestedSlices-8      4690128       4690128       +0.00%
Benchmark1000KBDeeplyNestedMaps-8        4690406       4690432       +0.00%
Benchmark1000KBDeeplyNestedIndents-8     2970173       2970174       +0.00%
Benchmark1000KB1000IndentLines-8         143718544     143718512     -0.00%
Benchmark1KBMaps-8                       218984        218984        +0.00%
Benchmark10KBMaps-8                      2178155       2178158       +0.00%
Benchmark100KBMaps-8                     22002814      22002824      +0.00%
Benchmark1000KBMaps-8                    220560496     220560570     +0.00%

@liggitt liggitt closed this as completed Oct 10, 2019
@niemeyer
Copy link
Contributor

Phew.. :-)

@cjcullen cjcullen mentioned this issue Nov 7, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@niemeyer @liggitt