Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance to call Mysql2::Result#each and Mysql2::Result#fields #1046

Merged
merged 2 commits into from
May 19, 2019

Conversation

Watson1978
Copy link
Contributor

@Watson1978 Watson1978 commented May 17, 2019

This PR will reduce the time to call Mysql2::Result#each and Mysql2::Result#fields.

If table records are small enough, the performance will improve as following.

Mysql2::Result#each : 901.633k -> 1.550M (1.71x faster)
Mysql2::Result#fields : 3.779M -> 6.233M (1.65x faster)
  1. Use rb_ivar_get/rb_ivar_set to improve performance
  • By convert C-string to Ruby string ID each time, it will take a slightly time to call method each time.
    This patch will catch the Ruby string ID to cut off the time.
  1. Check only whether block was given
  • rb_scan_args(argc, argv, "01&", ...) will generate Proc object from block. However, the object has used to only check whether block was given. To remove redundant object generating, this patch will use rb_block_given_p() to check whether block was given.

Environment

  • CPU : Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz
  • OS : Ubuntu 18.10
  • Compiler : gcc version 8.3.0

Before changing

Warming up --------------------------------------
               query   886.000  i/100ms
                each    67.339k i/100ms
              fields   195.612k i/100ms
Calculating -------------------------------------
               query      9.385k (± 4.5%) i/s -     46.958k in   5.016539s
                each    901.633k (± 0.2%) i/s -      4.512M in   5.003951s
              fields      3.779M (± 0.2%) i/s -     18.974M in   5.021533s

After changing

Warming up --------------------------------------
               query   864.000  i/100ms
                each   106.916k i/100ms
              fields   251.255k i/100ms
Calculating -------------------------------------
               query      9.457k (± 3.8%) i/s -     47.520k in   5.032949s
                each      1.550M (± 0.3%) i/s -      7.805M in   5.037029s
              fields      6.233M (± 0.1%) i/s -     31.407M in   5.039049s

Benchmarks

require 'benchmark/ips'
require 'mysql2'

def setup_database(client)
  # client.query('create database benchmark')
  sql = <<SQL
create table benchmark.users (
  id int NOT NULL AUTO_INCREMENT PRIMARY KEY,
  name text,
  email text,
  created_at datetime,
  updated_at datetime
)
SQL
  client.query(sql)

  5.times do |i|
    client.query("insert into benchmark.users (name, email, created_at, updated_at) values ('foo', '[email protected]', '#{Time.now.strftime("%Y-%m-%d %H:%M:%S")}', '#{Time.now.strftime("%Y-%m-%d %H:%M:%S")}')")
  end
end


client = Mysql2::Client.new(
  :host     => "127.0.0.1",
  :username => "root",
  :password => "",
  :database => "benchmark"
)

# setup_database(client)

Benchmark.ips do |x|
  query = 'select * from users'.freeze
  x.report "query" do
    client.query(query)
  end

  results = client.query(query)
  x.report "each" do
    results.each {}
  end

  x.report "fields" do
    results.fields
  end
end

By convert C-string to Ruby string ID each time, it will take a slightly time to call method each time.
This patch will catch the Ruby string ID to cut off the time.

* Before
```
Warming up --------------------------------------
               query   886.000  i/100ms
                each    67.339k i/100ms
              fields   195.612k i/100ms
Calculating -------------------------------------
               query      9.385k (± 4.5%) i/s -     46.958k in   5.016539s
                each    901.633k (± 0.2%) i/s -      4.512M in   5.003951s
              fields      3.779M (± 0.2%) i/s -     18.974M in   5.021533s
```

* After
```
Warming up --------------------------------------
               query   845.000  i/100ms
                each    86.916k i/100ms
              fields   231.527k i/100ms
Calculating -------------------------------------
               query      9.553k (± 2.0%) i/s -     48.320k in   5.059947s
                each      1.133M (± 0.3%) i/s -      5.736M in   5.062606s
              fields      6.319M (± 0.1%) i/s -     31.719M in   5.019960s
```
block = Qfalse;
if (rb_block_given_p()) {
block = Qtrue;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great explanation that actually capturing the block will generate a Proc object that is never called!

To really clarify this code, I suggest a deeper change:

  1. Add a comment above rb_scan_args that says something like "A block can be passed to this method, but since we don't call the block directly from C, we don't need to capture it into a variable here with the "&" scan arg"

  2. Remove the block variable entirely.

  3. Replace args.block_given = block; on line 890 with args.block_given = rb_block_given_p()

(Adjust the data type of args.block_given to match the return value of rb_block_given_p() if needed)

Copy link
Contributor Author

@Watson1978 Watson1978 May 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sodabrew Thank you for your review!! I fixed a patch with your idea at a734989

@sodabrew
Copy link
Collaborator

This is great work, thank you! One comment in the review.

@Watson1978 Watson1978 force-pushed the improve/performance branch from 2638717 to f7b0db4 Compare May 19, 2019 12:59
`rb_scan_args(argc, argv, "01&", ...)` will generate `Proc` object from
block.
However, the object has used to only check whether block was given.

To remove redundant object generating, this patch will use
`rb_block_given_p()` to check whether block was given.

* Before
```
Warming up --------------------------------------
               query   845.000  i/100ms
                each    86.916k i/100ms
              fields   231.527k i/100ms
Calculating -------------------------------------
               query      9.553k (± 2.0%) i/s -     48.320k in
5.059947s
                each      1.133M (± 0.3%) i/s -      5.736M in
5.062606s
              fields      6.319M (± 0.1%) i/s -     31.719M in
5.019960s
```

* After
```
Warming up --------------------------------------
               query   864.000  i/100ms
                each   106.916k i/100ms
              fields   251.255k i/100ms
Calculating -------------------------------------
               query      9.457k (± 3.8%) i/s -     47.520k in
5.032949s
                each      1.550M (± 0.3%) i/s -      7.805M in
5.037029s
              fields      6.233M (± 0.1%) i/s -     31.407M in
5.039049s
```
@Watson1978 Watson1978 force-pushed the improve/performance branch from f7b0db4 to a734989 Compare May 19, 2019 13:04
@sodabrew sodabrew merged commit 4e221cf into brianmario:master May 19, 2019
@sodabrew sodabrew added this to the 0.5.3 milestone May 19, 2019
@Watson1978 Watson1978 deleted the improve/performance branch May 19, 2019 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants