1. Computing

How Can I Benchmark Code in Ruby?

By

Question: How Can I Benchmark Code in Ruby?

Using faster code in your Ruby application is always ideal. To discover which is faster, benchmarking is necessary step. It measures the time it takes to execute code and compares it to other code that accomplishes the same task.

Ruby provides a benchmarking library to make your job easy. The 'benchmark' library provides a few classes that will automatically benchmark your code and allow you to inspect the results.

Answer:

It's known that the += operator for strings is slow. The += operator must make a copy of the string, append to the copy and assign that copy to your variable. For large strings, this is extremely slow and is really hard on the heap and garbage collector. The += operator should only be used if a copy must be made. For all other string append operations, the << operator should be used. To test this, we'll do a benchmark experiment.

This will be an extreme example. 1,000,000 append operations will be performed. Near the end of the run, the += operator will be making copies of a 1 megabyte string, appending to it and discarding the original copy. This will really thrash the heap and be extremely slow.

To use the benchmark library, require the 'benchmark' module. The Benchmark.bm method is the simplest interface to benchmarking your code.

#!/usr/bin/env ruby
require 'benchmark'

Benchmark.bm do|b|
  b.report("+= ") do
    a = ""
    1_000_000.times { a += "." }
  end

  b.report("<< ") do
    a = ""
    1_000_000.times { a << "." }
  end
end

Running this code will output a report of the times it took to execute each report.

$ ruby 1.rb
      user system total real
+= 462.680000 298.490000 761.170000 (814.969652)
<< 0.980000 0.090000 1.070000 ( 1.544154)

As you can see, the += operator is extremely slow, almost 550 times slower than the << operator! I know which one I'll be using in my code now.

Environmental Effects

Benchmarks can behave differently depending on the order they're run. In the previous example, the += operator was really hard on the heap. The heap will most likely be left in an undesirable state when the time for the << operator comes around. This can affect your test results and make faster solutions look slower than they are.

The Benchmark.bmbm method attempts to solve this. The bmbm method will take a "practice run" of all of the tests before measuring the actual results and calling GC.start to clean up between tests. This attempts to normalize the heap and give each of the tests and even playing field. This is not ideal, but it can help to make the results less skewed.

#!/usr/bin/env ruby
require 'benchmark'

puts "Testing without cleaning up"
Benchmark.bm do|b|
  b.report("+=") do
    a = ""
    100_000.times { a += "." }
  end

  b.report("<<") do
    a = ""
    1_000_000.times { a << "." }
  end
end

GC.start
puts

puts "Testing with cleaning up"
Benchmark.bmbm do|b|
  b.report("+=") do
    a = ""
    100_000.times { a += "." }
  end

  b.report("<<") do
    a = ""
    100_000.times { a << "." }
  end
end
$ ruby 2.rb
Testing without cleaning up
      user system total real
+= 4.000000 0.040000 4.040000 ( 4.108434)
<< 0.940000 0.100000 1.040000 ( 1.053191)

Testing with cleaning up
Rehearsal --------------------------------------
+= 3.890000 0.080000 3.970000 ( 4.052096)
<< 0.100000 0.000000 0.100000 ( 0.106958)
----------------------------- total: 4.070000sec

         user system total real
+= 3.870000 0.040000 3.910000 ( 3.977660)
<< 0.100000 0.000000 0.100000 ( 0.107489)

As you can see from the results, if the heap is cleaned up after the += made a mess of it, the << method was 10 times faster.

  1. About.com
  2. Computing
  3. Ruby
  4. Tutorials
  5. FAQs
  6. How Fast Does Your Ruby Code Run?

©2014 About.com. All rights reserved.