The alternate title for this post is “never trust a benchmark you haven’t manipulated yourself”.
Software benchmarks, like statistics, are easy to manipulate to make them say whatever you want. I just stumbled on this blog post showing a simple benchmark where Ruby comes out considerably faster than PHP. Since Ruby is generally considered a “slow” language, this result is interesting. Curious, I copied the program sources and tried it myself. Indeed, the Ruby version was considerably faster than PHP. I really like Ruby, so to me this was an interesting result. But skeptical as I am, I took a closer look.
I perused the source code and noticed a subtle difference between the Ruby and PHP versions. The program is basically a very tight while loop. At the core of the loop the program checks if an array is empty and it does an array shift. In the PHP version, the program counts how many elements there are in the array, and tests if the result is bigger than 0. This is wasteful. The moment you find one element, you know that the array is not empty. There is no need to visit every other element in the array. In contrast, the Ruby version only checks if the array is empty, as it should.
I edited the PHP version to just check if the array is empty, just like the Ruby version. Like magic, the execution time of the PHP version dropped more than half. The PHP version went from being ~70% slower than the Ruby version to being ~17% faster.
