PHP Performance Testing
Website Performance: PHP and similar articles list many PHP performance tips, but lists of tips sometimes contain spurious advice, or at least advice of minimal value. We really must measure the effectiveness of a tip in our own production environment before judging its worth. Test Harness for PHP offers us a simple little PHP program that we can use to test code-related tips. Today we will use the test harness to evaluate three tips that are listed in the PHP performance article. However, please remember that my results will differ from yours because the two servers I am testing on are different from yours in many ways.
The Two Servers
The following tests are being run on two different servers from two different web hosting companies. I paid about the same for the two services. Server #1 uses PHP Version 5.2.17 with Zend Engine v2.2.0. Server #2 uses PHP Version 5.3.10 with Zend Engine v2.3.0. Both are Linux boxes.
Selection Structures
I chose selection structures for my first test because I suspect this is a case of much ado about nothing. I think the three options will give similar results.Option #1 is an if-elseif-else construct:
if ($a==1) {
}elseif ($a==2) {
}elseif ($a==3) {
}elseif ($a==4) {
...
}elseif ($a==23) {
}elseif ($a==24) {
}elseif ($a==25) {
}else{
}
Option #2 is the same, except the keyword elseif is replaced by the two keywords else if:
if ($a==1) {
}else if ($a==2) {
}else if ($a==3) {
}else if ($a==4) {
...
}else if ($a==23) {
}else if ($a==24) {
}else if ($a==25) {
}else{
}
Option #3 is a switch-case construct:
switch ($a) {
case 1: break;
case 2: break;
case 3: break;
case 4: break;
...
case 23: break;
case 24: break;
case 25: break;
default:
}
In all cases, $a is initialized to 42 and the number of repetitions is set to 1,000,000 (what I usually use). The test results (in milliseconds) are:
Server # 1
------------------------------
Option 1 Option 2 Option 3
-------- -------- --------
1 2 1
1 2 2
2 2 2
2 1 2
1 2 1
2 1 2
1 2 1
2 1 2
1 2 1
1 2 1
Server # 2
------------------------------
Option 1 Option 2 Option 3
-------- -------- --------
0 1 0
0 1 1
0 1 0
1 0 1
1 0 1
0 1 0
1 1 0
1 1 0
0 1 0
0 1 1
Option #2 (if-elseif-else) is the slowest more than half the time. The other two are about equal.
However, are these results statistically significant? I don’t have the time or the expertise to do the calculations, but I remember from my statistics classes that anything with less than 50 trials is suspect. Let’s do 10 more trials and see if we get the same results.
Server # 1
------------------------------
Option 1 Option 2 Option 3
-------- -------- --------
2 2 1
1 1 2
2 1 2
2 2 2
1 2 2
1 2 1
2 2 2
2 1 2
1 2 1
2 1 2
Server # 2
------------------------------
Option 1 Option 2 Option 3
-------- -------- --------
1 0 1
1 1 0
0 1 1
1 0 1
0 1 0
0 1 0
1 0 1
1 0 1
1 1 0
0 1 0
These data lead to a different conclusion. Here there is no significant difference between the three options.
I’m stopping here because the tests are showing a maximum of 1 millisecond difference between the options. Even if we were to conclude that one option is faster than the other, one millisecond for a million tests is just not enough of a difference to matter.
After completing this article, I felt guilty about using nulls within the then and else clauses, so I reran the tests with $b = rand() inside every then, else, and case clause. The results were almost identical, and the conclusion was the same.
I get a gold star for guessing correctly on this one.
Iteration Structures
Does the iteration construct really matter? Once again, my guess is that the three options will give similar results.Option #1 is the traditional forthat we all learned in the beginning:
for ($i = 0; $i < $numReps; $i++) {$x = rand();}
Option #2 is the same, except the postincrement of the loop control variable is changed to a preincrement:
for ($i = 0; $i < $numReps; ++$i) {$x = rand();}
Option #3 combines the increment and test into one, which eliminates the increment section of the for. Note that option #3 is not equivalent to the first two options because the loop control variable ($i) increments in the first two options, but in the third option it decrements.
for ($i = $numReps; $i--;) {$x = rand();}
As always, the number of repetitions is set to 1,000,000. The results (in milliseconds) are:
Server # 1
------------------------------
Option 1 Option 2 Option 3
-------- -------- --------
0 1 0
1 0 1
0 1 0
0 1 0
0 0 0
1 0 0
0 1 0
1 0 0
0 1 0
0 0 1
Server # 2
------------------------------
Option 1 Option 2 Option 3
-------- -------- --------
0 0 0
1 0 0
0 1 0
0 0 1
0 0 0
0 0 0
0 0 1
1 0 0
0 0 0
0 0 0
With all those zeroes, one might wonder why we’re spending any time at all on this. According to these results, the biggest difference we can hope for is one millisecond per million tests.
We must be careful with our performance tests. I didn’t use empty bodies inside the loops because an optimizer could easily throw away the iteration construct and replace it with a simple assignment to $i.
Another gold star to add to my collection.
For vs. While
Again with the iteration structures, let’s see if for, while, or do-whileperform better than the others. My guess is that they are about the same, which seems to be the theme of this article.Option #1 is a forloop:
for ($i = 0; $i < $numReps; ++$i) {
$x = rand();
}
Option #2 is a while loop:
$i = 0;
while ($i < $numReps) {
$x = rand();
++$i;
}
Option #3 is a do-while loop. Note that option #3 is not equivalent to the other two because it forces the first iteration even if the condition is false. This doesn’t matter in the test we are conducting today, but it may matter in real-world code.
$i = 0;
do {
$x = rand();
++$i;
} while ($i < $numReps);
As always, the number of repetitions is set to 1,000,000. The results (in milliseconds) are:
Server # 1
------------------------------
Option 1 Option 2 Option 3
-------- -------- --------
0 1 0
1 0 0
0 1 0
0 1 0
0 0 1
0 0 1
1 0 0
1 0 1
0 1 0
0 0 1
Server # 2
------------------------------
Option 1 Option 2 Option 3
-------- -------- --------
0 0 1
0 0 0
1 0 0
0 0 0
0 1 0
0 1 0
0 0 0
0 0 1
1 0 0
0 0 0
And once again, we’re seeing such small numbers that it hardly seems worth our while. Even if we did consider these numbers significant, the test results show no real difference between the three options.
So I go away with three gold stars today. But I can’t really take that much credit – it was pretty obvious.
Conclusion
Some PHP performance tips we find on the Internet just don’t seem true when we look at actual measurements. We should use the PHP test harness to measure a tip’s effectiveness before we commit any resources to implementing it.
