mindexception: out of context

There is a well known benchmark on the shootout.alioth.debian.org website comparing the performance of many different programming languages. While I am far from discussing the superiority of one language over another, I realized that it would be interesting to perform a little speed test between a … PHP and PHP (compiled with the help of HipHop for PHP).

I hope that test results presented below will be helpful in answering, how much faster is PHP after compilation, and whether the HipHop for PHP is actually as efficient as claimed by Facebook developers.

Introductory remarks

As always, I do not intend to describe the exact configuration of the system, collect detailed statistics and their distributions, or make one thousand repetitions for each test.

I do not paste entire output if these results take more than a few lines of text.
In that case, I send output to the out.txt file and present you only the execution times received by the tail out.txt command.

It’s up to the reader to draw the final conclusions or perform more accurate tests.

Testing platform

Processor: Intel(R) Core(TM)2 Duo CPU E7600 @ 3.06GHz
Memory: 1GB RAM (upgraded to 2.5 GB of RAM for test no. 6)
System: Fedora 12 (64bit)
Kernel: 2.6.32.26-175.fc12.x86_64 #1 SMP

The server was used exclusively for testing purposes – it was running only the services associated with the benchmark.

Test results

Here I present comparative results between the standard PHP and the scripts compiled with HipHop for PHP. All test results are shown in seconds (of course lower times are better)

Test #1: n-body (Perform an N-body simulation of the Jovian planets)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=nbody&lang=php&id=3

PHP result:

[root@localhost src]# php -n bench.php 50000000
-0.169075164
-0.169059907
DONE IN 754.474136

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_z8ZAiC/program 50000000
-0.169075164
-0.169059907
DONE IN 360.381707

In this test HipHop for PHP is more than two times faster than standard PHP.

Test #2: fannkuch-redux (Repeatedly access a tiny integer-sequence)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=fannkuchredux&lang=php&id=1

PHP result:

[root@localhost src]# php -n bench2.php 12
3968050
Pfannkuchen(12) = 65
DONE IN 3545.183693

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_4gdbXK/program 12
3968050
Pfannkuchen(12) = 65
DONE IN 1360.819455

Still no surprises, compiled PHP script once again proved to be more than two times faster.

Test #3: fasta (Generate and write random DNA sequences)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=fasta&lang=php&id=3

PHP result:

[root@localhost src]# php -n bench3.php 25000000 > out.txt
DONE IN 138.252463

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_dKRE8V/program 25000000 > out2.txt
DONE IN 55.924453

HipHop for PHP is still maintaining a double advantage.

Test #4: spectral-norm (Calculate an eigenvalue using the power method)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=spectralnorm&lang=php&id=3

PHP result:

[root@localhost src]# php -n bench4.php 5500
1.274224153
DONE IN 238.810613

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_N7arkg/program 5500
1.274224153
DONE IN 93.717688

Test #5: mandelbrot (Generate a Mandelbrot set and write a portable bitmap)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=mandelbrot&lang=php&id=1

PHP result:

[root@localhost src]# php -n bench5.php 16000 > out.txt
DONE IN 1532.191153

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_cMY58k/program 16000 > out2.txt
DONE IN 569.224470

Here the difference between an ordinary PHP and compiled by HipHop gets even bigger, script execution time was reduced almost threefold.

Test #6: binary-trees (Allocate and deallocate many many binary trees)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&lang=php&id=3

PHP result:

[root@localhost src]# php -n -d memory_limit=2048M bench6.php 16
stretch tree of depth 17   check: -1
131072  trees of depth 4   check: -131072
32768  trees of depth 6  check: -32768
8192   trees of depth 8  check: -8192
2048   trees of depth 10   check: -2048
512  trees of depth 12  check: -512
128  trees of depth 14  check: -128
32   trees of depth 16  check: -32
long lived tree of depth 16    check: -1
DONE IN 37.286621

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_s93MOe/program 16
stretch tree of depth 17   check: -1
131072  trees of depth 4   check: -131072
32768  trees of depth 6  check: -32768
8192   trees of depth 8  check: -8192
2048   trees of depth 10   check: -2048
512  trees of depth 12  check: -512
128  trees of depth 14  check: -128
32   trees of depth 16  check: -32
long lived tree of depth 16    check: -1
DONE IN 11.450954

After performing many different types of tests we were finally able to cross the barrier of 300%. I suspect that in this case a better result corresponds to the faster implementation of functions in native C++, commonly used in the test loop.

Update report

Previous tests were performed on a smaller data sample (n = 16) than in the reference test, since the potential memory consumption exceeded the physical abilities of test server.

Intrigued by the above results and claims from Facebook developers concerning smaller memory usage of scripts compiled by HipHop for PHP, I decided to repeat the test with a reference sample of n = 20 (such as in test at shootout webpage). To make it possible, I upgraded a test machine with an additional 1.5 Gb of RAM. Here are the results.

PHP result:

[root@localhost src]# php -n -d memory_limit=2548M bench6.php 20
stretch tree of depth 21         check: -1
2097152  trees of depth 4        check: -2097152
524288   trees of depth 6        check: -524288
131072   trees of depth 8        check: -131072
32768    trees of depth 10       check: -32768
8192     trees of depth 12       check: -8192
2048     trees of depth 14       check: -2048
512      trees of depth 16       check: -512
128      trees of depth 18       check: -128
32       trees of depth 20       check: -32
long lived tree of depth 20      check: -1
DONE IN 1020.273547

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_s93MOe/program 20
stretch tree of depth 21         check: -1
2097152  trees of depth 4        check: -2097152
524288   trees of depth 6        check: -524288
131072   trees of depth 8        check: -131072
32768    trees of depth 10       check: -32768
8192     trees of depth 12       check: -8192
2048     trees of depth 14       check: -2048
512      trees of depth 16       check: -512
128      trees of depth 18       check: -128
32       trees of depth 20       check: -32
long lived tree of depth 20      check: -1
DONE IN 249.907754

As you can see, this time HipHop for PHP was faster by over 400%. Also the memory usage was much lower: 2465MB used by standard PHP vs 1802MB used by HipHop for PHP (more than 600MB of difference!).

Summary

Here is the summary of test results:

As you can see HipHop for PHP dramatically accelerates the PHP scripts. Differences in the time range from 200% to over 400% and have been achieved without any changes applied to the PHP source code files.

Also, memory consumption is much lower in the case of PHP compiled by HipHop for PHP (up to 25%).

Unfortunately, there were some incompatibilities with standard PHP language: for example, lack of support for gmp library (failure to perform the pidigits test).

I describe HipHop for PHP incompatibility problems in a separate article.

Other articles about breaking the limits of PHP

• HipHop for PHP: Benchmark – Revenge of PHP
• Strong data typing in PHP, part II: autoboxing and indestructable objects
• Mod_rewrite and HipHop for PHP on Apache Web Server
• How to install HipHop for PHP on Fedora 12 Linux (google translation)
• Beyond limits: hacking SAFE_MODE and max_execution_time (google translation)
• What really slows PHP down? (google translation)
• Howto: Strong data types in PHP (google translation)

Tags: alioth shootout, benchmark, Facebook, Fedora, Linux

This entry was posted on Saturday, April 2nd, 2011 at 23:24 and is filed under C++, Curiosities, HipHop for PHP, Optimizations, PHP. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.