Translate: 
EnglishFrenchGermanItalianPolishPortugueseRussianSpanish

HipHop for PHP: Benchmark

Benchmark icon

There is a well known benchmark on the shootout.alioth.debian.org website comparing the performance of many different programming languages. While I am far from discussing the superiority of one language over another, I realized that it would be interesting to perform a little speed test between a … PHP and PHP (compiled with the help of HipHop for PHP).

I hope that test results presented below will be helpful in answering, how much faster is PHP after compilation, and whether the HipHop for PHP is actually as efficient as claimed by Facebook developers.


See some shocking results in second part of the benchmark: HipHop for PHP: Benchmark – Revenge of PHP [english version]


Introductory remarks

As always, I do not intend to describe the exact configuration of the system, collect detailed statistics and their distributions, or make one thousand repetitions for each test.

I do not paste entire output if these results take more than a few lines of text.
In that case, I send output to the out.txt file and present you only the execution times received by the tail out.txt command.

It’s up to the reader to draw the final conclusions or perform more accurate tests.

Testing platform

Processor: Intel(R) Core(TM)2 Duo CPU E7600 @ 3.06GHz
Memory: 1GB RAM (upgraded to 2.5 GB of RAM for test no. 6)
System: Fedora 12 (64bit)
Kernel: 2.6.32.26-175.fc12.x86_64 #1 SMP

The server was used exclusively for testing purposes – it was running only the services associated with the benchmark.

Test results

Here I present comparative results between the standard PHP and the scripts compiled with HipHop for PHP. All test results are shown in seconds (of course lower times are better)

Test #1: n-body (Perform an N-body simulation of the Jovian planets)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=nbody&lang=php&id=3

PHP result:

[root@localhost src]# php -n bench.php 50000000
-0.169075164
-0.169059907
DONE IN 754.474136

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_z8ZAiC/program 50000000
-0.169075164
-0.169059907
DONE IN 360.381707

In this test HipHop for PHP is more than two times faster than standard PHP.

Test #2: fannkuch-redux (Repeatedly access a tiny integer-sequence)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=fannkuchredux&lang=php&id=1

PHP result:

[root@localhost src]# php -n bench2.php 12
3968050
Pfannkuchen(12) = 65
DONE IN 3545.183693

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_4gdbXK/program 12
3968050
Pfannkuchen(12) = 65
DONE IN 1360.819455

Still no surprises, compiled PHP script once again proved to be more than two times faster.

Test #3: fasta (Generate and write random DNA sequences)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=fasta&lang=php&id=3

PHP result:

[root@localhost src]# php -n bench3.php 25000000 > out.txt
DONE IN 138.252463

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_dKRE8V/program 25000000 > out2.txt
DONE IN 55.924453

HipHop for PHP is still maintaining a double advantage.

Test #4: spectral-norm (Calculate an eigenvalue using the power method)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=spectralnorm&lang=php&id=3

PHP result:

[root@localhost src]# php -n bench4.php 5500
1.274224153
DONE IN 238.810613

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_N7arkg/program 5500
1.274224153
DONE IN 93.717688

Test #5: mandelbrot (Generate a Mandelbrot set and write a portable bitmap)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=mandelbrot&lang=php&id=1

PHP result:

[root@localhost src]# php -n bench5.php 16000 > out.txt
DONE IN 1532.191153

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_cMY58k/program 16000 > out2.txt
DONE IN 569.224470

Here the difference between an ordinary PHP and compiled by HipHop gets even bigger, script execution time was reduced almost threefold.

Test #6: binary-trees (Allocate and deallocate many many binary trees)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&lang=php&id=3

PHP result:

[root@localhost src]# php -n -d memory_limit=2048M bench6.php 16
stretch tree of depth 17   check: -1
131072  trees of depth 4   check: -131072
32768  trees of depth 6  check: -32768
8192   trees of depth 8  check: -8192
2048   trees of depth 10   check: -2048
512  trees of depth 12  check: -512
128  trees of depth 14  check: -128
32   trees of depth 16  check: -32
long lived tree of depth 16    check: -1
DONE IN 37.286621

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_s93MOe/program 16
stretch tree of depth 17   check: -1
131072  trees of depth 4   check: -131072
32768  trees of depth 6  check: -32768
8192   trees of depth 8  check: -8192
2048   trees of depth 10   check: -2048
512  trees of depth 12  check: -512
128  trees of depth 14  check: -128
32   trees of depth 16  check: -32
long lived tree of depth 16    check: -1
DONE IN 11.450954

After performing many different types of tests we were finally able to cross the barrier of 300%. I suspect that in this case a better result corresponds to the faster implementation of functions in native C++, commonly used in the test loop.

Update report

Previous tests were performed on a smaller data sample (n = 16) than in the reference test, since the potential memory consumption exceeded the physical abilities of test server.

Intrigued by the above results and claims from Facebook developers concerning smaller memory usage of scripts compiled by HipHop for PHP, I decided to repeat the test with a reference sample of n = 20 (such as in test at shootout webpage). To make it possible, I upgraded a test machine with an additional 1.5 Gb of RAM. Here are the results.

PHP result:

[root@localhost src]# php -n -d memory_limit=2548M bench6.php 20
stretch tree of depth 21         check: -1
2097152  trees of depth 4        check: -2097152
524288   trees of depth 6        check: -524288
131072   trees of depth 8        check: -131072
32768    trees of depth 10       check: -32768
8192     trees of depth 12       check: -8192
2048     trees of depth 14       check: -2048
512      trees of depth 16       check: -512
128      trees of depth 18       check: -128
32       trees of depth 20       check: -32
long lived tree of depth 20      check: -1
DONE IN 1020.273547

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_s93MOe/program 20
stretch tree of depth 21         check: -1
2097152  trees of depth 4        check: -2097152
524288   trees of depth 6        check: -524288
131072   trees of depth 8        check: -131072
32768    trees of depth 10       check: -32768
8192     trees of depth 12       check: -8192
2048     trees of depth 14       check: -2048
512      trees of depth 16       check: -512
128      trees of depth 18       check: -128
32       trees of depth 20       check: -32
long lived tree of depth 20      check: -1
DONE IN 249.907754

As you can see, this time HipHop for PHP was faster by over 400%. Also the memory usage was much lower: 2465MB used by standard PHP vs 1802MB used by HipHop for PHP (more than 600MB of difference!).

Summary

Here is the summary of test results:

Test name PHP HipHop Ratio
n-body 754.474136 360.381707 209,35%
fannkuch-redux 3545.183693 1360.819455 260,52%
fasta 138.252463 55.924453 247,21%
spectral-norm 238.810613 93.717688 254,81%
mandelbrot 1532.191153 569.224470 269,17%
binary-trees (16) 37.286621 11.450954 325,62%
binary-trees (20) 1020.273547 249.907754 408,26%

HipHop for PHP benchmark

As you can see HipHop for PHP dramatically accelerates the PHP scripts. Differences in the time range from 200% to over 400% and have been achieved without any changes applied to the PHP source code files.

Also, memory consumption is much lower in the case of PHP compiled by HipHop for PHP (up to 25%).

Unfortunately, there were some incompatibilities with standard PHP language: for example, lack of support for gmp library (failure to perform the pidigits test).

I describe HipHop for PHP incompatibility problems in a separate article.


Other articles about breaking the limits of PHP


Tags: , , , ,

6 Responses to “HipHop for PHP: Benchmark”

  1. Isaac Gouy says:

    reverse-complement?

    k-nucleotide?

    regex-dna?

  2. Artur Graniszewski says:

    Ok, I’ll try to add those measurements today.

  3. Artur Graniszewski says:

    @Isaac Gouy, I’ve added two more tests for you: http://php.webtutor.pl/index.php/2011/04/04/hiphop-for-php-benchmark-revenge-of-php

    b.t.w. Results are shocking!

  4. Ian says:

    It’s worth considering also running these same tests with APC enabled since a lot of the people who may be candidates for HipHop are already running opcode-cached software.

  5. Artur Graniszewski says:

    That’s a good idea. I’ll repeat those test with APC and try to compare the results.

    Don you think this should be done in another article rather than this one?

  6. jacky says:

    I’ve tried n-body workload, but the result is really bad on HHVM:
    do you have experience on HHVM? Is there anything I should pay more attention? like system configuration….etc.

    root@ubuntu:/home/hhvmtest/n-body# time hhvm nbody.php 50000
    -0.169075164
    -0.169078071

    real 0m1.130s
    user 0m1.100s
    sys 0m0.024s
    root@ubuntu:/home/hhvmtest/n-body# time php nbody.php 50000
    -0.169075164
    -0.169078071

    real 0m0.667s
    user 0m0.656s
    sys 0m0.004s
    ==========================================
    do you have any data on hhvm vs. PHP?

Leave a Reply