Translate: 
EnglishFrenchGermanItalianPolishPortugueseRussianSpanish

HipHop for PHP: Benchmark – Revenge of PHP

In previous article I measured the performance of HipHop for PHP by performing some tests downloaded from the shootout.alioth.debian.org website. Unfortunately the test list was incomplete.

Only six tests were selected to avoid potential incompatibility issues with the HipHop compiler (those incompatibilities can be found in the compilers’ documentation). Three other PHP scripts had problems reading from the STDIN (tests: the reverse-complement, regex-base, k-nucleotide), while the fourth one failed because of lack of support for the gmp library (pidigits test).

As requested by readers however, I present you the results of two consecutive tests: regex-dna and k-nucleotide.

Introductory remarks

Unfortunately I was unable to carry out a test of reverse-complement, because it failed to run causing a segmentation fault. I’ll try to investigate the cause of this behavior in my free time.

I did not present these results in my previous article “HipHop for PHP: Benchmark”, because functionalities used in these tests are marked as unsupported by the compilers’ documentation.

Testing platform

Processor: Intel(R) Core(TM)2 Duo CPU E7600 @ 3.06GHz
Memory: 2.5GB RAM
System: Fedora 12 (64bit)
Kernel: 2.6.32.26-175.fc12.x86_64 #1 SMP

The server was used exclusively for testing purposes – it was running only the services associated with the benchmark.

Test results

In contrast to the tests (usually mathematical) described in the previous article, the following scripts operate mainly on a large amounts of text data (> 250MB) loaded from the files. To avoid potential IO bottlenecks I decided that the these files will be stored in the ramdisk:

[root@localhost ~]# mount -t tmpfs none /var/ramdisk -o size=900m

Test 1: regex-dna (Match DNA 8-mers and substitute Nucleotides for IUB code)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=regexdna&lang=php&id=2

PHP results:

[root@localhost ~]# php -n -d memory_limit=2512M dna.php 0 < /var/ramdisk/out5.txt
agggtaaa|tttaccct 356
[cgt]gggtaaa|tttaccc[acg] 1250
a[act]ggtaaa|tttacc[agt]t 4252
ag[act]gtaaa|tttac[agt]ct 2894
agg[act]taaa|ttta[agt]cct 5435
aggg[acg]aaa|ttt[cgt]ccct 1537
agggt[cgt]aa|tt[acg]accct 1431
agggta[cgt]a|t[acg]taccct 1608
agggtaa[cgt]|[acg]ttaccct 2178
 
50833429
50000017
66800253
DONE IN 34.079181

HipHop for PHP results:

[root@localhost src]# /tmp/hphp_Cv1BnI/program 0 < /var/ramdisk/out5.txt
agggtaaa|tttaccct 356
[cgt]gggtaaa|tttaccc[acg] 1250
a[act]ggtaaa|tttacc[agt]t 4252
ag[act]gtaaa|tttac[agt]ct 2894
agg[act]taaa|ttta[agt]cct 5435
aggg[acg]aaa|ttt[cgt]ccct 1537
agggt[cgt]aa|tt[acg]accct 1431
agggta[cgt]a|t[acg]taccct 1608
agggtaa[cgt]|[acg]ttaccct 2178
 
50833429
50000017
66800253
DONE IN 33.682081

As you can see, the difference in these tests is very small and falls into a margin of error.

Why compiled PHP script is as fast as a traditional one?

The answer can be found in the PHP script itself: the author of the source code used a very simple trick to accelerate the program: string operations were performed by a preg_replace() function which is already implemented in pure C++.

In this case the lower memory usage is the only advantage of a script's compilation. While HipHop for PHP consumed more virtual memory thanks to shared libraries, it also took 40% less real memory than a regular PHP:

                  VIRT   RES   SHR
PHP:              306m  100m  3048
HipHop for PHP:   427m   62m  9.9m

Test 2: k-nucleotide (Repeatedly update hashtables and k-nucleotide strings)

PHP Sourcecode:
http://shootout.alioth.debian.org/u32/program.php?test=knucleotide&lang=php&id=3

PHP result:

[root@localhost src]# php -n -d memory_limit=2500M rev.php < /var/ramdisk/out.txt > /var/ramdisk/ret0.txt
 
DONE IN 11.481418

HipHop for PHP result:

[root@localhost src]# /tmp/hphp_VLLBjz/program < /var/ramdisk/out.txt > /var/ramdisk/ret0.txt
 
DONE IN 125.345383

In this test I've stumbled upon some compatibility problems with the fastest PHP script found on shootout website. It could not read from the STDIN stream so I had to use another, somehow slower implementation.

Most shocking, however, is something else: this is the first test in which the traditional PHP is faster than the HipHop for PHP. What's more - it's eleven times faster.

It's not a mistake, I repeated the test several times - the difference remained at the same level.

How is this possible? The time command gives us some hints what's going on inside the program. Here's the result:

real    2m5.345s
user    0m7.764s
sys     1m57.552s

We can notice a huge amount of system time used by the PHP program. Let's take a look at the strace output:

[root@localhost src]# strace -o debug.txt /tmp/hphp_VLLBjz/program < /var/ramdisk/out.txt  > /var/ramdisk/ret0.txt 2

And here is the most interesting part of the result I received:

[...]
read(0, "CATGGTGAAACCCCGTCTCTACTAAA\nAATAC"..., 8192) = 8192
read(0, "TAAAAATA\nCAAAAATTAGCCGGGCGTGGTGG"..., 8192) = 8192
read(0, "GGCGTGGTGGCGCGCGCCTGTAATCCCAGCTA"..., 8192) = 8192
read(0, "ATCCCAGCTACTCGGGAGGCTGAGGCAGGAGA"..., 8192) = 8192
read(0, "AGGCAGGAGAATCGC\nTTGAACCCGGGAGGCG"..., 8192) = 8192
read(0, "CCGGGAGGCGGAGGTTGCAGTGAGCCGAGATC"..., 8192) = 8192
read(0, "AGCCGAGATCGCGCCACTGCACTCCAGCCTGG"..., 8192) = 8192
read(0, "TCCAGCCTGGGCGACAGAGCGA\nGACTCCGTC"..., 8192) = 8192
mmap(NULL, 847872, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe849495000
mremap(0x7fe849495000, 847872, 851968, MREMAP_MAYMOVE) = 0x7fe8493c5000
mremap(0x7fe8493c5000, 851968, 856064, MREMAP_MAYMOVE) = 0x7fe8493c5000
read(0, "GACT\nCCGTCTCAAAAAGGCCGGGCGCGGTGG"..., 8192) = 8192
mremap(0x7fe8493c5000, 856064, 860160, MREMAP_MAYMOVE) = 0x7fe8493c5000
mremap(0x7fe8493c5000, 860160, 864256, MREMAP_MAYMOVE) = 0x7fe8493c5000
read(0, "GGCGCGGTGGCTCACGCCTGTAATCCCAGCAC"..., 8192) = 8192
mremap(0x7fe8493c5000, 864256, 868352, MREMAP_MAYMOVE) = 0x7fe8493c5000
mremap(0x7fe8493c5000, 868352, 872448, MREMAP_MAYMOVE) = 0x7fe8493c5000
read(0, "ATCCCAGCACTTTGGGAGGCCGAGGCGGG\nCG"..., 8192) = 8192
[...]

The top of the log is uninteresting. But take a look at the last few lines - they're repeating for the rest of the script. HipHop for PHP after reading few lines from the beginning of the STDIN stream starts to dynamically allocate memory for the new concatenated string.

In this case, the number of mremap operations is really huge (two 4kB remappings per one 8kB data chunk retrieved from the STDIN stream), and this is the main reason for a huge script slowdown.

For comparison, I've included the log from the regular PHP script:

[...]
mremap(0x7f5821b62000, 24121344, 24383488, MREMAP_MAYMOVE) = 0x7f5821b62000
read(0, "GTGAAACCCCGTCTCTACTAAAAATACAAAAA"..., 8192) = 8192
read(0, "AATACAAAAATTAGCCGGGCGTGGTGGCGCGC"..., 8192) = 8192
read(0, "GGTGGCGCGCGCCTGTA\nATCCCAGCTACTCG"..., 8192) = 8192
read(0, "CAGCTACTCGGGAGGCTGAGGCAGGAGAATCG"..., 8192) = 8192
read(0, "AGGAGAATCGCTTGAACCCGGGAGGCGGAGGT"..., 8192) = 8192
read(0, "AGGCGGAGGTTGCAGTGAGCCGAG\nATCGCGC"..., 8192) = 8192
read(0, "AGATCG\nCGCCACTGCACTCCAGCCTGGGCGA"..., 8192) = 8192
read(0, "GCCTGGGCGACAGAGCGAGACTCCGTCTCAAA"..., 8192) = 8192
read(0, "CCGTCTCAAAAAGGCCGGGCGCGGTGGCTCA\n"..., 8192) = 8192
read(0, "GGTGGCTCACGCC\nTGTAATCCCAGCACTTTG"..., 8192) = 8192
read(0, "CAGCACTTTGGGAGGCCGAGGCGGGCGGATCA"..., 8192) = 8192
[...]

In this case, the ratio of remap operations to the amount of STDIN reads is much lower (one 256kB remap operation per 32 reads of 8kB data chunks).

Does HipHop for PHP have problems with basic string operations?

I decided to look into the issue by writing two simple scripts that do not use STDIN stream, but use only the functionality supported by the HipHop compiler:

sb-concat.php file:

<?php
 
$max = 160000000;
$result = "";
$start = microtime(true);
 
for($i = 0; $i < $max; $i++) {
    $result.= "a";
}
 
printf("GENERATED %d BYTES OF DATA IN %f SEC\n", strlen($result), microtime(true) - $start);

sb-malloc.php:

<?php
 
$max = 160000000;
$result = str_pad("", $max, chr(0));
$start = microtime(true);
 
for($i = 0; $i < $max; $i++) {
    $result[$i] = "a";
}
 
printf("GENERATED %d BYTES OF DATA IN %f SEC\n", strlen($result), microtime(true) - $start);

These scripts have been taken straight from my other article concerning the possibility of implementing the StringBuilder class in PHP.

Here is the output from sb-concat.php:

[root@localhost src]# /tmp/hphp_s1nTpN/program
GENERATED 160000000 BYTES OF DATA IN 63.566147 SEC

And the result of sb-malloc.php simulating StringBuilder functionality:

[root@localhost src]# /tmp/hphp_smmJbC/program
GENERATED 160000000 BYTES OF DATA IN 7.079259 SEC

The difference between these two scripts compiled by HipHop for PHP is as huge as 900%.

Both tests are nearly identical, the only difference is that sb-malloc.php preallocates 160MB of memory at the start of the script, while the second script performs dynamic memory allocation during the string concatenation.

We check the execution times to be absolutely sure:

[root@localhost src]# time /tmp/hphp_s1nTpN/program
GENERATED 160000000 BYTES OF DATA IN 61.368424 SEC
real    1m1.520s
user    0m7.492s
sys     0m54.005s

They prove our suspicions. Source code compilation brings major side effect known from other compiled languages ​​such as C, C++ or C#: slow string operations, and the need for StringBuilder functionality!.

It is worth mentioning that the execution times obtained by the compiled sb-concat.php script were highly unstable and fluctuated in a range of 33 seconds to over 120 seconds (60 seconds on average). A possible cause of this situation was... a memory fragmentation created by a huge amount of memory remap operations (this is a very unusual behaviour for Linux 2.6+ kernels ).

Inefficient memory allocation can be a big problem for the HipHop compiler: PHP is known for its high-speed string operations:

[root@localhost src]# php -d memory_limit=1024M sb-concat.php
GENERATED 160000000 BYTES OF DATA IN 11.122325 SEC
[root@localhost src]# php -d memory_limit=1024M sb-malloc.php
GENERATED 160000000 BYTES OF DATA IN 10.678082 SEC

In this case regular PHP is again up to 10 times faster than its HipHop counterpart. This example shows the power of the PHP language: its excellent performance as a WWW presentation layer (eg, HTML code generation).

Summary

Here is the summary of test results:

Test name PHP HipHop Ratio
regex-dna 34.079181 33.682081 101.17%
k-nucleotide 11.481418 125.345383 9.15%
concat 11.122325 63.566147 18.12%
stringbuilder 10.678082 7.079259 150.84%

HipHop for PHP benchmark #2

Is HipHop for PHP not suitable for the Web pages development?

The first article presented the strengths of the HipHop for PHP compiler: its positive impact on the speed of the PHP functions, loops, mathematical operations, and the language as a whole. This boost makes it possible to use PHP in other areas of programming, not necessarily connected with the world of web.

PHP was designed for web pages and with this in mind it was enhanced by developers of Facebook.

Did the HipHop make PHP better than before? It turns out that it didn't. PHP was designed for web development to produce dynamic web pages. In HipHop for PHP basic tasks like string operations (which are heavily used during web page generation) are up to 10 times slower than in the regular PHP script.


Other articles about breaking the limits of PHP


Tags: , , , ,

4 Responses to “HipHop for PHP: Benchmark – Revenge of PHP”

  1. Clement Mucosia says:

    You should pre-allocate the memory before you start the PHP application and be careful to not click in the close box until the window stops shaking. This way it will work properly without the loss of modesty.

  2. Inceptor says:

    Is compiled PHP faster than other interpreted languages such as Ruby or Python?

  3. Shoes Buttback says:

    This is interesting, but somewhat misses the point, because HipHop is *not* specifically intended to make PHP execution faster.

    The point of HipHop is to reduce memory and CPU usage, thus allowing Facebook to scale better and faster. By all accounts it has been very successful for them.

    If it makes the code run quicker too, then great.

  4. Artur Graniszewski says:

    Well, I disagree with you:

    slower PHP scripts = more running processes = higher memory usage = higher CPU usage by the system (overhead caused by context switching and various system locks) – this can create domino effect on very busy sites (exhaustion of system resources, like for example number of available HTTP threads, or MySQL connections)

    faster PHP scripts = less running processes (they finish faster) = lower memory usage = higher CPU utilization (CPU is used more efficiently)

    Of course, you have to choose in both scenarios: add more RAM and use slower CPU’s, or add faster CPU’s and use less RAM

Leave a Reply