Yii vs. APF

The article Performance of Yii under http://www.yiiframework.com/performance describes a performance comparison of different PHP frameworks. Due to the fact, that the APF is not among the candidates, we decided to empathize the test with the APF. The representative candidates are CakePHP 1.2.1.8004,Yii 1.0.2.r614 and APF 1.8-2009-01-09-2214. The result is summarized within the following picture:

Yii vs. APF - requests per second performance benchmark-->

1. Introduction

The measurement of the requests per second particularly stresses the performance overhead of using a framework. Generally speaking, this is the performance handicap you get using a framework. For this reason, a hello world! application is used for the benchmark. Another nice side effect ist, that the analysis of the tests are easy and the code files prepared for the test are possibly lack potential design errors (especialle good for greenhorns). Hence, the maximum throughput using the framework can be gathered.

As the article How to implement view based caching describes, performance is an important part of web applications. It is even more important, because the number of mobile devices grows. The result overview on the Yii page shows a significant difference what inspired us to re-perform the test with a few candidates and compare it with the performance of the APF.

To be able to evaluate the results, the article with the original benchmark on the Yii page can be used, as well as the PHP-Frameworks im Test (German) article published on the APF page.

2. Test description

2.1. Environment

To ensure comparability with the Yii benchmark, the test installation is equal. The test machine is a LINUX server (virtual VMWare machine) features the following performance characteristics:
Shell
[root@centos53 ~]# cat /etc/redhat-release CentOS release 5.2 (Final) [root@centos53 ~]# cat /proc/meminfo (gekürzt!) MemTotal: 1035244 kB [root@centos53 ~]# cat /proc/cpuinfo (gekürzt!) processor : 0 vendor_id : GenuineIntel model name : Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz cpu MHz : 2592.732 cache size : 6144 KB processor : 1 vendor_id : GenuineIntel model name : Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz cpu MHz : 2592.732 cache size : 6144 KB [root@centos53 ~]# lsscsi [0:0:0:0] disk VMware, VMware Virtual S 1.0 /dev/sda [root@centos53 ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/VG_LOCAL-LV_ROOT 18G 2.4G 15G 15% / /dev/sda1 99M 18M 77M 19% /boot tmpfs 506M 0 506M 0% /dev/shm [root@centos53 ~]# rpm -qa | grep -E "httpd|php" (gekürzt!) php-cli-5.1.6-20.el5_2.1 php-gd-5.1.6-20.el5_2.1 php-pecl-memcache-2.2.3-1.el5_2 php-mysql-5.1.6-20.el5_2.1 php-common-5.1.6-20.el5_2.1 httpd-2.2.3-11.el5_2.centos.4 php-5.1.6-20.el5_2.1 php-devel-5.1.6-20.el5_2.1
The caching of the compiled PHP files (aka. bytecode cache) is done by the current APC version (3.0.18) that was compiled from source and is configured with the following parameters:
Shell
[root@centos53 ~]# cat /etc/php.d/apc.ini extension=apc.so apc.enabled=1 apc.shm_segments=1 apc.optimization=1 apc.shm_size=32 apc.ttl=7200 apc.user_ttl=7200 apc.num_files_hint=1024 apc.mmap_file_mask=/tmp/apc.XXXXXX apc.enable_cli=1 apc.cache_by_default=1 apc.stat=0
As you can see in the last code box, apc.stat is disabled to minimize file system access. The configuration directive was choosen like that in the original test, too.

2.2. Test mode

The benchmark indicator is - as in the original test - the maximum request rate. For measurement issues, the Apache HTTP server benchmarking tool is used. To ensure, that the test is a real world test, the tool simulates 10 parallel users, that initiate requests 30 seconds long. Each test candidate was created a seperate VHOST, that was called during the test.

The test series contained four steps. The first two rounds are intended to gather the maximum requests per second without byte code caching, the latter ones show the performance boost using APC. Between the tests, the apache webserver was restarted to guarantee identical start conditions. In order to be not too idealistic, "warmup" was not done, because applications in real life environments do not have the chance to warmup either. During the test execution, no additional programs or daemons were executed.

3. Result

3.1. Overview

The detailed result is depicted in the following image including the absolute values:
RPS-Benchmark
  Without APC [RPS] With APC [RPS]
  1st round 2nd round average 1st round 2nd round average
CakePHP 30.94 31.53 31.24 141.83 141.33 141.58
Yii 121.23 116.36 118.80 935.78 935.63 935.71
APF 254.58 223.94 239.26 1023.29 986.34 1004.82
Yii vs. APF - requests per second performance benchmark

3.2. Details

The following list contains an interpretation on the results:
  • The difference between CakePHP and Yii seems to be identical to the Yii performance benchmark.
  • The optimization factor of the APF using APC is not as big as with CakePHP and Yii (APF: 1005/239 (~420%) and Yii: 936/119 (~780%)). This effect has already been discussed in the Zend framework 1.7rc vs. APF 1.8-BETA article.
    Nevertheless, the absolut performance indicator is about 100% higher without APC and roughly 8% higher using APC.
  • Yii claims, that the huge performance difference results from the architecture. This is also true for the APF. Please note, that the results for Yii turn worse, if apc.stat is enabled. Thus, the performance difference add up to 13% in favor to the APF. This leads to the conclusion, thet the lazy class loading of Yii is quite good, but the segmentation of the class file is some kind of unfortunate.
  • Further, the APF always uses template files to create the internal DOM tree per design. Hence, there is no chance (except with a front controller action) to generate the content by a controller directly. This means more file IO, that has to be initiated by the APF components. If we now add a file_get_contents() to the Yii controller to read the content from a text file, the performance indicators turn worse and the APF appears to be about 13% to 17% faster.

4. Files

The list presented below includes all relevant files, that were created for the test. These can be used to repeat it on your machine: The first three files contain the code files of the test candidates, *configfiles.tar.gz includes the server configuration files and in *resultfiles.tar.gz you can find the result files of the benchmark tool.


Comments

Do you want to add a comment to the article above, or do you want to post additional hints? So please click here. Comments already posted can be found below.
«   1   »
Entries/Page: | 5 | 10 | 15 | 20 |

1
Christian 03.09.2009, 14:26:47
Under http://forum.adventure-php-framework.org/en/viewtopic.php?f=10&t=26 you can see another benchmark (dooPHP vs. APF) that confirm the good performance of the APF.