Daniel's Corner

This is part I of a two-part comparison of PHP frameworks: CakePHP, CodeIgniter and Yii:

• Part 0 explains my comparison criteria.
• Part I is this part. I evaluate each framework after reading nearly all their documentation and writing a Hello World application.
• Part II will be the result of my experience as I use each framework to write a prototype for my real-world application.

I have spent 4 weeks studying each framework carefully. This document is long. I tried very hard to make it as short as possible without compromising on fairness, clarity or completeness with regards to my initial criteria

Part II will also take a few weeks, as I write an application prototype with each framework.

Contents

• Simplicity
• Documentation
• Performance
• Flexibility
• Security
• Other Notes
• Summary So Far

Simplicity

^Back to contents

It is hard to judge framework simplicity before making an application, but I can make some initial comments based on the documentation:

CakePHP

Installation: I had a problem with Apache that was not documented. I had to edit three .htaccess files to add a “RewriteBase” command.

Usage: Helpers, Actions, Behaviours, Layouts, Components… Do I need to know all that to make my program? Cake seems to have a higher barrier to entry and at the moment I don’t see any special feature it provides that the others don’t.

All frameworks come with a default layout and CSS. For my benchmark I had to remove this so all frameworks are doing the same work. With CakePHP I had a very hard time finding the layout file Cake was using. It was not in a logical place (inside my app directory).

CodeIgniter

Installation: Trivial. Just unzip the file and you are good to go.

Usage: CodeIgniter looks quite simple. The documentation never makes it look complex. My hello-world program was done a lot faster with CI than with Cake. Over-all, CI seems to have a low barrier to entry.

All frameworks come with a default layout and CSS. For my benchmark I had to remove this so all frameworks are doing the same work. With CI this was automatic when I made a new view. CI doesn’t insert layout unless you ask it to.

Yii

Installation: Easy. You have to run yiic to create the web root directory. There is also a PHP version (yiic.php) which is good if you don’t have ssh access. Yiic is well documented.

Usage: Yii also looks simple and straight forward, though a bit less so than CodeIgniter. My hello-world program was done almost as quickly as with CI.

All frameworks come with a default layout and CSS. For my benchmark I had to remove this so all frameworks are doing the same work. With Yii this was very easy. The layout file is in a logical place and it is well documented.

I am pleased to report that none of these frameworks require you to learn a templating language like Smarty. PHP itself makes a great templating language. I have used Smarty for several years, and honestly I think it’s more trouble than it’s worth. I note that Yii and CodeIgniter both come with an optional templating language for those who like templating languages. But as I said, it is optional.

Documentation

^Back to contents

Please take the following with a grain of salt. Docs often look different when you try to actually write an application. If the docs are inaccurate, or incomplete, you won’t know until you start writing. So far the only program I’ve written with these frameworks is hello world.

CakePHP

My first impression was quite positive. The documentation looks complete and easy to read. The tutorials were a bit hard to find (they are under “Example Applications”). When I wrote hello-world I hit a snag (404 error) and the problem was not documented. I had to edit Cake’s .htaccess files to fix it. I would have liked to see a trouble shooting section in the manual. I also had a very hard time figuring out how to remove the extra stuff (header, footer, style) that Cake includes by default.

CodeIgniter

My first impression was again positive. I note that it took me a lot less time to write hello-world with CodeIgniter than with Cake. I think this is mostly because CI is simpler, but I do think that the documentation had something to do with it.

Yii

Again, my first impression was positive, and I wrote hello-world very quickly. When I wrote hello-world I had to remove the extra things that Yii includes in the page to make the benchmark fair. Finding out how to do this was quick and easy.

Performance

^Back to contents

In this section I will discuss two different benchmarks. One carried out by myself, and one carried out by the Yii team. I believe that both are worth considering.

Key similarities:

• Both benchmarks try to measure the minimum overhead of each framework. They get the framework to print “Hello World” and nothing more.

Key differences:

• I use the default configuration of each framework. The Yii team tried to optimize each framework to print “Hello World” as fast as possible.
• Yii’s benchmark just runs die(). Mine uses the MVC: A controller puts “Hello World” in a variable which is passed to the view to print inside a basic HTML page.
• Yii used a dedicated server for the tests. I used the shared hosting service where I’m going to deploy my actual application.

Thus, Yii’s benchmark is more technically fair, but mine might give a better impression of real world behaviour. As always, all benchmarks should be taken with a grain of salt. The benchmarks are expressed in terms of Requests Per Second (RPS) so more is better.

Benchmark details:
Yii’s page describes their benchmark. For my benchmark, each framework had to run the following PHP code in a view:

<html>
<head>
  <title>Greetings</title>
</head>
<body>
  <h1>Greetings from XYZ</h1>
  <p>The greeting message is: "<?= $message ?>"</p>
</body>
</html>

Where $message is provided by the controller and its value is “Hello World”. The requests per second were obtained using the ApacheBench tool with the command “ab -t 30 -c 10 URL”. My testing environment is a shared hosting account running on:

• CentOS 4 Enterprise Linux.
• Apache 2
• PHP 5.2.6
• CPU: Quad-Core AMD Opteron(tm) Processor 2350
• Memory: 8GB

Flexibility

^Back to contents

CakePHP

CakePHP is very focused on conventions. It has a lot of expectations as to how you name your files, your classes, your database, your methods, etc. I can’t yet say whether this will present a problem for me, but I worry that it might.

CodeIgniter and Yii

CodeIgniter and Yii both look quite flexible. It is difficult to look at the documentation and say that one is definitely more flexible than the other. Both follow the MVC pattern, but they don’t feel rigid. CakePHP relies more heavily on conventions than Yii and CI.

Some examples of similarities and differences:

• All frameworks expect you to follow certain naming conventions for your controller.
• CakePHP requires that you create a model, a database and a table even if your program doesn’t require one.
• Cake calls a view automatically, based on the name of the controller. With CI and Yii you call the view explicitly. So Cake has a little more “convention over configuration”. But in turn, with CI and Yii you can call multiple view files.

Here is a code example:

/* CakePHP */
class GreetingsController extends AppController {
	var $name = 'Greetings';
	function index()
      {
		$this->set('message', 'Hello world');
	}
}

/* CodeIgniter */
class Greeting extends Controller {
	function index()
	{
		$data = array('message' => 'Hello world');
		$this->load->view('greeting', $data);
	}
}

/* Yii */
class GreetingController extends CController {
      public function actionIndex()
	{
		$data = array('message' => 'Hello world');
		$this->render('greeting', $data);
	}
}

As you can see, the CI and Yii versions don’t really look more complicated than the CakePHP version, but they give you an extra feature: you can load more than one view file. For example, you can load a special header, insert a widget, etc.

Security

^Back to contents

Security is complex because there are many possible attacks. This section is a bit long. I’ll try to stick to the point. This section is divided according to features or attacks.

Access control

For background, see Authentication and Authorization.

CakePHP

I found CakePHP’s access control complicated. You use a code generator (”cake bake”) to generate schema files and ACLs. The process generally felt complicated. Some of the terms (ACOs, AROs, Requester) threw me off initially.

I expect access control to be somewhat complex, but Cake’s was more complicated than I expected. It looks powerful, but hard to use. I felt I didn’t know what was going on and didn’t have control. And security is a place where I want to know what’s going on and want control.

Quick Test: I didn’t see an obvious way to add password strengthening. The tutorial says that if you hash passwords yourself the system won’t work.

CodeIgniter

As far as I can tell, CI does not offer access control features. Access control can be complex (groups, roles, etc) and it’s nice if the framework can help you.

Yii

I had a much easier time understanding Yii’s access control system. I won’t call it “simple”, but that’s because access control is not simple. I’d say that Yii’s access control is as simple as possible – but not simpler. As I read through the docs, I felt I knew what was going on and I felt I could modify it to fit my needs.

Quick Test: I could immediately see how to add password strengthening.

Some Yii features that I like

• Flexible: You can have access rules based on user IP or request type (GET vs POST) or rules that match anonymous or authenticated users.
• Return Url: You click on a link but the session has expired. You are sent to the login page. You login, and the system automatically takes you to the page you were trying to access. For me, this is very useful.
• Role Based Access Control: RBAC is more flexible and powerful than the traditional ACL. With Yii you can write your whole application ignoring RBAC and only use it when it solves a specific problem. So the system is as flexible as it needs to be, but your access rules are never more complex than they need to be.

Dictionary attacks

A dictionary attack consists of trying a large number of common passwords from a password dictionary. For a web app, this attack can be very effective. The recent attack on Twitter was a dictionary attack.

There are many ways to block dictionary attacks: Lock the account after X failed logins, time delays, CAPTCHAs. Each has its pros and cons. Locking accounts is easy and effective, but it opens a denial of service attack: Someone can try many logins, not to gain access, but to lock out users. Here is one I like: After 5 failed logins, you must solve a CAPTCHA to attempt additional logins. It balances dictionary attacks, DoS and user convenience.

None of the frameworks have features specific to dictionary or DoS attacks, nor do I expect them to. The real issue is, Can you implement your preferred login method with this framework? So it’s a question of control and flexibility. With CI and Yii I’m confident I can do it, but I’m not so confident I can do it with CakePHP.

Cross-site scripting attacks (XSS)

XSS has overtaken buffer overflows as the most common security hole. It works like this: Someone posts a comment on your blog and that post includes malicious JavaScript. When you view the blog, the JS code is executed. Since the JS code is coming from your domain it is “trusted” by the browser. It can read your cookies, or manipulate the page contents (e.g. “Session expired. Please enter your password”).

CodeIgniter and Yii both come with HTML filters that can remove JavaScript code from the user input. I am sad to report that CakePHP does not seem to include an XSS filter.

SQL injection attacks

SQL injection is one of the most common security flaws on web apps (funny comic). It happens when user input is interpreted as executable SQL code. For example:

$q = "SELECT * FROM users WHERE l = '$user' AND p = '$pass'";
$result = mysql_query($q);

Suppose the user enters the password foo' OR 1='1 Then the query becomes:

SELECT * FROM users WHERE u = 'foo' AND p = 'foo' OR 1='1'

Since 1='1' is always true, this query returns all users. The most common (and wrong) solution is to escape strings using a function like addslashes() or mysql_real_escape_string():

$pass = mysql_real_escape_string($pass);
$user = mysql_real_escape_string($user);

What’s wrong with this solution?

• It is error-prone, since you have to remember to escape every parameter.
• It depends on someone being able to find every possible character that can be exploited. The good guys have to find every attack, while the bad guys only have to find one. This is a losers game.

Chris Shiflett shows how to use a Chinese character to get an SQL injection through addslashes. How do we know that there isn’t some Tamil character that will get through mysql_real_escape_string?

The right solution: prepared statements

The right solution is to circumvent the problem entirely by compiling the SQL query separately from the user input. That is, to use prepared statements (aka parametrized statements, aka bound parameters). Prepared statements separate the SQL logic from user data (see here and here).

Does the PHP framework support prepared statements?

Unfortunately, of the three frameworks, only Yii offers prepared statements. I think that the lack of prepared statements is a major problem for CakePHP and CodeIgniter.

Cross-site request forgery attacks (CSRF)

Cross-site request forgery (CSRF) attacks are in some ways complementary to cross-site scripting (XSS) attacks. XSS abuses the client’s trust on the server. CSRF abuses the server’s trust on the client. Imagine that Bob visits a malicious website with the following image tag:

<img
src="http://mybank.com/transfer?from=Bob&to=Eva&USD=10000">

If Bob is currently logged in to his bank the bank’s server will receive a request from Bob to transfer $10,000 to Eva’s account.

What can you do to protect against CSRF? Other than expiring cookies sooner, the main solution is to use a secret token to authenticate any GET and POST parameters that can modify server data.

CakePHP and CodeIgniter do not seem to have any features for CSRF. Yii offers CSRF protection for POST requests using secret tokens (you have to enable this feature). To complete the CSRF protection you must make sure that GET requests cannot modify server data.

Cookie hijacking attacks

Cookies are routinely used to store sensitive information such as session IDs, user name or a password hash. Cookie hijacking is when a third party gets access to your cookie. This might be through cross site scripting (XSS) or through a packet sniffer.

How do you protect against cookie hijacking? SSL will protect you against packet sniffers. And you should take measures to prevent XSS. You can make the cookie data less useful by not storing long-term sensitive data in it. For example, instead of storing md5(passwd), use a hash that expires like HMAC(date,md5(passwd)).

To my knowledge, CakePHP and CI don’t offer any features against cookie hijacks. Yii has a cookie validation feature that uses HMAC to authenticate cookie data, so an attacker can’t modify an old cookie (e.g. change the expiration date) without detection.

Of course, this won’t help you if you do something dumb like store the plain text password in the cookie.

Other Notes

^Back to contents

These are some things which are not part of my core criteria, but I feel are worth mentioning.

Prototype

CakePHP uses the Prototype library. I have concerns with Prototype. Basically it pollutes the global name space and it can break valid JavaScript code. So Prototype doesn’t play well with your code or other libraries.

Scheduled Releases

Yii has scheduled releases. I wish all open source projects would do that.

Filtered Input

CodeIgniter has a very nice Input Class that sanitizes some of the input. It destroys the GET array, and all global variables except POST and COOKIE. This feature doesn’t fit exactly into any of my criteria, but it promotes good code and I think it deserves mentioning. I mentioned XSS filtering in the security section.

Summary So Far

^Back to contents

My experience with CakePHP was distinctly worse than with CI and Yii. The docs are generally good, but not better than CI or Yii. I can’t see any area where Cake clearly stands out above CI or Yii. It might be better for simple apps that do everything the way Cake expects. But generally Cake is more complex than CI or Yii and you don’t seem to get much in return.

Yii and CodeIgniter are a closer match. They are about equal on performance, documentation and flexibility. They are both fast, clearly documented and quite flexible. CodeIgniter seems somewhat simpler, but Yii has a strong edge in security features. I’ll have to wait until I write an application prototype with each before I form a strong opinion.

Generally I don’t like comparison tables because they over-simplify things, but I realize that people often want a quick summary table, so here is a rough one:

	CakePHP	CodeIgniter	Yii
Simplicity	Not great	Excellent	Very good
Documentation	Good [1]	Very good [1]	Very good [1]
Performance
Flexibility	Poor	Very good	Very good
Security features	Not great	So so [2]	Excellent

[1] Take this with a grain of salt. It’s hard to judge docs until you actually write your app.
[2] CodeIgniter gets a point for filtering XSS and for over-all flexibility.

The alternate title for this post is “never trust a benchmark you haven’t manipulated yourself”.

Software benchmarks, like statistics, are easy to manipulate to make them say whatever you want. I just stumbled on this blog post showing a simple benchmark where Ruby comes out considerably faster than PHP. Since Ruby is generally considered a “slow” language, this result is interesting. Curious, I copied the program sources and tried it myself. Indeed, the Ruby version was considerably faster than PHP. I really like Ruby, so to me this was an interesting result. But skeptical as I am, I took a closer look.

I perused the source code and noticed a subtle difference between the Ruby and PHP versions. The program is basically a very tight while loop. At the core of the loop the program checks if an array is empty and it does an array shift. In the PHP version, the program counts how many elements there are in the array, and tests if the result is bigger than 0. This is wasteful. The moment you find one element, you know that the array is not empty. There is no need to visit every other element in the array. In contrast, the Ruby version only checks if the array is empty, as it should.

I edited the PHP version to just check if the array is empty, just like the Ruby version. Like magic, the execution time of the PHP version dropped more than half. The PHP version went from being ~70% slower than the Ruby version to being ~17% faster.

This past week I setup a cool dual-boot system on my MacBook so I can boot into both Ubuntu and Mac. I’m a big Ubuntu fan and I was eager to put Ubuntu on my MacBook. Big thanks to the Ubuntu team who made the task easy and wrote excellent instructions.

My laptop has three main partitions: Mac OS, Ubuntu and a shared home partition that I use on both Ubuntu and OS X. So I share all my files between Ubuntu and OS X. The home partition is formatted with Apple’s HFS+ file system with journaling disabled. That is the only Unix-type file system that Ubuntu and OS X both support well.

I also setup Thunderbird and Firefox to use the same profiles directory in both. So I can go to OS X, browse the web, make some bookmarks, reboot to Ubuntu and find the exact same bookmarks, the same tabs, the same browsing history that I left with OS X. It’s quite amazing actually. My Firefox settings, my extensions, my language packs… everything works the same between Ubuntu and OS X. Ditto for Thunderbird. Kudos to the Mozilla team.

Recently Sigrid and I bought an external hard disk. It is connected to my computer which runs an SSH server. Combined with a password-less SSH key, Sigrid can access the external disk from her Mandriva box just as if it was a traditional “shared folder”. On my end, I use symlinks to hide the different mount points for Ubuntu and OS X. Sigrid can connect to my computer and find the external disk in exactly the same place whether I am currently on Ubuntu or OS X.