Transcending Frontiers

Paul Saab has written an interesting article about scaling memcached at Facebook. It looks like they’re doing some cool things over there.

I’m doing some modifications to a Copper installation for a customer that involved the addition of some new fields to their project table, namely CashContribution and InKind. They both track different donation types that can be made against a project.

I needed a quick way to populate some projects with dummy, random data, and here’s what I ended up with:

UPDATE Projects
SET CashContribution = (SELECT CONVERT(RAND() * 1000, UNSIGNED))
WHERE ID > 600;

This meant that I wasn’t looking at row-after-row of the same dummy data, and that the test system better represents the intended real-world use.

Typically these types of “25 Tips To…” or “10 Things That…” style articles have a few good points but are filled with fluff. However the article 25 Tips for Better jQuery is full of great points that cover architecture, coding conventions, compatibility and performance.

If you are looking to get a better handle on jQuery and more importantly how to use it well, I haven’t seen many better places to start.

Again I have a website where a lot of data transfer is done asynchronously and a large amount of the presentation is done using Javascript. Different users have different access to features across the site, and I can’t just rely on hiding links given the data is a simple HTTP request away. Protecting this data on the server side has always been easy to me, but I’ve typically found building the persistent abstractions I like to have far more difficult on the client side. As per usual, it’s probably just another issue I haven’t spent enough time to get a grip on.

It’s possible technologies such as Prism and Gears will help with this in the future. Unfortunately, it is the present.

This time I think I have a solution that I’m pretty happy with though. On the server side it involves using the existing HTTP response codes to indicate to the requester what happened with their request. On the client side the ajaxComplete() event is used to handle these codes.

jQuery will automatically call the function you specify as your callback in an AJAX request if the request is successful, so I’m only interested in handling failures. At the moment I’m assuming that all of my calls use JSON for their data format, but the alternative is a case I can handle later if need be. Only do what is necessary right now is a great credo I think.

So here is my event handler, it’s very simple but the documentation on the arguments to the event are a little slight. The success() call just makes a call to the function specified in the original call, hence passing in an empty array, simulating no records returned. The code I’m handling is for 401: Unauthorized, which in this case is the truth. This code will be sent back when I determine that the user is trying to access some data they aren’t supposed to. HTTP codes handle the majority of cases you’ll run into.

(function() {
	if (typeof(jQuery) != 'undefined') {
		jQuery().ajaxComplete(function(ev, req, settings) {
			if (req.status == 401) {
				settings.success([]);
				alert('You have insufficient privileges.');
			}
		});
	}
})();

The server side code is simple, it’s just a matter of sending back the appropriate header:

	header('HTTP/1.0 401 Insufficient Privileges', false, 401);

This function is specified in a global include where part of the website uses prototype, I’ve been slowly integrating jQuery. Therefore the first thing I do is check if jQuery has been defined, if it has then I register my function as the handler for the ajaxComplete event. Since it’s declared globally this will happen on every AJAX call. If the response code is 401 then first I pass back and empty array to my success handler so that the little loading notifier disappears, and then I notify the user of the error.

It seems to be a trend lately, but again this is just a very simple idea but I hope it saves someone some hassle. I know I’ve searched high and low on the topic and haven’t found a nice generic solution.

In August of this year we deployed a series of new server and storage devices which we expect will carry us through to 2011. One of the negative side effects of this deployment was that all of the sites and services we hosted on the new hardware performed significantly better than on the old configuration.

I hear you ask “how is that a negative side effect?” It meant that I put off the investigation into bench marking and tuning our server platforms simply because the move to newer hardware gave us significant gains. I had planned on doing some configuration tuning and system refinement, but the sheer increase in processing power meant that I didn’t bother with it back then.

Today, however, I decided to investigate what the addition of a PHP opcode cache would provide, and I was very happy with the results.

I came across an issue with running APC, the opcode cache developed by Rasmus and other core PHP developers, alongside the Zend Optimizer. The two are incompatible so you’ll have to play favourites and pick one or the other.

Installation of APC is easy on Ubuntu. There’s many guides online, but I found this one to be the most succinct. You end up installing the following packages (note: I didn’t have build-essential initially and PECL couldn’t build the package because I was missing make):

mlambie@prime:~$  sudo aptitude install php-pear php5-dev apache2-prefork-dev build-essential

The installation of APC is easy via PECL:

mlambie@prime:~$  sudo pecl install apc

Then enable the module by creating an apc.ini file:

mlambie@prime:~$ cat /etc/php5/apache2/conf.d/apc.ini
extension = apc.so
apc.enabled = 1
apc.shm_size = 48
apc.include_once_override = 1
apc.mmap_file_mask = /tmp/apc.XXXXXX

I ran some benchmarks using Apache Bench. The results were very encouraging. I saw a reduction in the time per request fall from 91ms to 37ms (250% improvement). The volume of requests per second increased from 11/sec to 27/sec (245% improvement). Lastly, 98% of all requests are served within 639ms instead of 1165ms (182% improvement).

Each actual apache process saw a significant memory reduction too, from 26MB down to 20MB (almost a 25% footprint saving). When you’ve got multiple processes running the reclaimed memory adds up quite quickly.

I found the figures staggering, and the improvements are actual, real things that are visible when interacting with the sites and applications hosted on prime.

It’s fair to say that we more then doubled our performance, from a single server, simply by adding PHP opcode cache. If you’re not running one on your server, you might want to consider why.

Animating table rows in the browser is problematic. You see, they aren’t block elements and as such don’t have a height or width property. Instead they take their constraints from the content inside them, and the elements that contain them. For rows this typically means they’re constrained by the containing table, and filled by the contained columns.

Today I wanted to slide a row up, and then when it had finished sliding I wanted to remove it from the DOM. Essentially giving it a nice effect when something is deleted.

Given that the height of a row is controlled by it’s content, I figured the easiest way to do this would be to wrap all of the content inside each column with a block element, in this case a div, and then resize those.

jQuery makes this extremely easy :

var el = $(options.element_prefix + id);
el.children("td").each(function() {
    $(this).wrapInner("< div />").children("div").slideUp(function() {el.remove();})
});

NOTE : The div tag in the wrapInner() is malformed because it won’t display properly otherwise. Please remove the space between the opening bracket and ‘div’.

It’s all pretty easy to understand. Essentially my root element is a row, and so for each td in that row wrap it’s content in a div. Then for the child divs in each td, run the slideUp() method. The callback in the slideUp() method says after the animation is done, remove the row. Given the speed of computers these days, no one will notice that the last few columns quite likely just vanish instead of complete their animation.

Further to my previous foray into the world of Applescript, I’ve modified my server management script to now prompt me for a sudo password. Previously I would have to tab between each Terminal window and enter my sudo password, but now I enter it once and a dynamic command is generated that looks like this:

echo <password> | sudo -S clear && sudo aptitude update && sudo aptitude dist-upgrade && sudo aptitude clean'"

I don’t like that my sudo password is displayed on the screen. I could get around this by manually editing /etc/sudoers to allow for password-less aptitude. Alternatively, perhaps I could encrypt my password inside the Applescript and send it, pre-encrypted, to sudo. They’re options I guess.

You’ll notice that the first thing I do is clear the screen, but when there’s a second or so lag it means my password is bare for all to see. I’ll consider that when I run the script.

Below is an Applescript snippet which shows you how to open a dialog box and take some simple text input:

set my_password to display dialog "Please enter your password:" ¬
	with title "Password" ¬
	with icon caution ¬
	default answer "" ¬
	buttons {"Cancel", "OK"} default button 2 ¬
	giving up after 295 ¬
	with hidden answer
if length of (text returned of my_password) is not 0 then
	display dialog "Running the application!" buttons ["OK"] default button 1
else
	display dialog "You didn't enter a sudo password!" buttons ["OK"] default button 1
end if

Having spent a bit of time with Ruby lately, I don’t like the syntax of Applescript very much, though it gets the job done.

I’ve written about Rails Rumble previously, the 48 hour event where teams turn an idea into a web application using Ruby on Rails. I was surprised to see the completeness of some of the applications that were developed this year as well as the utility of the ideas they implemented.

One of the more interesting articles I’ve seen in relation to Rails Rumble analysed the prevalence of various plug-ins and gems that teams utilised.

After a quick look there are a few points that specifically interest me :

• jQuery is the most used Javascript library, even though it isn’t the default included with Rails. I think this says a lot about where jQuery fits in the client side coding space these days. Under further scrutiny it seems clear that jQuery had an even larger base of use than shown in that graph, as explained here.
• 1 in 3 teams used a skeleton application. All of those teams used Bort. That’s a pretty overwhelming statistic for two reasons. Firstly it means you should be looking at using skeleton applications if you aren’t already. Secondly anything that you develop that could be used in other applications should possibly be a gem or plug-in. Reuse doesn’t seem to be something you just talk about anymore.
• Over 50% of people wrote tests as part of their application, and the majority of people used a Behaviour Driven Development framework such as Shoulda or rSpec (they got rSpec!). Keep in mind that even on a tight schedule most people using Rails are writing tests!
• Over a third of applications offered OpenID support for authentication. Given I don’t even remember where I signed up for OpenID this surprises me. Maybe it’s time to join this bandwagon?

I think the article gives a good indicator where you should look to implement certain parts of your application. Generally speaking, if a lot of passionate developers are using a particular library or piece of code then you’d be wise to make the same choice.

I have recently seen two examples of product / image rebranding that have caught my attention. Both for completely different reasons.

The first one is Coca-Cola’s “Mother“. About a year ago when the drink was first introduced, I decided to buy one and see what it was like. My taste buds rejected it immediately. And not in a “Oh that’s bad, but maybe in a few months I’ll forget how bad and try again” way. It was possibly one of the most horrid flavoured drinks this side of Sarsaparilla.

It got me wondering how on earth Coca-Cola let this slip through the taste-testing cracks. Given their massive customer base, well established other flavours and an already saturated energy drink market, I expected much better. I expected this drink to disappear from shelves as fast as it had appeared.

I was close. Not long after, a “new formula” was announced. Massive advertising and money spent to show that indeed their first attempt was a failure and now it’s much better. But for me, the damage was already done. After both literally and figuratively leaving a bad taste in my mouth, this is one cat who won’t be willing to give it a try again. I’m sure Coca-Cola have picked up plenty of “new recipe” Mother lovers, but I’m not going to be one of them. While you were out, I found Rockstar, and that’s where I will stay.

Note: If your sting loyal customers with a Z-grade product, you’ll probably lose them.

The second re-branding exercise I have witnessed is that of City Farmers. I remember their cringe-worthy song, their cheap looking ads and their country bumpkin image.

However this image proved successful. They are popular, and have found a great market over the last few years. I’ve become a loyal customer of theirs, and have no need to shop elsewhere for any pet related products. They are like Bunnings for pets.

However, It shocked me to drive past the massive warehouse for the second time that week, to see that it was no longer green. It was bright orange, with a brand-spanking new logo. And a much better one at that. I figured their web site would have been updated too, so I checked. It’s one of the better looking ones I’ve seen in a while.

City Farmers, you won me as a customer with your cheap and tacky image, and then you cemented me as a customer with your willingness to adapt and somehow completely change your image better than most companies ever do it. I’m not sure who has designed the web site, but they’ve certainly done well.

Note: Use the right professionals for the job and listen to their advice. Remember, they’re the ones with the expertise.