Transcending Frontiers

I wanted to integrate some branding functionality into an application we’re developing and so I needed test file upload functionality. We’re using Webrat for integration tests, though this will likely change as we increase the amount of Javascript in the app. I added Paperclip to handle the file attachments for logos, and everything was working.

When I added validation to the model, making sure that the file being attached was an image, this broke the tests. It didn’t seem to matter what type the file was, it would fail no matter what on the file type validation.

I used ruby-debug to debug my test and it seems by default Webrat sends file uploads as plain text. It does have the option to specify the file type when attaching the file, so the easiest way around this is just to specify the MIME type for the file. Now my Cucumber step looks something like this :

When /^I attach "([^\"]*)" image to the "([^\"]*)" file field$/ do |filename, field|
  type = filename.split(".")[1]

  if type == "jpg"
    type = "image/jpeg"
  end

  attach_file field, File.join(RAILS_ROOT, test_asset_path, filename), type
end

Obviously this will need some work as I progress, but it works. At this stage I have an assets folder in my features folder to store any files that I need for my tests.

On the confirmation end of the test I just have a simple tag test to check that the image tag is displaying, and it contains the correct src attribute :

Then /^I should see tag "(.+)"$/ do |selector|
  (Hpricot(response.body)/selector).should_not be_empty
end

So in my feature test I have :

Then I should see tag "img[@src*='']"

This just confirms that there is an image tag that contains the file name of the file that I uploaded in the test.

Ward Cunningham has been a contributor to a number of ideas that are used every day by developers and users alike. Besides developing the first ever Wiki, he is also a proponent of Extreme Programming and has contributed to the use and understanding of patterns in OOP.

I ran across a YouTube video he did, explaining the costs and benefits of proper software development practices, using debt as a metaphor. He said he developed this when he was trying to explain the business value of certain practices of good development to someone in the finance realm.

I’m summarizing some of what he said, and I’d recommend you watch the video.

Borrowing Money
Borrowing money allows you to do something right now, but you’ll be paying interest on this in the future. If you continue to borrow then eventually you’ll end up using all of your revenue to pay interest.

Revenue is essentially your development time, it’s a somewhat fixed resource. Debt is what is built up between an optimal implementation against a sub-optimal implementation. Payments are the time you take to refactor the code afterward. Interest is the time difference it takes between the easiest way to add a feature and the way you have to go about adding a feature.

You can continue to borrow to cover interest but at some point then you either increase developer time (more people) or you spend all your time bug fixing and patching rather than adding features. It’s fine to push out parts of the project that are sub-optimal, as long as you go back and improve these later. If you don’t then you end up making your system harder and harder to add features to and so you are paying all of your revenue into paying off your debt.

The idea is that you can put out software that is the best solution you have at the time, as long as you go back and repay the difference between the best solution and the solution you have implemented at a later date. If you continue to add these ‘best at the time’ solutions without using the knowledge you gain along the way to fix the old solutions you implemented, then the system as a whole becomes poorer and poorer in quality. I guess to use an old saying, “A stitch in time saves nine”.

He further explains that he doesn’t think you should build debt through poor code or design. What he means by borrowing is to implement with your best understanding at the time, and then to go back and revisit your design once you have a better understanding of the system. You should always be implementing in a way that documents what it was that you were trying to achieve, it could simply be that what you needed to achieve is different or changes over time.

Smashing Magazine recently published a great article on choosing a domain name for your site, product or company. It’s a problem that many of our customers face, so if you’re in the market for a new domain, or are interested in how your domain stacks up against their recommendations, be sure to check it out.

FCKEditor is a third party WYSIWYG editor for websites that is used in a great number of applications across the web. The problem with using third party applications is that you can never be sure of the quality of the code, or of the security model they’re enforcing. In the case of FCKEditor the default installation can be extremely insecure and information on how to secure it is pretty thin on the ground.

There are a few things that can be done to improve the security of FCKEditor after installation so hopefully these few pointers will help.

Integrate Your Own Authentication
You can integrate some basic authentication fairly easily using an htaccess file and the php auto_prepend directive. This directive will basically tell the php engine to prepend any script with the file that you specify, it’s essentially the same as issuing a require() call before each page is run. We use this to run our own authentication scripts before access to any part of our application is run. There are parts of FCKEditor that are freely open to the public, so this is quite important unless you want your site to have anonymous users uploading anything to you.

Turn Off The PHP Engine in Your Upload Directories
FCKEditor does have some very basic checking included to try and stop users from being able to upload any sort of executable code to your application; but none of it works very well. The best way to circumvent this is to stop PHP from functioning inside your upload directories. What this means is that although a user could circumvent your security and upload some malicious code to your server all that will happen is your web server will serve back the content as plain text.

Remove FCKEditor’s Testing Pages
FCKEditor includes some basic forms that allow you to check that your configuration is correct for uploading and browsing your website. These forms if they remain publicly available provide a very handy access point for users to upload content to your site that they shouldn’t be. The file exists in /editor/filemanager/connectors/test.html and is there only to allow you to test the connectors, you should be deleting it. This step will not stop those progressing up to step one or two, but I see this as putting up some fake security cameras, it’ll stop some lazy criminals.

Hopefully these tips will help anyone that is using FCKEditor and is worrying about any holes in their security this might open up. Obviously these can be generically applied to any application that allows uploading to occur, however with FCKEditor being quite prominent in older applications it’s probably worthwhile just having a look at your app to see how secure your editor really is.

These days I’m using a lot of asynchronous calls to get data and dynamically build the UI on the client side. It generally allows a far nicer experience to be provided to the user, being able to update parts of the UI without reloading the whole page is one of the first steps to your apps being able to wear a Web 2.0 moniker.

The general pattern for me these days has become :

var callback = function CallBack(data) {
    ... Do Some Processing ....
}

var input_data = GatherData();
MakeRequest(target_url, data, callback);

I tend to use jQuery and so my callback is passed in the returned data from the target_url. My call back function then generally performs some tasks on the UI based on what it receives.

The problem though is that in this pattern you can’t get any data from the context when MakeRequest() is called into the scope of CallBack. It’s a scoping issue that falls outside of this little post, but if you’d like an explanation of how Javascript handles scope of variables then you can Google for Javascript scope chain or take a shortcut to this article. Essentially when the call is made to CallBack() all it will have is it’s own variables and any globally accessible (window) variables.

This week though I had a thought and worked through it with one of my work mates, Tony. If you passed in a function that had the scope that included the context you wanted to pass, then maybe you’d be able to access whatever data from the callee you wanted. It turns out that this does work.

The way to do this is pretty simple, and I used it to create a simple function that would grab data and then populate a select element with options. In this case the context I’d like to keep is which select element is the target.

Say that I needed to run this over a bunch of select elements in quick succession then as the callbacks were issued they may end up out of the initial execution order, so the target element isn’t reliable if it’s been stored in a global variable. I could pass it in as a variable that would come back from the page that is returning the data but that just smells bad to me. I think potentially JSONP is an alternative too, but this felt like the right way.

$.getJSON(url, input_data,
	(function(target_element) {
		return function(response_data) {
			var html = [];

			for (var i = 0; i < response_data.length; i++) {
				item = response_data[i];

				html.push('' + item['value'] + '');
			}

			target_element.children(':gt(0)').remove();
			target_element.append(html.join(''));
		};
	})(target_element)
);

Essentially the main thing that has changed is that we are now running an anonymous function at the time that the AJAX call is issued. This anonymous function itself returns a function that matches the signature that jQuery is expecting for the callback function. The scope in which this function runs contains the target_element because it was passed into the anonymous function as a parameter. I’m tempted to say that it’s all crazy Javascript scoping, but in reality it’s very cool and very powerful.

If you want to see the execution order of this then just put a some logging into the anonymous function and the callback function and you’ll see what I mean. It will probably make it easier to see what is going on too.

I’ve run into issues trying to get around these problems before and thankfully as mine and the team’s knowledge of Javascript increases I’m finding better and better solutions. I thought my previous method of approaching this problem was quite hacky but now I don’t feel so dirty.

Thanks to Tony too for working through this with me!