Methodology

Web Applications

There are two distinct types of web application:

REST

Many applications are little more than a customised database front end. Users enter data, edit data, and extract data through reports. Whilst these applications can certainly be developed along general guidelines (see the following type), a relatively new application architecture is taking the web development world by storm. Application frameworks such as Ruby on Rails stress Representational State Transfer, typically abbreviated to REST, an the application architecture. A full description of REST can be found in the original paper by Roy Fielding. Each application resource (typically a database row or business object) is represented by a URL, e.g. www.example.com/people/3761. This resource is acted upon by the HTTP verbs; GET will retrieve the data, POST will update some attributes, DELETE will remove it, and PUT (e.g. to www.example.com/people/new) will create a new resource.

Stateful Web

Unfortunately, not all applications fall into this category. Some web sites follow a more traditional work flow, and must maintain state between requests. There are many different ways of achieving this, and the industry has not settled on any one as better than the others.

If an application can be RESTfully architectured, this is an ideal solution. There is no state to maintain, and HTTP provides a perfect interface. Furthermore, REST is cache and web farm friendly (although scalability issues are not part of this study). REST URLs are also clean - they clearly say what they mean. Bookmarking, back buttons and forking are no problem for a RESTful site. As this is a problem which has been adequately solved, it will not be considered further.

The Test

For the purposes of this study, a very simple application will be developed in a variety of frameworks:

1. In the first screen, the web site will ask the user for their forename, and whether they want to give their surname.
2. If the answer is positive, a second page will ask for their surname.
3. Finally, the friendly application will greet the user by name.

Each page (except the first) will have a means of navigating back to the previous page.

For basic security, the user cannot cause markup or script to be executed in any page.

Although this application is trivial, it has two of the critical elements which confound web application developers:

• Each page after the first depends on data gathered by previous pages.
• One of the pages is optional, depending on state. This becomes particularly awkward when navigating backwards, or by bookmark.

Cheating

There are a variety of ways in which we could "cheat", relying on the application's simplicity to take shortcuts. As an extreme example, we could put all the variables into the query string, e.g. simpleapp?first=John&last=Smith. In order to make a sensible comparison, we will use each framework as intended, and treat the application as if it was of moderate complexity.

What are we Looking for?

The application should be quick and easy to develop. Ideally, the back button, tab/window duplication and bookmarks will work in an intuitive fashion. Generic error pages or timeouts are unacceptable.

What are we not Looking for?

• Object Relational Mapping. Frameworks sometimes include an Object Relational Mapper, e.g. ActiveRecord in Ruby on Rails. Although highly valuble, and worthy of a study in their own right, ORMs are orthogonal to Web Applications. Frameworks do sometimes feature particularly good integration with their packaged ORM, but this would take the study out of scope.
• The line between a Web Application Framework and Content Management System can become blurred. This study will consider internationalisation and document management to be features of a CMS, and are not expected to appear in a Web Application Framework. (Of course, the presence of these features may be desirable in practice.)
• Authentication and user management straddle the line between WAF and CMS. Although such features are desirable, their absence is not considered to be a failing.
• Development time is an important metric; however, this is largely dependant on the developer's skills and experience. Any individual's skills and experience will vary widely across different platforms, and there is no clear way of standardising this. The best we can do is to detail all the steps required to build the application on a particular framework, which should give a reasonable (although unquantifiable) indication of required effort.
• Lines of code, and similar metrics, are popular measurements of development effort. This may be appropriate when comparing similar projects, but when the projects, and particularly the underlying platforms, differ, this metric becomes meaningless. One line of code (or function point) may require much more effort than another, particularly if a tool assisted in its generation.
• Scalability is a complex topic, difficult to test in the real world, and is beyond the scope of this study.

What Else Should We be Looking for?

• Validation is an important framework feature. Values may be required or optional; they may have to fall within a particular range. Validation rules can be complex functions of multiple values. Validation must be performed on the data sent by the browser, but it is also desirable to validate data before it is submitted (using JavaScript). The varying and complex requirements for validation put a thorough evaluation beyond the scope of this study.
• POSTing values to a server requires a fresh page to be sent to the browser in response, even if only a small portion of the page needs updating, creating a poor user experience. The alternative, popularly known as AJAX (Asynchronous JavaScript and XML), uses XMLHttpRequest to transfer small of amounts of data between script running within the page and the server. AJAX is an important technology, and the support (if any) provided by a web framework should certainly be considered, but is beyond the scope of this study.