Berro Visual

The problem

We were fortunate enough to deserve the trust of our local internet access leaders to do the task of revampimg their internet portal. The implementation of this portal had several recognized problems. These problems doomed its popularity with a clear reflection on the user access statistics. The recognized problems were slowness, unreliability (servers refused to serve pages from time to time), search engine blindness and usability.

Slowness

The portal is used as a main entry point for ISP users to access their account and webmail.Beeing heavy on javascript and images slowed page rendering, which drove users away.The YSlow stats of those days revealed that 149 HTTP requests were necessary to render the page with a total weight of 718KB with an empty cache. That’s heavy! What about with a primed cache? 148 requests and 283Kb. Better, but still very heavy.It loaded 11 javascript files, including several javascript frameworks on the homepage, which slowed rendering.It used 6 stylesheets on the homepage, instead of compressing it to one.It rerely used expires headers.It used no compression.It had 2091 DOM elements on the homepage.It rated an overall YSlow grade of F.

Unreliability

The portal implementation had technical issues that prevented page rendering, from time to time (several times a day), yielding server errors.

Search engine blindness

The portal interaction was mainly based on AJAX popups and lightboxes on top of the homepage itself. There was rarely an link to a new page, every click generated content fetched by AJAX. That’s not a very sound techique for a portal thatis heavy on content, since google and other search engines don’t know how to crawl it.AJAX loading would be fine and dandy if the URL changed dinamically (as gmail does), but this was not the case.

Usability

Another side effect of all this AJAX content loading was that no one could send a link to a friend containing an interesting piece of new, or any other content for that matter.The implementation broke the web. The people that implemented this portal had thrown REST out the window.

Initial goals

Well, the portal by the time had all these problems.So, we needed  to:

• Reduce the load on the homepage (and other pages also, please)
• Make content linkable and crawlable
• Make the service reliable
• Make the rendering faster by following these rules

It would be nice to have:

• Search engine friendly URLs for SEO
• Make the page rendering incredibly fast (at least on the homepage)
• Make the portal navigation friendlier and more useful

The Solution

Early on we decided that we had to redesign te portal so it would be less dependent on images for layout.And since we had to reimplement it (since we did not have the source code), we shoud get rid of the horrendous Microsoft infrastructure.Rails started to seam a good fit for this project, since most of the content is cacheable.The timeframes proposed for the several stages of the project were also tight, which was also a good reason to use Rails.

Search engine and usability optimizationPage URLs

We used a combination of two techniques for creating great search engine friendly URLs for the portal contents:1. Rails routes. Rails really makes it wasy to construct URLs, so a) you don’t have to spread the URL constructing knowledge throughout your code and b) and your controller code gets automatically called.2.Slugs. Slugs are what we call strings like “this-is-the-latest-news-in-the-world”. This slug string can be constructed automatically using model callbacks where you populate the slug attribute (based on the article name, for instance).

Image URLs

We used the same techique for building search engine friendly image URLs in order to increase search engine optimization.We used this great fleximage plugin by Squeegy to store, render and operate on images.

Titles

We never forgot to put a different and meaningful title on the page. Sure, it’s cumbersome if you have many controllers, but worth it for the user and search engine optimization.

Homepage loading and rendering speed – YSlow

The YSlow grade on the new portal homepage was a B. We didn’t get an A because YSlow insisted we used a CDN for our static files, but most of the portal users were inside the ISP, so it was as fast as it could get. Also we had publicity, and YSlow insisted we should cache the javascript calls, which we can’t.Overall, I think a B grade is very good, specially comparing it to the previous F grade and to the C, D, E and F grades of the majority of big portals – feel free to try YSlow on them.

Rails cache and Nginx

We chose to use Nginx as a webserver behind our mongrel cluster instances. Nginx has a very good reputation for handling static content, and since most of our content is page-cacheable, this was a good choice.Later on, when we were doing a simple load testing, the memory and processor profile were amazinlgy quiet – almos no CPU activity…

Nginx and browser cache

We configured Nginx to add cache expire headers to all pages and assets, minimizing the needed HTTP  requests and the page loading time.

nginx compression

Nginx compression configuration is a breeze, and public and cached files were compressed, which saved about 25% on transmission time.

assets.netmadeira.com

Browsers are configured to open two TCP connections to one server. For an application that needs various files (as most web apps do), this can slow page loading and rendering, usually lost on javascript, css and image loading.Rails allows you to do the serving of public static files from a different host name than your server main name. We used this capability, which accelerated assets loading.

Reliability

We are using monit to monitor resources and restart mongrel cluster instances that get too big.

Us and the clients are very pleased with the reliability of the current platform.