To get a handle on this, I decided to use YSlow to evaluate the "loadability" of various vendors' home pages. If you haven't heard about it before, YSlow is a Firefox plug-in (or "add-on," I guess) that analyzes web pages and tells you why they're slow based on Yahoo's rules for high performance web sites. (Note that to use YSlow, you first need to install Firebug, a highly useful add-on in its own right. Every Firefox user should have this add-on. It's a terrific tool.)
It's important to understand what YSlow is not. It is not primarily a profiling tool (in my opinion, at least). The point of YSlow isn't to measure page load-times. It's to score pages based on a static analysis of their design-for-loadability. There are certain well-known best practices for making pages load faster. YSlow can look at a page and tell if those best-practices are being followed, and to what degree.
YSlow assigns letter grades (A thru F) for a page in each of 13 categories of best-practice. I decided to run YSlow against the home pages of 35 well-known WCM and/or ECM vendors, then calculate a Grade Point Average. The scores are posted below.
Please note that the full results, with a detailed breakout of exactly how each vendor did in each of the 13 YSlow categories, is available in a (free) 121-page report that I put together over the weekend. The 1-megabyte PDF can be downloaded here. It contains some important caveats about interpreting the results, and also talks about methodology.
Hot Banana (Lyris)
Maybe others can undertake similar sorts of testing (I'd particularly like to see some actual timing results, comparing page load times for the various vendor pages, although this can be notoriously tricky to set up). If so, let me know.
Does it mean a whole lot? Not really. I think it just means some vendors have more of an opportunity than others, perhaps, to improve the performance of their home pages. But a lot of factors are at play any time you talk about Web site performance, obviously, and therefore it's not really fair to form any kind of final judgment based on the scores shown here. Use it as a starting point for further discussion, perhaps.