I do a lot of in-depth product evaluations in my day job, mostly involving Content Management Systems (ranging from open-source systems to boutique $10K solutions to $100K-and-up suites), but sometimes also involving enterprise-search products, and a number of Digital Asset management offerings.
Many of these products are what I call "one size fits none" products. Some of them come with a lot of functionality out of the box, but almost no licensee will simply reskin the login page and use the product in production, unmodified: Complex organizations have complex needs that can only be met by building custom applications, or extending the product's existing applications. So at the level of something like Oracle UCM or Documentum, APIs become very important. A good API becomes a value multiplier, leading directly to better ROI. A bad API becomes a productivity sink and a vendor lock-in mechanism, leading to missed rollout dates, ongoing pain of ownership, and Rogaine addiction.
For this and other reasons (such as sheer morbid curiosity), I always try to get a look at a vendor's APIs before (or in the process of) evaluating a product. And I recommend you do, too. If you are involved in a product-selection effort, get input from your developers -- have them evaluate APIs as part of the product-evaluation process. Don't wait until after the deal is inked to find out whether the product's APIs are so problematic that your rollout schedule might have to undergo serious changes.
Since almost any serious system rollout involves a significant amount of custom development, your best chance at rolling out ahead of time and under budget is to go with a product that either already implements most of the features you need (not likely) or has powerful, easy-to-learn APIs that span a wide range of abstraction levels. At the lowest level, you want direct programmatic access to system services, and the ability to create or extend core objects. At a higher level, you want access to business objects, data sources, and component libraries, either through a low-level language, or a scripting language, preferably with a combination of server-side and client-side APIs (e.g., AJAX-enabled). At the highest level, you want to be able to get to services via REST calls.
What about SOAP? It depends. If you have sophisticated requirements around security, fault-handling policies, mediation between "nodes," complex payload bundles, etc., or you need to use JMS (say) instead of HTTP, a SOAP-based Web Services API can be important. It's not a terribly performant way to go, but if your SOAP services are used in overnight batch processing or long-running BPM processes, the performance penalty probably won't matter. What you want to avoid is using SOAP to fetch stock quotes (or similar sorts of things). That's where SOAP's overhead is not worth it.
Because of the potential for rapid application development, it's important (IMHO) that any product you're evaluating has some level of scripting support. (In other words, you don't want to be forced to use C# or Java for every little thing.) Here, the product support standard languages as opposed to proprietary, one-off languages. In this day and age, there is no excuse for making customers learn a proprietary language. "Proprietariness" should not surface as language-level syntax. Once you start investing in writing code in a proprietary language, you're entering the wonderful land of Vendor Lock-In. Beware.
I find that if a product is standards-driven and has well-designed APIs, developers don't need to take classroom training in order to be productive. They can go to work immediately (or at least, same-day). This is a good litmus test, I think. If your best coders have to take 3 days of classroom training in order to get beyond "Hello World," look long and hard at the product architecture and APIs.
Another danger sign: proprietary development tooling. If it's impossible to develop a custom component without a special purpose-designed vendor-supplied IDE, think about why that is. (Eclipse or Visual Studio plug-ins are fine; that's not a problem. What's potentially problematic is when a vendor has created a one-off IDE that you have to use in order to do custom development.) A standards-driven product seldom forces you to use special tools. That's one of the benefits of standards.
Ideally, a product's highest-level APIs should be able to support many, if not most, of your most common ("80% use case") needs. For example, basic CRUD operations (create, read, update, delete) should be possible using APIs that live at a higher level than raw Java. Granted, this won't always be possible; there are many scenarios in which safe, transactional access to data requires custom-written compiled code. But my point is, you shouldn't have to drop down into Java to get a stock quote. Simple things should be easy.
Look for API features that favor declarative versus imperative-style programming. Configuration settings, hard-coded values, and things that don't change very often should be in an XML (or other text-based) file. You shouldn't have to code raw Java to change the color of a UI component.
There's a lot more to say on the subject of APIs; I've only touched on some very-high-level points here. But these are the kind of basics you need to pay attention to when evaluating a complex product like a CMS or DAM system, because it's easy to be (falsely) comforted by a vendor's claim of "extensive Web Services support" or "AJAX support," or "comprehensive Java APIs." Every middleware solution has APIs. The question is, how many hours, days, or weeks does it take to become productive with them -- and can they really do all the things you need them to do?
