Deliverable Acceptance Criteria is often cited as the critical measuring stick for project success. The basic idea is simple, if all deliverables meet all predefined Acceptance Criteria (within scope, schedule, and cost constraints), the project would naturally be considered a success. Is this really the case? If we dig deeper, we find this logic is overly simplistic. Acceptance Criteria is actually one of several methods which must be employed to identify the success of a project. Ultimately, meeting stated Project Goals and Objectives that an organization desires to attain is probably the most important measure; but this must always be balanced with the triple constraints (scope, schedule, and cost).
One challenge with Deliverable Acceptance Criteria is that it must always be clearly defined and measurable. This can be problematic, as some criteria are difficult to measure or are plainly subjective in nature (based on human preference). However, subjective criteria can be a critical success factor on a project; this is especially true in software application development. Usability usually refers to User Interaction and Functionality of an application (i.e. the product). This can be categorized as measurable criteria, in essence, we can test for Usability (e.g. conduct pass/fail test cases). For example, we can develop tests to determine if an alarm clock on a smart phone OS works or not.
The area where Acceptance Criteria can be muddied is in the realm of User Experience. User Experience usually refers to the End Users’ attitudes and opinions of the product itself, not just whether it works as specified. In some ways, this can be measured, such as conducting Focus Groups or using surveys. However, the answers gained here aren’t always clear cut. To illustrate this further, consider the launch of the iPhone and iPhone 3G. It is clear Apple developed Project Goals and Objectives, defined measurable Acceptance Criteria, and conducted numerous Usability tests prior to launch. However, the product delivered went above and beyond, the iPhone 3G revolutionized smart phones and became the benchmark for its competition. At that time, there were many smart phones on the market and competition was fierce. However, Apple was already well known for its exceptional quality and was quite adept at tapping into the voice of the customer; they understood User Experience very well. The rest is history, the smart phone they introduced set the bar for the industry for years to come. Many manufacturers attempted to replicate this success. Touch screens, add on features, and state-of-the-art hardware (and software) became standard, however none have matched this company’s success. The underlying reason for this success was User Experience (from packaging, to the sales experience, to the available software applications, and the phone itself). Many manufacturers have initiated projects to launch similar smart phones, meeting Acceptance Criteria and passing countless Usability tests; however none have matched the User Experience.
So what does this mean for your project? Well, all Deliverable Acceptance Criteria must be met in order to have a chance at succeeding. Your products (what your project produces), must also pass its Usability tests. If your product is made to specification, but doesn’t work, what good is it? Lastly, the User Experience must meet or exceed customer expectations. If we have a product which meets Acceptance Criteria and has passed its Usability Tests, yet is still one that the customer dislikes (based on subjective human preference), we will likely not meet stated Project Goals and Objectives; a project failure.