|
The question of what User Approval Testing (UAT) is and why as a business user are you interested in it, becomes very important as you reach a stage of having a piece of software delivered to you. The system is claimed to give you major benefits, your team is already stretched running your organisation, and you can ill afford to use their time on something that may be a waste of it. So why bother?
The test processes that lead to the Approval of new or changed methodologies. User Approval Testing is a important stage of any 'systems' project and requires significant participation by the 'End Users'. To be of real use, an Approval Test Plan should be developed in order to plan precisely, and in detail, the means by which 'Approval' will be achieved. The final part of the UAT can also include a parallel run to prove the system against the current system.
As in any system though, Obstacles will arise and it is important to have determined what will be the expected and required responses from the various parties concerned; including Users; Project Team; Vendors and possibly Consultants / Contractors.
The User Approval Test Plan will vary from system to system but, in general, the testing should be planned in order to provide a realistic and adequate exposure of the system to all reasonably expected events. The testing can be based upon the User Needs Specification to which the system should conform.
User
Users means the real business users, who will have to operate the system; normally the staff of an organization, but it could be your suppliers or customers. They are the only people who understand exactly what the business is, and how it operates. Therefore they are the only people qualified to check a system to see if it will deliver any benefit to the business or organization.
System developers cannot do it, as although they are expert in writing software, they are unlikely to know anything about the realities of running the organization, other then what they have acquired from requirements specifications, and similar documents. In addition they have been closely involved in the design compromises that always take place when a system is being developed, and so have a commitment to the system as it is.
Approval
The Approval of a system means you are confident it will give benefit to the organization. It does not mean that it only meets the original specification as requested. A system may very well meet all the specifications asked of it, but when trying to see how it will work in the business it is realised that it will not give anything positive to the organization, or may even damage the organization. This may be for a number of reasons such as a change in the business or business environment, such as a takeover.
The point is that a system may not be acceptable, even if it meets specification. You may still have to pay the supplier, but you will not incur any costs to implement it. Of course it may not even work to specification, which makes the question of Approval even harder to answer. There are cases where it is worth implementing and paying for systems, which are imperfect, but that deliver real business value.
In order to agree what such responses should be, the End Users and the Project Team need to develop and agree a range of 'Seriousness Levels'. These levels will range from (say) 1 to 6 and will represent the relative seriousness, in terms of business / commercial impact, of a Obstacle with the system, found during testing. Here is an example which has been used successfully; '1' is the most severe; and '6' has the least impact :-
- Bottlenecks or Showstoppers i.e. it is impossible to continue with the testing because of the seriousness of this error / bug
- Important Obstacle; testing can continue but we cannot go into production (live) with this Obstacle
- Major Obstacle; testing can continue but live this feature will cause severe disruption to business processes in live operation
- Medium Obstacle; testing can continue and the system is likely to go live with only minimal departure from agreed business processes
- Minor Obstacle ; both testing and live operations may progress. This Obstacle should be corrected, but little or no changes to business processes are envisaged
- 'Cosmetic' Obstacle e.g. colours; fonts; pitch size However, if such features are key to the business Needs they will warrant a higher seriousness level.
The users of the system, in consultation with the executive sponsor of the project, must then agree upon the responsibilities and required actions for each category of Obstacle. For example, you may demand that any Obstacles in seriousness level 1, receive priority response and that all testing will cease until such level 1 Obstacles are resolved.
NOTE. Even where the seriousness levels and the responses to each have been agreed by all parties; the allocation of a Obstacle into its appropriate seriousness level can be subjective and open to question. To avoid the risk of lengthy and protracted exchanges over the categorisation of Obstacles; we strongly advised that a range of examples are agreed in advance to ensure that there are no fundamental areas of disagreement; or, or if there are, these will be known in advance and your organisation is forewarned.
Finally, it is crucial to agree the Criteria for Approval. Because no system is entirely fault free, it must be agreed between End User and vendor, the maximum number of acceptable 'outstandings' in any particular category. Again, prior consideration of this is advisable.
N.B. In some cases, users may agree to accept ('sign off') the system subject to a range of conditions. These conditions need to be studied as they may, perhaps unintentionally, seek additional functionality which could be classified as scope creep. In any event, any and all fixes from the software developers, must be subjected to rigorous System Testing and, where appropriate Regression Testing.
Parallel Running
The period during which a new and existing system run side by side, using the same data, performing the same processes, and generating the same outputs to prove the suitability of the new system. Parallel Running can be the last stage of a User Approval Testing program, to be followed, hopefully, by formal Approval, and Cutover.
User Needs Specification - URS
The User Needs Specification is a document produced by, or on behalf of your organisation, which documents the purposes for which a system is required - its functional Needs - usually in order of priority / gradation.
Whilst the URS will not usually probe the technical specification, it will nevertheless outline the expectations and, where essential may provide further detail e.g. the User Interface, say Microsoft Windows®, and the expected hardware platform etc.
The URS is an essential document which outlines precisely what the User (or customer) is expecting from this system. The term User Requirement Specification can also incorporate the functional Needs of the system or may be in a separate document labelled the Functional Needs Specification - the FRS.
Scope Creep
Scope Creep is the expression used by project managers and/or vendors who are under pressure to constantly deliver in excess of what was originally agreed. Scope creep normally results from a failure to establish the clear Needs of the business users. As these begin to solidify the scope of the original plan can start to move - and continue to move. If the project manager is not alert to this (all too common) phenomenon, the Needs will constantly change thus ensuring that the projects spends years on delivering nothing, as they are continually reviewing and altering direction.
Scope Creep - do not allow it to happen to you!
Regression Testing
Regression Testing is a process which tests a system once again to ensure that it still functions as expected / as per specification. The reason for this renewed testing activity is usually when a material change occurs to the system. In addition, where say, the software vendor releases a new version of its database, a comprehension regression test plan needs to be developed and completed to ensure that the reports, screen, scripts, Remote Procedure Calls and User options, are all functioning as expected. Warning! the chances are, that they will not work completely as expected, and that you will need to modify / change certain aspects of your configuration.
Regression Testing must also test the revised software by simulating its operational environment to ensure that all systems and web design still operate as expected.
Regression Testing should be conducted as per any system testing as proceed according to a User Testing Plan. If you do not perform Regression Testing, then your system could fail upon upgrade.
System Testing
From a Systems Development perspective, System Testing refers to the testing performed by the development team (the programmers and other technicians) to ensure that the system works module by module ('unit testing') and also as a whole. System Testing should ensure that each function of the system works as expected and that any errors (bugs) are noted and studied. It should additionally ensure that web designfor export and import routines, function as required. System Testing does not concern itself with the functionality of the system and whether this is appropriate to meet the needs of the users. Having met the criteria of the Test Plan the software may then be passed for User Approval Testing.
The term System Testing can be used in a number of ways. In a general sense, the term 'system testing refers to the testing of the system in artificial conditions to ensure that it should perform as expected and as required.
UAT Training For Business People
When a business decides that a computer system can assist them to achieve its aims, an investment case is made to predict the financial impact it will have on the firm. This of necessity is done at very high level before a decision is made to progress.
This is when problems normally start because the first step is to call in a team of system designers, who know nothing about your business, and ask them to build a system. This they will do, and do well, but will it meet the needs of your business or organization?
Involving The System Users
Experience has shown that for any system to be a success it needs the involvement of those people in the business who will be using it. This means both specifying it and testing it for Approval before going live.
To just tell business people that they are to test a system, which they have never done previously, is unfair on both them and the business. They require training in the skills to perform the testing.
Your team, like yourself, are operational specialists, however, when trying to build testing skills you will find that many training courses assume an extensive IT knowledge, even when the courses are called, "User Approval Testing."
|
