Traditionally, User Acceptance Testing (UAT) is performed at the end of a long software development cycle by the intended audience under real-world conditions. By then, engineering has performed a battery of technical tests to ensure the software works as expected. However, in some cases, engineering’s understanding of the business requirements and user needs versus what the user was actually looking for are two very different things. This is where UAT usually comes in.
But UAT is not simply for new software development. It is just as applicable for environmental changes such as an application upgrade or ensuring that application functionality works under a different operating system version.
However, since this can be a resource-hungry, expensive, and long-winded process, many enterprises would rather skip it. Yet, a well managed UAT process allows you to find and fix major (or minor) user experience or functionality flaws that could prevent adoption or halt productivity before rolling it into a larger audience.
So what about today?
With Windows as a service, enterprises are starting to understand the potential impact of the UAT requirement against a multitude of new operating system releases. Microsoft are telling us not to worry – most applications will work just fine. Do you really need User Acceptance Testing as more and more enterprise IT environments are managed in “Business-as-Usual” (BAU) or “Evergreen IT” mode? 
Today, I want to take a closer look at why UAT is even more critical today than ever before, what an efficient testing process looks like and what options you have.
Why Is UAT Becoming More Important In The Future?
Today’s IT teams have to face completely different challenges compared to 10 to 15 years ago. While back then, we struggled with big-bang OS rollouts every three to four years, we are faced now with a mix of on-premise and cloud-based applications in various lifecycle stages and very fast-paced upgrade timelines. Our entire estate management has become much more fluid, and therefore incredibly complex.
While larger IT projects were overwhelming and almost always over-budget and over-time, there was an end in sight at some point. You could come up for air once in a while, rethink your strategy, and start fresh. Also, your application discovery, packaging, and testing was done in one more or less efficient fell swoop before you could move on into your deployment phase.
Once the project was complete, standard processes were engaged for UAT of new applications into the environment, but the existing applications did not really require any further thought unless the vendor forced an upgrade. However, with a greater velocity of change and faster upgrade cycles that come with new IT management models such as Windows 10 Branching, application testing and other migration-tasks must now be considered an ongoing effort.
What Does An Efficient UAT Process Look Like?
To start your Windows 10 Branching UAT process, you will need to first identify which applications need testing and run through a thorough discovery process to unearth the relevant MSI or source files and any other resources required. You may have a DSL (Definitive Software Library) to help, but if not, your inventory tools such as SCCM will provide a list of what is out there. Once you have identified all the required assets, you will have to prioritize and notify the application owner within your environment.
Traditionally, if you want to perform a User Acceptance Test for a business application in a new environment (e.g., Windows 10) you would then follow this process:
- First, your IT team installs the application on a Windows 10 machine that exactly mirrors real-world conditions (e.g., permissions, authorizations) for the type of business users that will be using this app.
- IT will provide the user with the machine, the log-in credentials and a mechanism to capture the results.
- Then, the end user will open the app, run through usage scenarios (user stories). and record their behavior and outcomes.
- After the test is completed and the results are recorded, the end user will give back the machine.
- IT will wipe it clean and prepare it for the next application test.
Depending on how you structure this process and which testing model you choose to follow, this can take a long time and many resources. It can lead to inconsistent results as it is much based on the willingness and diligence of the tester.
The modern approach to UAT would be to use virtual machines. With the right tooling, you can easily spin up pools of Virtual Machines in parallel, notify your users automatically including all credentials and instructions, and record testing sessions to provide evidence of successful testing. After the test is completed, a workflow can route the app forward (success) or backwards for manual testing (failure), and the virtual machine can be killed off in an instant — resulting in significant time and cost savings!
As you will be testing many applications on an ongoing basis, you will have to think about how you will prioritize which applications to test first and last. There are several options for structuring your Acceptance Testing — i.e., by