Pass/FailAnyone who has ever played a part in an Identity Management implementation knows that testing is an essential step in the process, but few understand how to test effectively. There has never been – and will never be – a bug-free implementation. There will always be broken links, inadequate code, and behaviors that simply don’t match what was expected. That’s why we do testing in the first place. As code is implemented, different methods of testing are utilized; including integration, functional, non-functional, and user-acceptance testing. In this three part series, I want to share some of the lessons we’ve learned about testing.

Specifically, I’m going to talk about how we apply the Plan, Buid, Run (P-B-R) mentality specifically to the testing and QA (Quality Assurance) stage of a project.  The reason P-B-R is an effective model is that it forces us to think through what we’re trying to achieve and give thought to how best those objectives can be met.  This is just as true in the QA phase as it is overall for the whole project. People are often anxious to get in and poke around their new IAM system, especially since the most they have seen up to this point may have been some mockups, design documents, and maybe a few demonstrations. Running through the workflows, getting to see how everything works, entering invalid data, trying to break the system – that’s the fun part. But if you don’t conduct your testing according to a set plan, it will run long and you will inevitably fall short. Don’t allow yourself to skip the valuable planning steps; it will not save any time.

Lesson #1: Clearly Define Test Objectives

They key to successful testing is documentation. You not only need to know what you are testing, but you will need to define how your tests will be considered a success. As new code is developed and moved to the testing environment, functional testing should occur to ensure that the code which was delivered meets the functional requirements outlined. However, your test cases shouldn’t just encompass the positive results, you should think about how it should fail as well. For example, when testing the ability for a user to perform their own password reset, you may be inclined to just test and make sure the password resets; but you should also make sure the system prevents people from selecting a password that doesn’t meet the minimum requirements. Your test objectives should ensure that all expected behaviors are tested.

Prior to go-live, user-acceptance testing is done to validate that the solutions meet the business requirements which were mutually agreed upon at the start of the work. Proper documentation will help you keep everything in order.

Lesson #2: Create a Plan

In order to fully test your new IAM solution, you will want to perform various levels of testing and the best way to keep all of your testing efforts coordinated is by creating a test plan. Your test plan should include your overall objective and purpose, which will enable you to conduct your tests within a set scope.

You should also include your test strategy, which will outline all of the various testing you plan to conduct including Unit, Functional, Integration, System, Usability, Performance, etc. For example, a functional test will focus on ensuring that the features being tested work as expected, whereas acceptance testing will validate that the system meets all of the business requirements.

A well thought-out test plan will ensure that you cover every aspect of your implementation, but don’t try to boil the ocean. It will be impossible to test every possible combination of users, roles, request types, field values, etc., so it’s important to consider how you will test using broad strokes in order to come up with comprehensive test sets that will apply to the majority of your end-users.

In Part 2 of 3, we’ll get into more of the details on building out individual test cases. 


SCUID Operations Data Sheet

Bryan Cole

Bryan Cole