What are Specialized Testing Services?
Most organizations know the value of testing their software. What they may not know is the value that specialized tests add to a test effort beyond high-level functional blackbox or user acceptance.
Imagine building a house but neglecting to add the roof, or making a sandwich but forgetting the bread. This is what often happens in the testing process.
There are many different types of tests and test efforts that, when combined with blackbox testing, provide an effective and comprehensive assessment of the overall application, including increased opportunities to uncover critical defects. Unfortunately, these tests are often overlooked and worse, test teams often lack the unique skill set needed to oversee their implementation.
How IQ Can Help
IQ’s Test Strategy & Design Specialists have years of hands-on experience in strategy, planning, designing, and executing various specialty testing services. These testing services are data migration and conversion testing, SOA (service-oriented architecture) and backend data integration testing, performance test strategy and design, multi-system end-to-end data flow testing, smoke & regression suite strategy and design, and automated functional testing.
While your organization may have the resources to perform some basic functional testing, skilled experts are required for serious testing challenges. We firmly believe that large-scale, complex, multi-system integrations require strategy and analytical foresight to be effective, scalable, flexible, and repeatable.
Our Test Strategy Specialists are not only experts in testing, but in environment management, configuration management, change management, and defect management. Such expertise is vital in building an effective and viable test effort.
The Details – Test Strategy & Design: Automated Performance Testing
We’ll help you determine the type of performance testing you require to effectively test your application’s system response times at different predetermined loads (user and/or data), stress, and volume. Then we’ll review your performance acceptance requirements based on historical data and/or contract SLAs. Next, we’ll assess the available testing software and environments to perform the actual tests.
If your organization doesn’t have performance testing software, we’ll help determine the best means to acquire the needed tools or to find hosted solutions. We’ll then work with your project, environment, and DBA teams to plan and design a suite of repeatable tests with detailed acceptance metrics from the server, O/S, and database perspective as well as from a user and response time viewpoint. We’ll design a repeatable manual test suite that will be developed into an automated suite of tests. Most importantly, we’ll plan and design the test data library required to execute the tests. For example, to execute an approval workflow on a sales order, the exact same test data needs to exist in the same state. So, if this test needs to run 500 times, then there needs to be 500 sales orders, ready to be approved.
When the tests, test data library, and environments are ready, the tests will be executed and we’ll provide three levels of reports: raw test data reports, detailed assessments of the results as compared to the predefined metrics, and an executive summary stoplight report.
Ideally, you’ll be able to re-execute the test suite over and over again, comparing the relative performance of your application against prior releases/versions to validate an increase or decrease in performance.
IQ’s Test Strategy Specialists can manage your existing testers (or developers), whether on or offshore, to execute the performance test plan effectively. Or we can help you augment your project team temporarily with skilled testers to effectively execute these tests, all the while managing the test effort.
The Details – Test Strategy & Design: End-to-End Data Flow Testing
Many times, system integrations cross multiple IT/Business divisions and no one has a view of the “Big Picture” of data flow between multiple applications. Building this view out from the requirements and design is the initial step in identifying defects designed into the applications, which is where many defects are found—at system boundaries. IQ’s Test Strategy Specialists can help map and trace all of the data objects flowing through your application(s), including imports from 3rd party source systems and data files as well as exports to 3rd party systems, data files, and reports. The next step in End-to-End testing (E2E) is to understand the rules as to when and where data objects can and cannot be modified within the “Big Picture” data flows and to ensure that the data flows can support changes while maintaining integrity. All of this can be accomplished as Static Testing, without any hands-on test execution of the system.
One of the most challenging aspects of end-to-end testing is planning and designing the test data library. In a complex data flow, it’s difficult to test functionality near the end of a flow without first generating data at the beginning and building data from the start. For example, in a basic sales order system, one cannot thoroughly test the fulfillment processing of an order without first creating the order to be filled. In an ideal world, there would be enough time to create all of the set-up data for every individual test at the time that the test needs to be run. This is rarely the case. Many times we need to use the output data of a prior test (i.e., creating an order) as the input for a later test (i.e., fulfilling an order). We call this Systematic Testing and consider it the ultimate balancing act; it offers risks as well as rewards as we’re relying on prior tests to function properly and generate the correct data in order for us to execute the later tests. Again, this is where the experience of IQ’s Test Strategy Specialists is key.
Another challenging aspect of E2E is environment management and configuration management. Environments and code can’t be changing during test execution, but in a multi-system E2E test effort where there might be 10 systems managed by disparate IT divisions, with each application following its own individual development schedule, understanding this is of utmost importance. Freezing 10 applications for 4 weeks for an End-to-End test effort is a highly unlikely scenario. Our Test Specialists understand this all too well and have the experience to plan and manage this while still conducting thorough and effective testing.
While IQ is working on the Test Strategy and Design of the test data library, the test case suites, and the environment and configuration management approach during E2E, we’ll work with you to prioritize tests based on risks and requirements, time and available resources. We can manage your existing testers, whether on or offshore, to design and execute the E2E plan effectively. Or, we can help you augment your project team temporarily with skilled testers to effectively execute these tests, all the while managing the testing process.
The Details – Test Strategy & Design: Data Conversion & Migration Testing
Data Conversion & Migration Testing is a very special niche area of testing and requires an understanding of how to test data and data combinations, without having a GUI screen to view.
This specialty test effort is required when a legacy system is being replaced by a newer system and when all of the historical data and the active data records need to be migrated. Data conversion needs to take into account both historical data—data that is no longer active but may be used in cumulative data reporting (e.g., budget forecasting data from the prior year) and active data. The challenge in Data Conversion and Migration Testing is figuring out how an active record can be migrated to the new system (and possibly a new workflow) and ensuring that the data remains usable. Converting data from one database to another is not a simple one-to-one mapping. Sometimes select data needs to be prevented from migrating into the new system. You can flip that switch at go-live and your new system may function perfectly. But, if your new system has incorrect data, missing data, or duplicated data, it will be rendered useless by your users.
IQ’s Test Strategy Specialists can plan and design this effort for you. We can manage your existing testers (or developers), whether on or offshore, to execute the data conversion test plan effectively. Or we can help you augment your project team temporarily with skilled testers to effectively execute these tests, all the while managing the test effort.
The Details – Test Strategy & Design: Smoke & Regression Suites
What’s the big deal about designing a Smoke Test Suite? Isn’t it just a few simple tests to see if the system is stable and ready to perform our real test effort?
Well, yes. But, let’s review the previous statement. When designing Smoke and Regression Test Suites we need to ask ourselves how are the “few tests” selected from an entire suite—why some and not others?
We want to ask ourselves:
- What is the defining criteria for including a test case to confirm stability? Define “stable.”
- What is the definition of the system being unstable and not ready for full testing? What about “kinda-sorta” stable?
- Where is the exact line in the sand for accepting a system for testing, which is what a Smoke Test is supposed to determine?
Having a viable Smoke Test is especially important if you are working with a software vendor responsible for customizing its application for your organization.
IQ’s Test Strategy Specialists can help you determine your acceptance criteria and then help select tests from your existing test suites or help to design these for you. We’ll build a Smoke Test Suite, consisting of three subsets of test cases—a Smoke Test for an entire new version of the application (major upgrades), a Smoke Test for individual modules of your application (smaller scale patch releases), and a Smoke Test for an emergency patch.
We apply the same approach to designing the Regression Suite, keeping in mind that the Regression Suite is more comprehensive than the Smoke Test Suite and typically requires the creation of a larger scale test data library. Our approach, however, remains the same—we’ll select three subsets of test cases—a full system Regression Suite, a smaller module-based Regression Suite, and an emergency patch Regression Suite.
IQ can manage your existing testers (or developers), whether on or offshore, to execute the Smoke & Regression plans effectively, or we can help you augment your project team temporarily with skilled testers to efficiently execute these tests, all the while managing the test effort.
The Details – Test Strategy & Design: Automated Functional Testing
Automated testing. Sounds great, right? Click a button, run a bunch of tests, and then a report spits out and tells you what is and is not working properly in your system. If that sounds like magic, you are absolutely correct!
In the testing world, that is not how automated testing works. In order to have automated tests, a developer needs to develop the code that takes a series of steps, inputs, and outputs (known as a manual test case) and to convert that into a testing development language. These team members typically aren’t testers but coders who specialize in a niche programming language used by automated test software tools. Therefore, just like in any development effort, requirements and design documents are needed to hand over to the developer for him/her then to code.
The important thing to remember about automation is that it’s a continuous parallel development effort; as the application under test undergoes enhancements and modifications, so should your automated test suite to adjust for the new changes. This is often the cause of failure for most automation efforts—the lack of the ongoing, continuous, parallel development efforts simply because the cost of maintaining the automation suite is expensive and time-consuming.
The reality is that automation can be advantageous if there is an effective strategy in place that addresses areas of functional stability, excludes volatile functionality and modules, and takes into consideration the parallel development effort. IQ’s experienced Test Strategists can help you with all of these facets.
IQ’s Test Strategy Specialists can manage your existing testers (or developers), whether on or offshore, to develop and execute the automated test plan effectively. Or we can help you augment your project team temporarily with skilled testers to effectively execute these tests, all the while managing the test effort.
If you’re thinking that specialized testing seems complicated, that’s because it is. But with IQ helping you along the way, the process becomes a whole lot simpler.