This blog explores the benefits of utilizing automation to validate migration, introduces our Automated Testing solution, and answers some frequently asked questions (FAQs) about our process.
A detailed validation of your migration project is extremely important, which makes testing an integral part of any modernization effort. This is because although key business logic is retained through modernization efforts, the underlying applications have to be adapted to work within the new target operating environment.
If you’re looking for the most comprehensive insight into your applications possible, then the automated testing route is the way to go as opposed to the more burdensome manual testing route – especially if you’re running large, business-critical applications with long life spans. But that’s not to say that the human element of the test and deploy phase is removed entirely with automated testing – rather, that the automation tool allows human testers to be more productive and increase their scalability, while taking care of time-consuming and / or repetitive tests. This reduces the potential for miscommunication, removes the risks associated with common human oversight and expedites the more complex elements, thus making the process much more time and cost effective.
It’s worth noting that the most successful modernization projects can also incorporate manual testing in specific areas where automation is not possible. The key is the more testing that’s done, the more likely the project is to be successful, and a preliminary test strategy workshop will identify testing needs, scope solutions, and resources for each phase.
There are two main aspects we consider in our own Automated Testing solution:
- Online testing: Your organization’s subject matter experts connect to the test environment using a terminal-based 3270 emulator (such as IBM Personal Communications) and turn on tracing to allow our Automated Testing solution to record sessions. These sessions consist of the execution of specific test case scenarios (for example logging in, selecting menu items, etc.) – basically, normal day-to-day actions that are essential mainframe use cases. Then, the recordings are encrypted and transferred into our Modernization Platform-as-a-Service (ModPaaS) engine, where the 3270 trace files are replayed and processed. Test cases for the target environment are automatically generated from these recordings, injected into to the Continuous Integration / Continuous Deployment (CI / CD) pipeline, compared using our Automated Testing toolset, and finally presented by way of our centralized monitoring dashboard.
- Batch testing: From the mainframe test environment, batch-related assets, including source code and job scheduler definitions, are loaded and parsed into ModPaaS which is integrated into the CI / CD pipeline. The Automated Testing engine analyzes batch assets and determines batch streams. It then uses eavJES to generate batch test assets (test scripts, test cases, etc.), which are executed against the mainframe (source) environment and the target (Cloud or data center) environment. Finally, the test results are compared using our Automated Testing toolset, which generates reports and insights for review in the centralized dashboard.
It’s important to keep in mind that testing shouldn’t be considered independently from the rest of the modernization process. Let’s break it down. In a typical modernization project, around 40 per cent of the effort may be focused on application source code and data conversion, with 10 per cent spent on designing, implementing and managing the target operations and hardware infrastructure. The other 50 per cent would account for testing. These proportions can differ hugely between projects, and sometimes testing may require less effort – but more often than not, it can amount to much more – sometimes even 70-80 per cent of the process can be spent on testing, as was the case with our recent modernization project with The New York Times. But this was not because of failure – it’s the result of comprehensive testing performed throughout every stage of the project timeline. In fact, the media giant ended up cutting IT costs by 70 per cent as a result of its modernization, a testament to this incredibly-thorough process!
If you have any questions about our Automated Testing solution, feel free to contact us here. We’ve answered a few questions we’ve received about the process below:
How do you deal with test data, e.g. on the online tests?
Using an identical set of test data on both the system under test (i.e. the mainframe) and target system is one of the fundamentals of performing automated online testing. Test data is converted to the target data state using the data migration process appropriate for the project. A best practice might involve creating a test script to initialize test data prior to test playback, a practice that can prove beneficial when the test script is modifying data as part of the overall test case.
When batch test cases are created using eavJES and ModPaaS, is there anything you particularly look for? Do you try to modularize the breadth of tests?
It depends, there will be requirements for different types of batch test cases to prove different things, which all need to be repeatable. There will be preliminary tests which are small tests with only a few steps, and then there are longer test cases which might be creating end of quarter reports. There is no fixed set of requirements, but the main thing is that they need to be repeatable.
How would test transactions on a non-IBM system be captured for subsequent re-playing against a converted system?
Online VME session capture is supported. We're constantly expanding our capabilities, both from a testing and integrations perspective, and much of that expansion is part of customer demand. Contact us if you have an interesting use case and we can discuss a plan!
