What is Test Automation?
In software testing, there are mainly two types of testing: manual testing and automation testing. Manual testing requires physical time and effort to ensure the software code does everything it is supposed to do. In addition, manual testers must record their results. This includes checking log files, external services, and database errors. If you are familiar with manual testing, you know that this process can be very time-consuming and repetitive.
Tests that are performed manually versus those that are conducted automatically are fundamentally different. During manual testing, the tester is personally responsible for testing the software’s functionality in the way the user would do it. Tests can be automated using an automation tool so that more time can be spent on more valuable tasks like exploratory testing and regression testing. In spite of the time it takes to maintain test scripts, you will be able to improve test coverage and scalability as well.
An advantage of manual testing is that it enables a human to draw insights from a test that could otherwise be overlooked by an automated test. In addition, automated testing is ideal for large projects, projects that require testing the same areas repeatedly, and projects that have already undergone manual testing.
How to choose which Tests to Automate
If you want to be successful with test automation, you need to have the plan to help you get the most out of your automated tests. Since not all tests can be automated, choosing the right tests for automation at an early stage is an important step in creating an automation plan. You don’t have to start from scratch when deciding which tests to automate. Best practices for automated testing are defined, including the selection of tests to be automated. To get you started, here’s a general checklist of tests where automation can best streamline your process. You should watch out for:
- repetitive tests that run multiple builds
- tests that are prone to human error
- tests that require multiple datasets
- commonly used functions that cause high-risk conditions for tests that cannot be run.
- manual Tests that run on multiple hardware or software platforms and configurations
- tests that require a lot of effort and time to test manually
Common Challenges of Test Automation
With the increased awareness of Software testing, there is also an increase in test automation. And every time it’s discussed how when to start the automation, and which tool will be best for the team.
But with pros test automation comes with cons as well. It is defined by ROI, success ratio, etc.
Team Collaboration Issues
With the growing test automation team, it’s been observed that team collaboration issue raises, and team productivity is reduced. Following are some of the ways to reduce collaboration issues:
- Keeping the team aware of what changes will be done
- Following coding standards
- Following the code commit process (Pull Request process)
- The new team member is given a framework walk-through and made aware of different process
- Creating a common knowledge-sharing repository or channel for communication.
Network Issues
It is observed that the network is many a time not stable during the time of scripting or while the test is executed, due to this many issues occur such as false failure, execution disruption, delay in providing sign-off, and losing access to a remote machine or virtual machine.
Cost Factor
At the initial phase of automation, the cost is high and the return is less and is often seen that the resource required at the initial stage is very high and cost plays an important factor. Sometimes tool cost is high and even the test bed preparation is costly. At times it’s seen that when processes are not followed the maintenance cost of automation is very high.
Right Expectations
It’s important to understand that expectations should always be right, whether you’re selecting testing tools, implementing methods, or anticipating results. Despite the amount of automation we develop, human interaction and involvement will play a very important role in resolving bugs and running tests.
Testing Accuracy
You will get false outputs if you try to run processes with outdated data, which is one of the most common errors in automation. In order to avoid such chaos, it is essential for a team to be aligned in one channel that can facilitate clear communication. DevOps needs testers to respond quickly to automation problems in order to improve data accuracy. To achieve testing accuracy, data relevancy and accuracy are the only factors to consider. To enhance productivity, QAs must also develop a robust analytical solution.
Selection of Tool
Choosing the right tool is the most important part of the testing phase. Lack of expertise can hamper the entire development cycle. Typically, typical testing consists of regression, functional, unit testing, integration, etc. In addition, the challenge is deciding whether your project needs automation or manual testing. Remember to consider the following aspects before choosing a tool for your project:
- Gather information about your project and its requirements.
- Technical Support and Support.
- Check cross-browser testing compatibility.
- Ease of maintenance
- Cost
What should be Automation Build Process
Following are some of the recommended actions, which may vary based on how the CI/CD tool is set up and how build deployment is done on your project.
- The Automation build runs every night after the Nightly CI Job has been completed and deployed latest build to QA boxes
- The Automation build runs all the GUI / Automation tests against the QA website.
- Next Morning, the QA team will be responsible to go through the automation results
- if all the tests have passed, then the QA team will not be sending out any emails and nightly regression will be considered a pass.
- if any of the tests fail then the QA team will be responsible for following the steps under ‘What should be QA Team Responsibility if Regression Fails‘
What should be QA Team Responsibility if Regression Fails
Following are some of the recommended actions, which may vary based on how the team is structured and how automation is done within the sprint.
- The team will analyze the tests that have failed and determine the reason for the test failure. The test failure can be broadly classified into 3 categories
- Test failed due to a valid bug:
- The QA team create a bug in JIRA for the development team in the current sprint.
- The development team will prioritize these bugs before any feature developments.
- The QA will label these bugs with the “AutomationBug” label.
- Test failed due to screen/functionality/locator being updated:
- The QA team will temporarily Ignore this test from the nightly build.
- QA team will prioritize updating the test before picking up any new tests.
- QA team will re-enable the test after the tests have been updated.
- The test failed due to a bug in the test:
- The QA team will temporarily Ignore this test from the nightly build.
- QA team will prioritize updating the test before picking up any new tests.
- QA team will re-enable the test after the tests have been updated.
- Test failed due to a valid bug:
- QA team will send out an email to the group with the following details:
- Test Results HTML (attached)
- List of bugs that resulted from nightly regression
What Do You Think?
Did this work for you?
Could I have done something better?
Have I missed something?
Please share your thoughts using the Contact Us form. Also, let me know if there are particular things that you would enjoy reading further.
Cheers!