Performance Testing

For this week’s blog, I decided to research a bit on performance testing because it is listed as a topic for a class. While doing research, I found a blog called “How To Do Performance Testing: Tips And Best Practices” by Volodymyr Klymenko. In this blog, he discusses the primary purpose of this kind of software testing, types of performance testing, its role in software development, common problems revealed by performance testing, tools, steps, best practices, and common mistakes to avoid. The blog was well organized and clearly simplified each topic.

The purpose of performance testing is to find potential performance bottle necks to identify possible errors and failures. This is to ensure that the software meets the performance requirements before it is released. Some types are load testing(response under many simultaneous requests), stress testing ( response under extreme load conditions or resource constraints), spike testing (response under a significant and rapid increase in workload that exceeds normal expectations), soak testing(simulates a gradual increase in end users over time to test the system’s long-term stability), scalability testing(how the system scales when the volume of data or users increases), capacity testing (which tests the traffic load based on the number of users but differs in scope) and volume testing(which tests the response to processing a large amount of data).

The next section discusses the role of performance testing. I found some of the bullet points to be a bit obvious like “improving the system’s overall functioning”, “monitoring stability and performance”, and “assessing system scalability”. The other points mentioned were “failure recovery testing”, ”architectural impact assessment”, “resource usage assessment” and “code monitoring and profiling”.

After discussing the role of performance testing, the author discussed the common issues revealed by performance testing. The problems discussed were speed problems, poor scalability, software configuration problems, and insufficient hardware resources. This section is broken down into many bullet points to explain each issue. I found it to be very beneficial in simplifying the ideas presented.

The last three sections are the steps for performance testing, best practices, and common mistakes to avoid. The steps presented in this article for performance testing were: 1.Defining the Test Environment, 2.Determination of Performance Indicators , 3. Planning and Designing Performance Tests , 4. Setting up the Test Environment, 5. Development of Test Scenarios, 6. Conducting Performance Tests , and 7. Analyze, Report and Retest. The section after explaining the steps includes some best practices like “Early Testing And Regular Inspections “ and “Testing Of Individual Blocks Or Modules”. Some of the common mistakes to avoid that I found to be noteworthy were “Absence of Quality Control System” and “Lack of a Troubleshooting Plan”. While working on the Hack.Diversity Tech.Dive project, I had to do some performance testing and even used a tool that was mentioned in the article called Postman. In the future, I will definitely be more prepared to do performance testing now that I can reference this article.

Pairwise Testing

For this week’s blog, I decided to do more research about pairwise testing. During my search for more information I found the blog “Pairwise Testing | What It Is, When & How to Perform?” by Kiruthika Devaraj. In this blog, Devaraj breaks down the process of pairwise testing in simple steps, along with the advantages and disadvantages. She first defines pairwise testing as a “permutations and combinations based software testing technique that tests each pair of input parameters to verify that the system functions correctly for all possible discrete combinations”. It uses a minimum set of test cases to test all possible discrete combinations. The purpose of this kind of testing is to test all possible discrete combinations of inputs without the need to test every single combination. This reduces the time, cost, effort, and number of cases that need to be written. It also allows the tester to identify errors that may occur when multiple different parameters are combined which in turn increases the overall quality of the software.

Devaraj then explains the disadvantages and challenges of this testing technique. One of the disadvantages that is explained is that it can be time-consuming especially if there are a large number of test cases. Another case is where assumptions can be false because pairwise testing relies on all pairs of parameters interacting to determine the results. She explains the challenges in bullet points to make it easier for the reader to comprehend. Some of the bullets include challenges in finding meaningful inputs and the correct set of inputs. If you don’t test carefully then the test coverage may not be enough. One also needs to determine how parameters interact with each other. 

The last few paragraphs of the blog describe using pairwise testing to speed up cross-browser testing. In this example, she explains the parameters that one would test (browser, operating system, device type, and screen resolution) and how to follow through with a real-world example. The author also includes a list of testing tools to help with automation for this type of testing.

The last section of this blog is the answers to frequently asked questions like “Is pairwise testing completely automatic?” and “How effective is pairwise testing at finding bugs?”. It helped clarify some aspects of the testing technique that weren’t explicitly answered earlier in the blog. 

Overall I think this blog was helpful in understanding pairwise testing especially because the author included a lot of examples. It gave me a better understanding of a testing technique that I may use in the future when I have to test a complex system that has multiple input parameters and multiple possible values for each parameter. 

Writing Testable Code

For this week’s blog I decided to dive a bit deeper into writing testable code because I think it is a valuable skill. While searching for blogs that discuss this topic I found “TDD: Writing Testable Code” by Eric Elliott. In this blog, Elliott explains the importance of writing testable code barriers to writing testable code, and different tips to keep in mind to write testable code. 

The first thing he touches on is how a good quality process is essential to continuous delivery. One of the most important principles, he mentions, is the separation of concerns because “simplifies complexity and enhances the overall quality of your work”.

In the overview about testable code, he explains that achieving testable “allows for more efficient and effective identification of defects, ensuring higher quality and reliability”. The key characteristics of testable code are modularity (when code is organized into discrete units), clarity (code that is easily comprehensible), and independence (when code can be tested in isolation).One of the barriers of writing testable code that Elliott describes is tight coupling in code. The coupling in code is the instance where changing one part of the code can heavily impact or even break the functionality of another part in the code. If one part is changed there can be a chain reaction of bugs in various other parts of the code. Right coupling can be caused by parent class dependencies, shared mutable states, concrete class dependencies, event chains, state shape dependencies, control dependencies or temporal coupling. He explains that to write testable code you must reduce the forms of tight coupling mentioned because it makes testing easier and promotes a “more sustainable and scalable codebase”. This is why testable code and maintainable code are the same thing. 

Later he explains the test first vs test after approaches. The test first approach is the test driven development approach where tests are written before the code. The benefits include better developer experience and clearer requirements and design. The test after approach is where the tests are written after the implementation which leads to a probability of biased tests and reduced code coverage.

One of the last sections provided is about the separation of concerns which a  design principle for programs into distinct sections that  addresses separate concerns”. This principle is important because it allows independent testing and allows developers to isolate problems more effectively in addition to allowing the ability to make changes without unintended consequences. He describes that he likes to isolate into 3 distinct sections business and state logic (the core logic of the application), user interface, and I/O and effects (the part of the application that interacts with the database).

I think this source was well organized and easy to follow. It made the text easy to understand and provided great advice for writing testable code. I think the last section about how he likes to isolate the sections was especially helpful and I will keep it in mind for the future.

Test-Driven Development

For this week’s blog, I decided to find a blog that discusses test-driven development (TDD). I found a blog called “Test Driven Development is the best thing that has happened to software design” by Arek Torczuk. In this blog, the author walks through how they apply key elements through real examples. Throughout the blog, they explain their reasons for implementing TDD and walk through how to use TDD on code that is impossible to test. They split their explanation of how to test impossible code into small sections that include bits of example code to aid in the explanation. I found the organization of the blog along with the various examples to be particularly helpful in aiding the reader understand the topic and stay engaged.

The author first explains that TDD is implemented in a fail, pass, refactor cycle approach. They shorten the idea by explaining that “our focus remains on the ‘what’, and our knowledge of ‘how’ is postponed.” In this section, the author also provides a visual which I find to be effective in blatantly pointing out the important details of this section. They then go on to explain the differences between when tests are driven by code and when code is driven by tests. When tests are driven by code although the tests written for the code may pass it may not be the right approach as feedback about implementation and design may be lacking. There are 2 questions that one has to ask themselves when they take this approach: “When should I stop adding new tests?” And “Am I sure the implementation is finished?”. The latter approach (when code is driven by tests) forces you to ask yourself “What do I expect from the code?”. When code is driven by tests, you know when to stop. TDD provides fast feedback.

The various examples of code that are impossible to test were interesting to read through. One example I found fascinating was the “Bloated setup” and the “Mocking Hell” scenarios. The first scenario is where an object either has too many dependencies or the Single Responsibility principle has been broken. The author explained that in this case you should you should rethink your design. The latter is where a mock returns a mock that returns a mock.

Overall, I think this was an interesting read. I think reading through the various examples of how to test impossible code will be useful for the future as I will have to think through how to test different kinds of code and when I should rethink certain designs.

Unit Testing

This week I decided to discuss Unit testing because although we have finished a few activities on the subject, I would like to read up on it to get a better understanding of it. While searching for a blog to discuss, I found this blog called “Unit Testing Best Practices: 9 to Ensure You Do It Right” from Testim. This blog also discusses what a unit test is, why we write them, the benefits of unit testing, how to achieve testable code, who creates them, and the difference between unit testing and integration testing.

The text begins by explaining that unit tests focus on very small parts of the application in complete isolation and compare their actual behavior with the expected behavior. In this section, the idea of complete isolation is thoroughly explained with the idea that “you don’t typically connect your application with external dependencies”. This is what makes unit tests so fast and stable.

The next section explains why they are created in bullet points and have the main ideas bolded. This structuring is helpful to simplify the ideas for the reader and make sure that the main ideas stick out for them. These were the bolded parts of the bullet points in the section: unit tests help you find and fix bugs earlier, your suite of unit tests becomes a safety net for developers, unit tests can contribute to higher code quality, unit tests might contribute to better application architecture, unit tests can act as documentation, and detect code smells in your codebase. I think a lot of the ideas presented were easy to comprehend based on the bolded part but I had to read more about the last bullet point because the term” code smells”(signs that something is wrong with your code ) was not obvious to me. 

The section that explained the difference between integration testing and unit testing was fairly short because they narrowed the main idea down to a few sentences: “It’s all about the level of isolation. While unit tests have to be completely isolated, integration tests don’t shy away from using external dependencies….integration tests offer a more high-level view of the application than unit tests do. Because of that, the feedback they provide is both more realistic and less focused.”

The 9 best practices at the end are titled: Tests Should Be Fast, Tests Should Be Simple, Test Shouldn’t Duplicate Implementation Logic, Tests Should Be Readable, Tests Should Be Deterministic, Make Sure They’re Part of the Build Process, Distinguish Between The Many Types of Test Doubles and Use Them Appropriately, Adopt a Sound Naming Convention for Your Tests, and Don’t Couple Your Tests With Implementation Details. These tips are easy to understand based off of the title alone but throughout each section, the author uses bits of code as examples and bullet points to thoroughly explain each idea. In the future, I will refer to these tips when I need to write tests for a project I will work on.

Static vs. Dynamic Testing

This week I decided to touch on static and dynamic testing. I wanted to find a blog that talks about different techniques for each type of testing. While searching I found the blog Static Testing VS Dynamic Testing – Key Differences testing by Kiruthika Devaraj. It first begins by generally defining the two types of testing. Devaraj first defines static testing as involving “testing the software without running it” and dynamic testing as involving “executing the program and testing its behavior in different scenarios”. They then go on to explicitly outline the differences in the next section. 

In the section titled “Differences Between Static Testing and Dynamic Testing”, they explain that static testing involves reviewing the documentation, design or code to check for defects and errors without executing the software. They also mention that this type of testing aims to identify issues early in software development when they are easier and less expensive to fix. They explained that dynamic testing involves executing the software and testing its behavior in different scenarios. They explain that during this process testers create test cases and run tests to identify defects and ensure the software meets the specified requirements. By testing the program against a variety of inputs, expected outputs, and error scenarios, this kind of testing essentially tries to evaluate the product’s functionality, performance, and security and ensure that it operates as predicted. In this section, the author uses bullet points and spacing to organize the information and make it easier to digest. 

The next section is a table that clearly outlines the differences in features like definitions, objectives, types of testing, timing, and results for static and dynamic testing. Static testing involves code review, walk-through, and inspection while the other involves unit, integration, system, acceptance, performance, security, and user acceptance testing. The author then explains how to choose between the two types of testing. They mention that it comes down to the specific requirements, objectives of the testing process, and desired outcomes. I enjoyed this article because the author got straight to the point when writing the article while clearly defining the use of each type of testing. I think the way they went about writing this informational piece was effective in keeping the reader’s attention by outlining the information in the simplest way possible. The use of bullet points and tables helped me to remember the information better. I am likely going to have to test my future personal projects so knowing the differences between the two types of testing and how to choose it was important to read about.

Black-Box, White-Box, and Grey-Box Testing

This week I decided to find a blog that discusses the differences between Black-Box, White-Box, and Grey-Box Testing because even I didn’t grasp it during the class activity. The article “Difference Between Black-Box, White-Box, and Grey-Box Testing” by TestFort Expert explains each way of testing in depth by using bullet points to outline the techniques of each type of testing and reviews the pros and cons of each.

The blog summarizes that the black-box testing method involves testing the software without knowledge of the internal structure and source code. You use this method to test the interface against the specifications that the client gives. The techniques that are involved are decision table testing, error guessing, all-pairs testing, and equivalence partitioning. The first technique tests with if-then-else and switch-case statements to find errors related to conditions. The second technique describes testing based on intuition, the third tests combinations of each pair of input parameters to find bugs from interacting parameters and the last technique involves splitting up parts to reduce testing time. The method tests for functionality but can only be applied to small segments of code.

White-box testing tests the internal structure of software and the logic behind it. One needs full knowledge of the code and the software for this method. It uses these techniques: control flow testing, data flow testing, and branch testing. The first technique logic of the code by executing input values and comparing for expected results. The second detects improper use of data values and data flow by coding errors. The last technique focuses on validating branches in the code. The blog mentioned that technical debt is reduced  by maintaining the code quality which is something I dint think about. Another thing I didn’t think about is that it can result in false positives because test results are strictly tied to the way the code was written. 

Grey-box testing is a combination of the previous methods. It tests for the interface, functionality, and internal structure. It requires some knowledge of the source code but takes more of a straightforward approach. The technique of this approach is matrix testing. regression testing and pattern testing. The first method involves tracing user requirements to identify missing functionality. The second technique involves testing the software after modifications, and the last technique involves analyzing the design and architecture of the software to find the root cause of a defect. This method isn’t suitable for algorithm testing. 

I think this blog is a good resource to learn about each testing method in depth. I think the lists of techniques were helpful in understanding what each method involves. I left class without truly understanding each method. I think the pros for the methods were straightforward but I didn’t think about the cons so I found that section helpful. Now that I know the techniques of each method I am better equipped to think about how to test software.

CS 443 Introduction blog

This semester I will be writing blogs as I learn more about software quality assurance and testing. Hopefully this will be a fun journey.