By Greg Paskal, CTFL
Necessity and Invention
As the moving truck pulled away from our newly rented house, I pondered this new adventure we had begun. I was happy our family was back in Southern California, and this job with Kinko’s Corporate Offices would introduce me to an engineering field I’d never worked in before—the world of Quality Assurance.
As new jobs go, I had much to learn. There were team members to get to know and various projects to learn. Each of these helped our company meet our niche in the marketplace. I was assigned to assist a web developer named Adam Fisher who created each webpage that made up Kinkos.com. Soon, Adam and I were working like a well-oiled machine. He would create a few webpages, and I would test them. The rhythm of the day seemed predictable and well balanced. I took time between webpage changes to learn more about software testing and improve my HTML and JavaScript skills.
Some time progressed, and I thought I had this QA thing figured out pretty well. Then an email came, followed shortly by a phone call: “Greg, I need you to test this change. It needs to go live in an hour.” I thought, Wait a second. That’s not the way this works. I get a day or two to test things, and then I let you know when everything is good to go. What’s with the fire drill? As it turned out, the changes were important, and I was able to get everything tested quickly enough to meet our deadline.
When the dust settled, I realized this would likely not be the last time this scenario would arise. So I began considering how to better prepare for the next emergency change. My efforts resulted in a simple list of items to test when time was scarce. Before long, my list grew to fill an entire page, and I realized a different format for this information was necessary.
METS Is Born
Translating my scribbled notes into a spreadsheet, I realized some things captured were not as important as other things. So I added a second column for these less critical tests. In doing so, I realized not all tests have the same importance given the urgency of the moment. I decided to call my testing strategy the Minimal Essential Testing Strategy, or METS for short.
METS continued to evolve over the next couple of years, including added columns to capture categories and importance levels identified as Critical, High, Medium and Low. METS was maturing, and I had an excellent tool to capture more tests as I identified them along the way.
METS as Useful as Ever
METS began in the early days of the web, but it continues to keep up with mobile and other technologies to this day. QA engineers can really take advantage of a tool like METS. It enables smart testing with quick turnarounds of work requests and allows QA Engineers to adjust their testing efforts midflight as testing is happening. METS can also complement other testing strategies and provide options within a testing organization, helping them to adapt to real-world testing challenges.
Getting Started With METS
Let’s talk about what it takes to get started with METS for your testing needs. Imagine you’re going to build a METS testing strategy to test a minivan. Let’s begin by viewing this minivan from two different perspectives: Physical and Functional.
Physically, we observe Tires, Glass and Lights.
Functionally, we expect Acceleration, Lighting, Stopping and Cooling.
All of these are important aspects of a minivan.
Let’s prioritize our tests to add them to the METS Physical Test Grid category related to Tires.
- It’s critical that the minivan has four tires.
- It’s of high importance that the tires are in good condition.
- It’s of medium importance that the tire pressure is correct.
- It’s of low importance that the tires are balanced.
This is what the information would look like placed in the METS – Physical Test Grid.
Example - METS Physical Test Grid for Tires category
Next, let’s prioritize tests we would add to the METS Functional Test Grid category related to Stopping.
- It’s critical that the minivan completely stops.
- It’s of high importance the minivan stops in the expected time frame.
- It’s of medium importance the minivan stops under wet conditions.
- It’s of low importance that the minivan stops using the emergency brake.
This is what the information would look like placed in the METS – Functional Test Grid.
Example - METS Functional Test Grid for Stopping
By considering the vehicle from both the Physical and Functional perspectives, we come away with different tests that are unique and important to the workings of a minivan. In the same way, when considering our application under test, we identify tests that ensure our product is both Physically and Functionally working as designed.
METS in Your World
Now that you have the basic METS concepts, let’s apply them to your application. Using the same approach as we did in the minivan example, write down Physical characteristics of your application. For example:
My Application – Physical Characteristics
Next, write down Functional characteristics of your application. For example:
My Application – Functional Characteristics
- Account Creation
- Adding Items to Cart
- Checkout
These two groups will become the categories of your METS Physical and Functional Test Grids.
Once these categories have been added to your METS Test Grids, capture Critical, High, Medium and Low tests that would align themselves with each category. Keep these simple. You’ll discover more as you use METS on a regular basis.
Note: Keep details for tests brief. METS should guide the QA tester who is likely a subject-matter expert of the application.
Below are examples of a Physical and a Functional Test Grid. Remember: To build these, you first capture the Categories, and then you identify appropriate Critical, High, Medium and Low tests.
Example – METS Physical Test Grid
Note: Remember, the difference between these two METS grids is the perspective of how you view the application under test. The Physical Test Grid contains the items you can touch within the application. The Functional Test Grid contains the items you experience within the application.
Example – METS Functional Test Grid
A Day in the Life With METS
Once the basic METS documents are created, have your team begin using them to guide their daily testing efforts. Testers should begin with the Critical tests and progress to less important tests as time allows.
Note: Like many QA engineers, when I first began testing I believed I needed to test everything and execute every single test. METS helps alleviate this problem by providing priority and order to the most critical items to test leading to less critical items. In a typical testing engagement, it’s unlikely every test in the METS Testing Grid will be executed.
Test team leaders should encourage their teams to add to their METS documents as they identify new tests. Many teams publish their METS documents to online or cloud-based tools, enabling centralized access, execution, searching and maintenance across the entire METS test suite.
Bring awareness to development teams and customers regarding your METS Testing resources and teach them how to utilize them. Recognize that each user of the METS process may identify tests previously not considered. Put into place a simple process to capture test ideas and concepts for new categories and groups of tests. METS Testing resources mature over time and will become an invaluable part of your test toolset.
Advanced METS and Benefits
After using METS for a number of years, I started to find other creative ways to get more out of these tools. I recognized that each test had unique metrics that could be captured.
For example, taking the METS Physical Test Grid and breaking each test into a single row provided the ability to capture details about Importance, Time Required and Potential Severity. These metrics can help bring greater awareness about where to focus testing efforts.
Example – METS Physical Test Metrics Grid
With some creativity within your spreadsheet application, you can use these and other added metrics to help estimate projected execution times and other valuable information.
METS and Manual Test Cases
Many QA teams begin with METS as their initial testing strategy. As teams mature and recognize the need for documenting detailed test cases, METS can bring guidance and order as to where to begin this important process. Move from critical to less-critical tests in your documentation efforts. The initial METS documents created along with manual test cases will provide a great structure to follow for new hires and contractors. This will help them to quickly ramp up and apply their testing efforts in the right places.
METS and Test Automation
With test automation becoming essential among QA teams, it’s critical to know what tests should be automated. Leveraging the insights from your METS document can provide direction to great test automation candidates. Focus your automation efforts on the critical column as a place to begin. By building out test automation to cover all your critical METS tests, you have an exceptional, automated first line of defense that can be executed against the application daily.
Tip: In many cases, automating just the critical test of your METS Testing Grid will be significant enough to identify a good number of defects. In my experience, running this automated test suite daily leads to finding many defects and also ensures the test environments are in good working order. An organization can save hundreds of valuable testing hours leveraging this technique alone.
Final Thoughts
METS continues to be an invaluable tool in our testing organization. I suspect it will continue to provide new advantages not yet considered, and I am certain it will add value to your testing toolbox as well.
For more information about METS, please visit http://www.metstesting.com.
 Greg Paskal is a natural innovator, pioneering new approaches across the Quality Assurance landscape. Greg enjoys mentoring others in highly sustainable, maintainable approaches to manual and automated testing. Author of Test Automation in the Real World and StickyMinds.com publications, Greg can also be heard on the TestTalks podcast with Joe Colantonio. Learn more about Greg at MissionWares.com, GregPaskal.com and RealWorldTestAutomation.com.
|
