About this Course
This course introduces state-of-the-art techniques for automated analysis. Automated analysis encompasses both approaches to automatically generate a very large number of tests to check whether programs meet requirements, and also means by which it is possible to *prove* that software meets requirements and that it is free from certain commonly-occurring defects, such as divide-by-zero, overflow/underflow, deadlock, race-condition freedom, buffer/array overflow, uncaught exceptions, and several other commonly-occurring bugs that can lead to program failures or security problems. The learner will become familiar with the fundamental theory and applications of such approaches, and apply a variety of automated analysis techniques on example programs.
After completing this course, a learner will be able to: – Understand the foundations of automated verification: randomization and symbolic representations – Distinguish the strengths and weaknesses of random testing, symbolic analysis, static analysis, and model checking – Use a variety of state-of-the-art static analysis and automated testing tools for automated verification – Create executable requirements as an oracle suitable for automated testing and symbolic analysis – Understand how the choice of oracle affects fault-finding for automated analysis strategies. – Use automated testing to achieve full mutation coverage – Create a test plan that utilizes both manually-written tests and automated tests towards maximizing rigor, minimizing effort and time, and minimizing test costs. This course is intended for learners interested in understanding the principles of automation and the application of tools for analysis and testing of software This knowledge would benefit several typical roles: Software Engineer, Software Engineer in Test, Test Automation Engineer, DevOps Engineer, Software Developer, Programmer, Computer Enthusiast. We expect that you have some familiarity with the Software development Life-Cycle, an understanding of the fundamentals of software testing, similar to what is covered in the Introduction to Software Testing and Black-box and White-Box Testing Courses. Familiarity with an object-oriented language such as Java or ability to pick-up Java syntax quickly to write and modify code, and willingness to use tools and IDEs are assumed.
SKILLS YOU WILL GAIN
- Software Testing
- Formal Verification
- Test Automation
Syllabus – What you will learn from this course
WEEK1
6 hours to complete
Introduction to Automated Analysis
In this module we will learn about a range of techniques for analysis of programs and methods to automate testing. Along the way we will learn how to specify properties of interest to check about a program and capture assumptions about the environment for effective testing. To reinforce some of the important concepts learned we will practice automated testing using effective tools on a concrete example.
8 videos
- Introduction to Automated Analysis13m
- Automated Analysis Techniques9m
- Symbolic Representations12m
- Property Specification8m
- Environmental Specification and Assumptions6m
- Parameterized Unit Testing using junit-quickcheck12m
- Environmental Specifications in junit-quickcheck2m
- (Optional) Installation of Eclipse and Gradle2m
1 reading
Overview and Syllabus10m
5 practice exercises
- Introduction to Automated Analysis30m
- Automated Analysis Techniques30m
- Symbolic Representations30m
- Property Specification30m
- Environmental Specifications and Assumptions
WEEK2
6 hours to complete
Automated Test Generation
The focus of this module is to understand how various techniques can help us automate the generation of useful and numerous tests. We will examine ways to specify and use properties of the system and the environment to guide the generation of test data.
- Overview of Automated Test Generation6m
- Automated Test Generation using Random Testing14m
- Automated Test Generation using Symbolic Execution15m
- Automated Test Generation using Metaheuristic Search16m
- Property-Based Testing for Real-Time Systems6m
- Biasing Input Values in junit-quickcheck6m
- Using Generators to Create Complex Inputs in junit-quickcheck9m
- Explanation of Test Harness and Assignment for Microwave Example8m
4 practice exercises
- Overview of Automated Test Generation7m
- Automated Test Generation using Random Testing30m
- Automated Test Generation Using Metaheuristic Search12m
- Property-Based Testing for Real-Time Systems30m
WEEK3
Static Analysis
5 hours to complete
The goal of this module is to introduce the learner to the principles of statically analyzing programs, understand how analysis techniques work by looking at some example analyses, and some good practices to follow when designing programs to enable the tools to help us detect and avoid defects. The learner will gain an understanding of using static analysis tools by looking at one concrete tool.
5 videos
- What is Static Analysis14m
- Dataflow Analysis16m
- Program Wellformedness Properties12m
- Designing programs for analyzability9m
- Static Analysis with Infer11m
1 reading
Analysis Exercise with Infer2h
3 practice exercises
WEEK4
3 hours to complete
Effective Automated Verification
This module will examine how to use effective automation techniques for a variety of purposes such as performing effective regression testing, discovering security vulnerabilities and monitoring software at run-time for critical properties.
6 videos
- Automating Regression Testing10m
- Automating Security Testing Using Fuzz Testing14m
- Runtime Monitoring7m
- Where Automation Fails9m
- Using Multiple Methods Effectively9m
- The Evolution of Software Testing10m
2 readings
- Fuzz Testing with AFL10m
- Runtime Monitoring Tools10m
4 practice exercises
- Automating Regression Testing30m
- Automating Security Testing Using Fuzz Testing30m
- Runtime Monitoring30m
- Summative Review of Week 4 material