| D. Kalinsky Associates |
| Home | Training Courses | Online Learning | Resources | About Us | Contact | Site Map |
Masterclass:
"Embedded Software Testing and Debugging"
* An Advanced Course for Experienced Real-Time Embedded Software Developers, Testers and Quality Engineers
* How to Systematically Identify and Correct Defects in Real-Time Software Systems
* 3- or 4- Day Intensive Class (lectures, discussions, example debugging scenarios, classroom exercises)
"Embedded Software Testing and Debugging"
* An Advanced Course for Experienced Real-Time Embedded Software Developers, Testers and Quality Engineers
* How to Systematically Identify and Correct Defects in Real-Time Software Systems
* 3- or 4- Day Intensive Class (lectures, discussions, example debugging scenarios, classroom exercises)
COURSE OVERVIEW
This Masterclass examines the activities involved in testing and debugging errors, flaws, faults and failures in
real-time and embedded systems software.
Testing and debugging are two distinct activities with the shared goal of extracting defects from software and systems.
Testing uses systematic, engineering-style methods, while debugging uses investigative, research-style methods.
The class begins with a presentation of the main concepts and principles for systematic testing of embedded
systems software. Emphasis is placed on uniquely embedded issues such as flaws in interfacing, multitasking and
timing, rather than on general data processing issues.
The class continues with an examination of approaches important in embedded software testing. High-level testing
approaches are discussed for advanced stages of system development and integration, including security testing,
stress testing and independent verification and validation. Techniques and tools are presented to support these
approaches.
The debugging portion of the course starts with a survey of the wide range of categories of bugs from which
embedded and real-time software can suffer . It then focuses on a detailed discussion of the "top 5" kinds of bugs
that can be "project killers" or "product killers", including stack overflows, race conditions, deadlocks, timing problems,
and re-entrancy conditions . Emphasis is placed on multitasking and timing behaviors.
The class continues with an examination of debugging tools that are helpful in real-time software development.
'Liveness' issues including deadly embrace, lockout, memory starvation and CPU starvation are discussed in depth.
As are the particular sorts of bugs that tend to create problems in multi-tasking software when using real-time
operating systems. Emphasis then focuses on the unique sorts of bugs that plague Interrupt Service Routines
("ISRs"), such as interrupt overflow.
Participants are asked to do detailed exercises on many of the techniques presented, so that the concepts and
methods taught are reinforced and absorbed into the participant's arsenal of testing and debugging skills.
This course is not a general course about software testing and debugging, but rather it is highly focused on the
testing and debugging of embedded, time-constrained, resource-constrained multitasking software that may run
under the control of a modern real-time operating system ("RTOS").
WHO SHOULD ATTEND ?
This course is intended for practicing real-time and embedded systems software designers, software developers,
project managers, testers and quality engineers who have responsibility for planning, implementing, testing and
debugging the software for real-time and embedded computer systems.
Course participants are expected to have practical knowledge of coding for real-time and embedded systems, plus
some familiarity with at least one RTOS. [This knowledge can also be gained by attending one of our prerequisite
introductory courses "Introduction to Embedded Systems and Software" or "Introduction to Real-Time Operating
Systems".]
COURSE OBJECTIVES
The primary goal of this course is to give the participant the skills necessary to systematically identify and correct
defects in software for real-time computer systems developed with or without an RTOS. This is a very practical,
results-oriented course that will provide knowledge and skills that can be applied immediately.
CO-REQUISITE COURSE
Additional knowledge useful for attendees of this course can be obtained at the co-requisite advanced course
"Architectural Design of Real-Time Software". [For on-site presentation, both courses can be combined in a 4- or
5-day format.]
COURSE CONTENTS
Embedded Testing Fundamentals
8 Principles of Testing Embedded Software
Black Box Approaches: Equivalence Partitioning, Boundary Condition Testing
White Box Approaches: Stubs and Drivers
Code Coverage Levels
High Payoff White, Gray, and Black Box Tests
Unique Issues in Embedded Testing
Exercise: DO-178B "Modified Condition / Decision Coverage"
Real-Time Correctness: 'Liveness', Race Conditions
Embedded Testing Techniques
Input Data Testing
Exercise: Backus-Naur Form of a Protocol
State Testing: State Transition Diagrams and Harel StateCharts
Exercise: State Transitions of High-Availability Switchover
Pairwise Testing and Orthogonal Arrays
Exercise: Pairwise Test Design
Fault Injection Testing
Life Cycle for Embedded Software Testing
Top-Down vs. Bottom-Up Integration: Embedded Testbeds
System & Acceptance Testing
Security Testing
Stress Testing: Case Studies of NASA Spacecraft
Cause-Effect Graphing & Decision Tables
Exercise: Cause-Effect Test Design for a Temperature Display
Independent Verification and Validation
When to Stop Testing ?
Concepts for Debugging of Embedded / Real-Time Software
What is a Bug ?
The Debugging "Cookbook"
9 Golden Guidelines for Debugging
A Taxonomy of Real-Time / Embedded Bugs
Exercise: Bugs when Porting to New Hardware
David's "Top 5" Real-Time Bugs
Stack Overflows
Race Conditions
Deadlocks
Timing Problems
Re-entrancy Conditions
Real-Time Debugging Tools and their Limitations
Where Traditional Debugging Techniques are Insufficient
The Correctness Killers: Violations of Liveness
Deadlock: 4 Ingredients, and 8 'Fixes'
Task Lockouts and Starvations
The 3 Classes of Real-Time Software Debuggers
Cutting Edge Debuggers: Time-Reversal, SensorPoints
Logic Analyzers
Post-Mortem Debugging
Exercise: Bug in a Real-Time Controller
Mindsets for Debugging Real-Time Software
Cause and Effect: The Scientific Method
Assertions and Invariants
Memory Allocation/De-allocation Bugs
Proactive Debugging Techniques
Bugs in Multi-tasking and ISRs: The Mis-use of RTOS's
Task Decomposition and Communication Defects
Multi-tasking Debugging Techniques
Interrupt Service Routine Design Defects
RTOS Usage Defects
Exercise: Bug in Interrupt Handling
Programming Bugs in Real-Time Software
C-Language Ambiguities and Coding Disciplines
C-Language Code Analysis and Tools
Static vs. Dynamic Code Analysis
Exercise: C-Language Shenanigans
Metrics Predict Bugs
Exercise: Cyclomatic Complexity
Taxonomy of Embedded Testing Tools
Update on Static Code Analysis
The JPL "Power of 10" Coding Rules
Optional Sections:
Hard Real-Time Performance Bugs and Calculations
Bugs in Multi-Core and Symmetric Multi-Processing Software
Final Examination.
INSTRUCTOR: Dr. David Kalinsky
This Masterclass examines the activities involved in testing and debugging errors, flaws, faults and failures in
real-time and embedded systems software.
Testing and debugging are two distinct activities with the shared goal of extracting defects from software and systems.
Testing uses systematic, engineering-style methods, while debugging uses investigative, research-style methods.
The class begins with a presentation of the main concepts and principles for systematic testing of embedded
systems software. Emphasis is placed on uniquely embedded issues such as flaws in interfacing, multitasking and
timing, rather than on general data processing issues.
The class continues with an examination of approaches important in embedded software testing. High-level testing
approaches are discussed for advanced stages of system development and integration, including security testing,
stress testing and independent verification and validation. Techniques and tools are presented to support these
approaches.
The debugging portion of the course starts with a survey of the wide range of categories of bugs from which
embedded and real-time software can suffer . It then focuses on a detailed discussion of the "top 5" kinds of bugs
that can be "project killers" or "product killers", including stack overflows, race conditions, deadlocks, timing problems,
and re-entrancy conditions . Emphasis is placed on multitasking and timing behaviors.
The class continues with an examination of debugging tools that are helpful in real-time software development.
'Liveness' issues including deadly embrace, lockout, memory starvation and CPU starvation are discussed in depth.
As are the particular sorts of bugs that tend to create problems in multi-tasking software when using real-time
operating systems. Emphasis then focuses on the unique sorts of bugs that plague Interrupt Service Routines
("ISRs"), such as interrupt overflow.
Participants are asked to do detailed exercises on many of the techniques presented, so that the concepts and
methods taught are reinforced and absorbed into the participant's arsenal of testing and debugging skills.
This course is not a general course about software testing and debugging, but rather it is highly focused on the
testing and debugging of embedded, time-constrained, resource-constrained multitasking software that may run
under the control of a modern real-time operating system ("RTOS").
WHO SHOULD ATTEND ?
This course is intended for practicing real-time and embedded systems software designers, software developers,
project managers, testers and quality engineers who have responsibility for planning, implementing, testing and
debugging the software for real-time and embedded computer systems.
Course participants are expected to have practical knowledge of coding for real-time and embedded systems, plus
some familiarity with at least one RTOS. [This knowledge can also be gained by attending one of our prerequisite
introductory courses "Introduction to Embedded Systems and Software" or "Introduction to Real-Time Operating
Systems".]
COURSE OBJECTIVES
The primary goal of this course is to give the participant the skills necessary to systematically identify and correct
defects in software for real-time computer systems developed with or without an RTOS. This is a very practical,
results-oriented course that will provide knowledge and skills that can be applied immediately.
CO-REQUISITE COURSE
Additional knowledge useful for attendees of this course can be obtained at the co-requisite advanced course
"Architectural Design of Real-Time Software". [For on-site presentation, both courses can be combined in a 4- or
5-day format.]
COURSE CONTENTS
Embedded Testing Fundamentals
8 Principles of Testing Embedded Software
Black Box Approaches: Equivalence Partitioning, Boundary Condition Testing
White Box Approaches: Stubs and Drivers
Code Coverage Levels
High Payoff White, Gray, and Black Box Tests
Unique Issues in Embedded Testing
Exercise: DO-178B "Modified Condition / Decision Coverage"
Real-Time Correctness: 'Liveness', Race Conditions
Embedded Testing Techniques
Input Data Testing
Exercise: Backus-Naur Form of a Protocol
State Testing: State Transition Diagrams and Harel StateCharts
Exercise: State Transitions of High-Availability Switchover
Pairwise Testing and Orthogonal Arrays
Exercise: Pairwise Test Design
Fault Injection Testing
Life Cycle for Embedded Software Testing
Top-Down vs. Bottom-Up Integration: Embedded Testbeds
System & Acceptance Testing
Security Testing
Stress Testing: Case Studies of NASA Spacecraft
Cause-Effect Graphing & Decision Tables
Exercise: Cause-Effect Test Design for a Temperature Display
Independent Verification and Validation
When to Stop Testing ?
Concepts for Debugging of Embedded / Real-Time Software
What is a Bug ?
The Debugging "Cookbook"
9 Golden Guidelines for Debugging
A Taxonomy of Real-Time / Embedded Bugs
Exercise: Bugs when Porting to New Hardware
David's "Top 5" Real-Time Bugs
Stack Overflows
Race Conditions
Deadlocks
Timing Problems
Re-entrancy Conditions
Real-Time Debugging Tools and their Limitations
Where Traditional Debugging Techniques are Insufficient
The Correctness Killers: Violations of Liveness
Deadlock: 4 Ingredients, and 8 'Fixes'
Task Lockouts and Starvations
The 3 Classes of Real-Time Software Debuggers
Cutting Edge Debuggers: Time-Reversal, SensorPoints
Logic Analyzers
Post-Mortem Debugging
Exercise: Bug in a Real-Time Controller
Mindsets for Debugging Real-Time Software
Cause and Effect: The Scientific Method
Assertions and Invariants
Memory Allocation/De-allocation Bugs
Proactive Debugging Techniques
Bugs in Multi-tasking and ISRs: The Mis-use of RTOS's
Task Decomposition and Communication Defects
Multi-tasking Debugging Techniques
Interrupt Service Routine Design Defects
RTOS Usage Defects
Exercise: Bug in Interrupt Handling
Programming Bugs in Real-Time Software
C-Language Ambiguities and Coding Disciplines
C-Language Code Analysis and Tools
Static vs. Dynamic Code Analysis
Exercise: C-Language Shenanigans
Metrics Predict Bugs
Exercise: Cyclomatic Complexity
Taxonomy of Embedded Testing Tools
Update on Static Code Analysis
The JPL "Power of 10" Coding Rules
Optional Sections:
Hard Real-Time Performance Bugs and Calculations
Bugs in Multi-Core and Symmetric Multi-Processing Software
Final Examination.
INSTRUCTOR: Dr. David Kalinsky
Check it out! Get the "flavor" of this course by perusing
several of this course's reading assignments:
"The Enigmatic Avionic", or
"Mutexes Battle Priority Inversions".
several of this course's reading assignments:
"The Enigmatic Avionic", or
"Mutexes Battle Priority Inversions".
| © Copyright 2010, D. Kalinsky Associates, All Rights Reserved. This page Updated February 18, 2010 |