Testing and Quality
If it’s not tested, it doesn’t work’.’ When your tests passing lets you deploy without any concerns, your tests are good enough. Otherwise you’ve got more work to do
- Why test?
- Types of tests
- Acceptance Tests - Validation that requirements are met
- End To End Tests - Expensive but critical
- Integration Tests - Correct interactions with external world
- Snapshot Tests - Validating UX
- External Dependency Testing - The Humble Object
- Unit Tests - The code does what the developer wants.
- Compiler and static analysis based testing
- Non-functional testing
- Testing in production and monitoring
- To be categorized
- Great books
Why test?
Software needs to work, and if it’s not tested, it doesn’t work. In the rare event it works today, I assure you it’ll stop workign after you make a change. If you can’t make a change safely, your system will go to shit instantly.
Very related to design
- SOLID designs (OO joke there, haha!)
- Safe to refactor
- Always safe to deploy
- System is well documented
Types of tests
Acceptance Tests - Validation that requirements are met
End To End Tests - Expensive but critical
Include running in multiple configurations, like different devices, or different browsers, or different screen orientations
Integration Tests - Correct interactions with external world
Snapshot Tests - Validating UX
External Dependency Testing - The Humble Object
Often people try to text external dependancies, like drawing on the screen,or opening the door, and think it’s impossible. That is impossible, but you don’t need to test the external dependancy. You need to test everything else, all the business logic that does everything before the external dependancy.
To enable this, create a humble object that does only the screen operations, and test all interactions to and from that.
Extract all the logic from the hard-to-test component into a component that is testable via synchronous tests. This component implements a service interface consisting of methods that expose all the logic of the untestable component; the only difference is that they are accessible via synchronous method calls. As a result, the Humble Object component becomes a very thin adapter layer that contains very little code. Each time the Humble Object is called by the framework, it delegates to the testable component. If the testable component needs any information from the context, the Humble Object is responsible for retrieving it and passing it to the testable component. The Humble Object code is typically so simple that we often don’t bother writing tests for it because it can be quite difficult to set up the environment need to run it.
Unit Tests - The code does what the developer wants
Back in the 2000s, “amazing developers” walked through all their code in the debugger to make sure it was doing what was expected. But like all manual activities this gets dreary, error prone and skipped. Instead, write unit tests to ensure your code works as you expect.
Compiler and static analysis based testing
If you use a strongly typed language, and use types as much as you can, you get lots of testing for free from the compiler! Similar, if you can have automated code analysis you get testing for free through analysis. See AWS Code Guru, and some bugs it detects
These bugs are the kinds of errors that developers can unknowingly introduce in the course of their day-to-day work. Introducing bugs is easy, but tracking down their root causes can be hard. Some of the bugs even found issues that went against the official documentation. One team found a race condition with the Java ConcurrentHashMap type; the documentation said it was thread-safe, but if two threads picked up the process at the same time, the values of instantiated ConcurrentHashMap objects could be overwritten.
The first involved key derivation and password hashing using the Argon2 algorithm. Chan knew about password hashing, but not with the fairly new Argon2 algorithm. The second came from invoking a shell comment with subprocess.Popen([cmd], shell=True), which could risk unwanted privilege escalation in the shell. Instead, he used the shlex.split() and shlex.quote() commands to avoid invoking a shell at all.
From Programming with types:
Although a weak type system is easier to work with in the short term, as it doesn’t force programmers to explicitly convert values between types, it does not provide the same guarantees we get from a stronger type system. Most of the benefits described in this chapter and the techniques employed in the rest of this book lose their effectiveness if they are not properly enforced.
Examples:
- Making types for primitive types, like encoding units - E.g. a type for meters vs inches (space catastropy)
- A type for velocity vs volume.
- Rust for borrow
- JS to TS
- Pythong type system
- Implicit Typing vs Duck Typing
- It’s hard to make it compiler, but once compiles it works.