1# Chapter 5 - Automated Testing
2 
3> Tests are not just for correctness. They are the first place people look to understand how your code works.
4 
5* Tests in rust are declared with the attribute macro `#[test]`. Most code editors can compile and run the functions declared under the macro individually or blocks of them.
6* Test can have special compilation flags with `#[cfg(test)]`. Also executable in code editors if it contained `#[test]`, it is a good way to mock complicated functions or override traits.
7 
8## 5.1 Tests as Living Documentation
9 
10In Rust, as in many other languages, tests often show how the functions are meant to be used. If a test is clear and targeted, it's often more helpful than reading the function body, when combined with other tests, they serve as living documentation.
11 
12### Use descriptive names
13 
14> In the unit test name we should see the following:
15> * `unit_of_work`: which *function* we are calling. The **action** that will be executed. This is often be the name of the the test `mod` where the function is being tested.
16```rust
17#[cfg(test)] 
18mod test { 
19    mod function_name { 
20        #[test] 
21        fn returns_y_when_x() { ... } 
22    } 
23}
24```
25> * `expected_behavior`: the set of **assertions** that we need to verify that the test works.
26> * `state_that_the_test_will_check`: the general **arrangement**, or setup, of the specific test case.
27 
28#### ❌ Don't use a generic name for a test
29```rust
30#[test]
31fn test_add_happy_path() {
32    assert_eq!(add(2, 2), 4);
33}
34```
35#### ✅ Use a name which reads like a sentence, describing the desired behavior
36> Alternatively, if you function has too many tests, you can blob them together in a `mod`, it makes it easier to read and navigate.
37 
38```rust
39// OPTION 1
40#[test]
41fn process_should_return_blob_when_larger_than_b() {
42    let a = setup_a_to_be_xyz();
43    let b = Some(2);
44    let expected = MyExpectedStruct { ... };
45 
46    let result = process(a, b).unwrap();
47 
48    assert_eq!(result, expected);
49}
50 
51// OPTION 2
52mod process {
53    #[test]
54    fn should_return_blob_when_larger_than_b() {
55        let a = setup_a_to_be_xyz();
56        let b = Some(2);
57        let expected = MyExpectedStruct { ... };
58 
59        let result = process(a, b).unwrap();
60 
61        assert_eq!(result, expected);
62    }
63}
64```
65 
66> When executing `cargo test` the test output for each option will look like:
67> Option 1: `process_should_return_blob_when_larger_than_b`.
68> Option 2: `process::should_return_blob_when_larger_than_b`.
69 
70### Use modules for organization
71 
72Most IDEs can run a single module of tests all together.
73The test name in the output will also contain the name of the module.
74Together, that means you can use the module name to group related tests together:
75 
76```rust
77#[cfg(test)]
78mod test { // IDEs will provide a ▶️ button here
79 
80    mod process {
81        #[test] // IDEs will provide a ▶️ button here
82        fn returns_error_xyz_when_b_is_negative() {
83            let a = setup_a_to_be_xyz();
84            let b = Some(-5);
85            let expected = MyError::Xyz;
86            
87            let result = process(a, b).unwrap_err();
88            
89            assert_eq!(result, expected);
90        }
91 
92        #[test] // IDEs will provide a ▶️ button here
93        fn returns_invalid_input_error_when_a_and_b_not_present() {
94            let a = None;
95            let b = None;
96            let expected = MyError::InvalidInput;
97 
98            let result = process(a, b).unwrap_err();
99 
100            assert_eq!(result, expected);
101        }
102    }
103}
104```
105 
106### Only test one behavior per function
107 
108To keep tests clear, they should describe _one_ thing that the unit does.
109This makes it easier to understand why a test is failing.
110 
111#### ❌ Don't test multiple things in the same test
112```rust
113fn test_thing_parser(...) {
114    assert!(Thing::parse("abcd").is_ok());
115    assert!(Thing::parse("ABCD").is_err());
116}
117```
118 
119#### ✅ Test one thing per test
120```rust
121#[cfg(test)]
122mod test_thing_parser {
123    #[test]
124    fn lowercase_letters_are_valid() {
125        assert!(
126            Thing::parse("abcd").is_ok(),
127            // Works like `eprintln`, `format` and `println` macros
128            "Thing parse error: {:?}", 
129            Thing::parse("abcd").unwrap_err()
130        );
131    }
132 
133    #[test]
134    fn capital_letters_are_invalid() {
135        assert!(Thing::parse("ABCD").is_err());
136    }
137}
138```
139 
140> `Ok` scenarios should have an `eprintln` of the `Err` case.
141 
142### Use very few, ideally one, assertion per test
143 
144When there are multiple assertions per test, it's both harder to understand the intended behavior and 
145often requires many iterations to fix a broken test, as you work through assertions one by one.
146 
147❌ Don't include many assertions in one test:
148 
149```rust
150#[test]
151fn test_valid_inputs() {
152    assert!(the_function("a").is_ok());
153    assert!(the_function("ab").is_ok());
154    assert!(the_function("ba").is_ok());
155    assert!(the_function("bab").is_ok());
156}
157```
158 
159If you are testing separate behaviors, make multiple tests each with descriptive names.
160To avoid boilerplate, either use a shared setup function or [rstest](https://crates.io/crates/rstest) cases *with descriptive test names*:
161```rust
162#[rstest]
163#[case::single("a")]
164#[case::first_letter("ab")]
165#[case::last_letter("ba")]
166#[case::in_the_middle("bab")]
167fn the_function_accepts_all_strings_with_a(#[case] input: &str) {
168    assert!(the_function(input).is_ok());
169}
170```
171 
172> Considerations when using `rstest`
173>
174> * It's harder for both IDEs and humans to run/locate specific tests.
175> * Expectation vs condition naming is now visually inverted (expectation first).
176 
177## 5.2 Add Test Examples to your Docs
178 
179We will deep dive into docs at a later stage, so in this section we will just briefly go over how to add tests to you docs. Rustdoc can turn examples into executable tests using `///` with a few advantages:
180 
181* These tests run with `cargo test` **BUT NOT** `cargo nextest run`. If using `nextest`, make sure to run `cargo t --doc` separately.
182* They serve both as documentation and correctness checks, and are kept up to date by changes, due to the fact that the compiler checks them.
183* No extra testing boilerplate. You can easily hide test sections by prefixing the line with `#`.
184* ❗ There is no issue if you have test duplication between doc-tests and other non-public facing tests.
185 
186```rust
187/// Helper function that adds any two numeric values together.
188/// This function reasons about which would be the correct type to parse based on the type
189/// and the size of the numeric value.
190/// 
191/// # Examples
192/// 
193/// ```rust
194/// # use crate_name::generic_add;
195/// use num::numeric;
196/// 
197/// # assert_eq!(
198/// generic_add(5.2, 4) // => 9.2
199/// # , 9.2)
200/// 
201/// # assert_eq!(
202/// generic_add(2, 2.0) // => 4
203/// # , 4)
204/// ```
205```
206 
207This documentation code would look like:
208```rust
209use num::numeric;
210 
211generic_add(5.2, 4) // => 9.2
212generic_add(2, 2.0) // => 4
213```
214 
215## 5.3 Unit Test vs Integration Tests vs Doc tests
216 
217As a general rule, without delving into *test pyramid naming*, rust has 3 sets of tests:
218 
219### Unit Test
220 
221Tests that go in the **same module** as the **tested unit** was declared, this allows the test runner to have visibility over private functions and parent `use` declarations. They can also consume `pub(crate)` functions from other modules if needed. Unit tests can be more focused on **implementation and edge-cases checks**.
222 
223* They should be as simple as possible, testing one state and one behavior of the unit. KISS.
224* They should test for errors and edge cases.
225* Different tests of the same unit can be combined under a single `#[cfg(test)] mod test_unit_of_work {...}`, allowing multiple submodules for different `units_of_work`.
226* Try to keep external states/side effects to your API to minimum and focus those tests on the `mod.rs` files.
227* Tests that are not yet fully implemented can be ignored with the `#[ignore = "optional message"]` attribute.
228* Tests that intentionally panic should be annotated with the attribute `#[should_panic]`.
229 
230```rust
231#[cfg(test)]
232mod unit_of_work_tests {
233    use super::*;
234 
235    #[test]
236    fn unit_state_behavior() {
237        let expected = ...;
238        let result = ...;
239        assert_eq!(result, expected, "Failed because {}", result - expected);
240    }
241}
242```
243 
244### Integration Tests
245 
246Tests that go under the `tests/` directory, they are entirely external to your library and use the same code as any other code would use, not have access to private and crate level functions, which means they can **only test** functions on your **public API**. 
247 
248> Their purpose is to test whether many parts of the code work together correctly, units of code that work correctly on their own could have problems when integrated.
249 
250* Test for happy paths and common use cases.
251* Allow external states and side effects, [testcontainers](https://rust.testcontainers.org/) might help.
252* if testing binaries, try to break **executable** and **functions** into `src/main.rs` and `src/lib.rs`, respectively.
253 
254```
255├── Cargo.lock 
256├── Cargo.toml 
257├── src 
258│   └── lib.rs 
259└── tests 
260    ├── mod.rs 
261    ├── common 
262    │   └── mod.rs 
263    └── integration_test.rs
264```
265 
266### Doc Testing
267 
268As mentioned in section [5.2](#52-add-test-examples-to-your-docs), doc tests should have happy paths, general public API usage and more powerful attributes that improve documentation, like custom CSS for the code blocks.
269 
270### Attributes:
271 
272* `ignore`: tells rust to ignore the code, usually not recommended, if you want just a code formatted text, use `text`.
273* `should_panic`: tells the rust compiler that this example block will panic.
274* `no_run`: compiles but doesn't execute the code, similar to `cargo check`. Very useful when dealing with side-effects for documentation.
275* `compile_fail`: Test rustdoc that this block should cause a compilation fail, important when you want to demonstrate wrong use cases.
276 
277## 5.4 How to `assert!`
278 
279Rust comes with 2 macros to make assertions:
280* `assert!` for asserting boolean values like `assert!(value.is_ok(), "'value' is not Ok: {value:?}")`
281* `assert_eq!` for checking equality between two different values, `assert_eq!(result, expected, "'result' differs from 'expected': {}", result.diff(expected))`.
282 
283### 🚨 `assert!` reminders
284* Rust asserts support formatted strings, like the previous examples, those strings will be printed in case of failure, so it is a good practice to add what the actual state was and how it differs from the expected.
285* If you don't care about the exact pattern matching value, using `matches!` combined with `assert!` might be a good alternative.
286```rust
287assert!(matches!(error, MyError::BadInput(_), "Expected `BadInput`, found {error}"));
288```
289* Use `#[should_panic]` wisely. It should only be used when panic is the desired behavior, prefer result instead of panic.
290* There are some other that can enhance your testing experience like:
291    * [`rstest`](https://crates.io/crates/rstest): fixture based test framework with procedural macros.
292    * [`pretty_assertions`](https://crates.io/crates/pretty_assertions): overrides `assert_eq` and `assert_ne`, and creates colorful diffs between them.
293 
294## 5.5 Snapshot Testing with `cargo insta`
295 
296> When correctness is visual or structural, snapshots tell the story better than asserts.
297 
2981. Add to your dependencies:
299```toml
300insta = { version = "1.42.2", features = ["yaml"] }
301```
302> For most real world applications the recommendation is to use YAML snapshots of serializable values. This is because they look best under version control and the diff viewer and support redaction. To use this enable the yaml feature of insta.
303 
3042. For a better review experience, add the CLI `cargo install cargo-insta`.
305 
3063. Writing a simple test:
307```rust
308fn split_words(s: &str) -> Vec<&str> {
309    s.split_whitespace().collect()
310}
311 
312#[test]
313fn test_split_words() {
314    let words = split_words("hello from the other side");
315    insta::assert_yaml_snapshot!(words);
316}
317```
318 
3194. Run `cargo insta test` to execute, and `cargo insta review` to review conflicts.
320 
321To learn more about `cargo insta` check out its [documentation](https://insta.rs/docs/quickstart/) as it is a very complete and well documented tool.
322 
323### What is snapshot testing?
324 
325Snapshot testing compares your output (text, Json, HTML, YAML, etc) against a saved "golden" version. On future runs, the test fails if the output changes, unless humanly approved. It is perfect for:
326* Generate code.
327* Serializing complex data.
328* Rendered HTML.
329* CLI output.
330 
331#### ❌ What not to test with snapshot
332* Very stable, numeric-only or small structured data associated logic (prefer `assert_eq!`).
333* Critical path logic (prefer precise unit tests).
334* Flaky tests, randomly generated output, unless redacted.
335* Snapshots of external resources, use mocks and stubs.
336 
337## 5.6 ✅ Snapshot Best Practices
338 
339* Named snapshots, it gives meaningful snapshot files names, e.g. `snapshots/this_is_a_named_snapshot.snap`
340```rust
341assert_snapshot!("this_is_a_named_snapshot", output);
342```
343 
344* Keep snapshots small and clear. 
345```rust
346// ✅ Best case:
347assert_snapshot!("app_config/http", whole_app_config.http);
348 
349// ❌ Worst case:
350assert_snapshot!("app_config", whole_app_config); // Huge object
351```
352 
353> #### 🚨 Avoid snapshotting huge objects 
354> Huge objects become hard to review and reason about.
355 
356* Avoid snapshotting simple types (primitives, flat enums, small structs):
357```rust
358// ✅ Better:
359assert_eq!(meaning_of_life, 42);
360 
361// ❌ OVERKILL:
362assert_snapshot!("the_meaning_of_life", meaning_of_life); // meaning_of_life == 42
363```
364 
365* Use [redactions](https://insta.rs/docs/redactions/) for unstable fields (randomly generated, timestamps, uuid, etc):
366```rust
367use insta::assert_json_snapshot;
368 
369#[test]
370fn endpoint_get_user_data() {
371    let data = http::client.get_user_data();
372    assert_json_snapshot!(
373        "endpoints/subroute/get_user_data",
374        data,
375        ".created_at" => "[timestamp]",
376        ".id" => "[uuid]"
377    );
378}
379```
380* Commit your snapshots into git. They will be stored in `snapshots/` alongside your tests.
381* Review changes carefully before accepting.
382