diff --git a/posts/realistic-benchmarks.md b/posts/realistic-benchmarks.md index d824865..ad2ba1c 100644 --- a/posts/realistic-benchmarks.md +++ b/posts/realistic-benchmarks.md @@ -11,7 +11,7 @@ This depends on the power and configuration of our machine and doesn't convey an What we really care about is the relative relationships between different algorithms. -By itself, a number like 2 nanoseconds is non-significant and not useful to anyone. +By itself, a number like 2 nanoseconds is non-significant and not useful. However, once we add another competing algorithm to the mix, it becomes interesting. When algorithm A takes 2 ns to compute and algorithm B takes 4 ns to compute, we can see a relationship between A and B and that is the fact that A is twice as fast as B. @@ -44,6 +44,7 @@ Ideally we want an infinite amount of test samples, because the more we have, th This means that in the case of a web server, we want as much load as possible from our stress testing tool, because more samples will bring us closer to the real relative performance relationships of the algorithms tested. +The less stress we put on the server, the worse our results become. Some people have the misconception that a realistic benchmark should produce realistic absolute numbers. Realism in absolute numbers, especially when it comes at the cost of a worse approximation of `s`, is not useful to anyone.