This was definitely a good technical foray into the mechanics required to find the discrepancy that pointed to the evidence that something was tampered with,… In Cliff Stolls case some years ago it was accounting for computer time on a big shared system (90 cent discrepancy on their monthly charge back at the University Data Center). In Andres case it was a 500 millisecond time difference in an app he was profiling looking to find regressions based on performance tuning he was in charge of. Those 500 milliseconds didn’t make any sense,… how could the app be using so many cycles. #amirite ? And it bothered Andres that he didn’t know why it was doing that. But eventually that thread lead to the discovery that the xz compression library had been altered and was injecting code to projects downstream that used it as part of their inventory of code to make things run. Yup, long chain of dependencies there, all intended to target the Secure Shell (ssh).