IMC 2020 · Problem 2

Let $A$ and $B$ be $n \times n$ real matrices such that

$$\operatorname{rk}(AB - BA + I) = 1$$

where $I$ is the $n \times n$ identity matrix.

Prove that

$$\operatorname{trace}(ABAB) - \operatorname{trace}(A^2B^2) = \frac{1}{2}n(n - 1).$$

($\operatorname{rk}(M)$ denotes the rank of matrix $M$, i.e., the maximum number of linearly independent columns in $M$. $\operatorname{trace}(M)$ denotes the trace of $M$, that is the sum of diagonal elements in $M$.)