The following is a true (and well-known) theorem: \(\newcommand{\Tr}{\mathop{\rm Tr}}\)

Suppose \(A\) and \(B\) are bounded operators on a Hilbert space, and \(AB\) and \(BA\) are trace class.  Then \( \Tr(AB)=\Tr(BA) \).

This is easy to prove if one of the operators \(A,B\) is itself of trace class, or if they both are Hilbert-Schmidt (the obvious calculation works).  In the general case it is a bit harder.  The “usual” argument proceeds via Lidskii’s trace theorem – the trace of any trace-class operator is the sum of the eigenvalues – together with the purely algebraic fact that the nonzero eigenvalues of \(AB\) and \(BA\) are the same (including multiplicities). 

Lidskii’s theorem is a surprisingly delicate result though.  In class, I claimed that one can prove the theorem \(\Tr(AB)=\Tr(BA) \) in general without using Lidskii.  This is true; however, the “obvious” argument does not work any more.   (You want to rearrange a double series and a priori it is only conditionally convergent, rather than absolutely convergent.)  Rather, one needs something like the following reasoning (adapted from Lemma 2.1 of   On triangularization of algebras of operators, by Nordgren and Rosenthal, Crelle 327 (1981), 143–155).

Consider first the case where \(A\) is self-adjoint.  Let \(P_n\) be the spectral projection of \(A\) corresponding to \({\mathbb R}\setminus (-1/n,1/n)\).  We have

\[ \Tr(AB) = \lim_n \Tr(P_nAB) = \lim_n \Tr (P_nAP_n \cdot P_nBP_n). \]

(We moved the extra \(P_n\)’s into the middle because they commute with \(A\), and round to the far end using the trace property.)   Now consider \(P_nAP_n\) and \(P_nBP_n\) as operators on the Hilbert space \(H_n\) which is the range of \(P_n\).  Their composite equals \(P_nABP_n\) so is of trace class.  But \(P_nAP_n\) is invertible on \(H_n\) by construction.  Thus \(P_nBP_n\) is trace class on \(H_n\).  Consequently

\[ \Tr (P_nAP_n \cdot P_nBP_n) = \Tr (P_nBP_n \cdot P_nAP_n) \]

by the easy case of the result.  Unwinding the limits as before completes the proof for self-adjoint \(A\).

Now for the general case, we use polar decomposition to write \(A = VP\) with \(P\) positive and \(V\) a partial isometry. Then \(PB=V^*AB\) is of trace class.  Consequently

\[ \Tr(AB) = \Tr(VPB) = \Tr(PBV) = \Tr(BVP) = \Tr(BA) \]

where the first step (commuting V) uses the “easy case” and the second step (commuting P) uses the self-adjoint case proved above.

I was pointed to this argument by Math Overflow.