congruences - Showing a number $n$ is prime if $(n-2)! equiv 1 pmod n$

I need to show that if $(n-2)! \equiv 1 \pmod n$ then $n$ is prime.
I think that if $n$ is composite, then we'll have every factor of $n$ in $(n-2)!$, and it would yield that $(n-2)! \equiv 0 \pmod n$.
However, I didn't use the fact that it specifically congruent to $1 \bmod n$, so I think I'm getting something fundamental wrong. Is my solution correct? Why do we demand congruence to $1 \bmod n$?