matrices - "Inverse" of an $ntimes k$ matrix.

I know that only square matrices can have inverses. Nevertheless, assume that we have an $n \times k$ matrix $A $ with $n\neq k$. Then it may still be possible to construct a matrix $B$ such that
$$BA=I$$

For example, assume that $B=(A^TA)^{-1}A^T$, then $BA=(A^TA)^{-1}(A^TA)=I$

I know that $B$ cannot be called an inverse, due to the definition of inverse requiring $A$ to be a square matrix. Nevertheless, it is sort of like an inverse, so do we have a name/theory about these kinds of "inverses"?

Answer

If we see these matrices as representing linear maps, then your $B$ is called a left inverse of $A$. An $n\times k$ matrix $A$ has a left inverse iff $\operatorname{rank} A = k$, in other words, if all $k$ columns of $A$ are linearly independent. Similarily, a matrix has a right inverse iff it has linearly independent rows.

If a matrix has both a right and a left inverse, then both its rows and its columns must be linearly independent, so there must be equally many of them, and the matrix is therefore square. It also turns out that the left inverse and the right inverse must be the same matrix, and we call it the inverse.