Elementary matrices are invertible/nonsingular.

Theorem. Every elementary matrix is invertible (nonsingular), and its inverse is also an elementary matrix of the same type.

Specifically:

1. Row interchange: $E_{ij}^{-1} = E_{ij}$ (self-inverse)
2. Row scaling: $E_i(c)^{-1} = E_i(1/c)$ for $c \neq 0$
3. Row replacement: $E_{ij}(c)^{-1} = E_{ij}(-c)$

Proof. Each elementary row operation can be reversed by another elementary row operation. The matrix that reverses the operation is the inverse.

Example. If $E = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 2 & 0 & 1 \end{pmatrix}$ adds 2 times row 1 to row 3, then $E^{-1} = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -2 & 0 & 1 \end{pmatrix}$ subtracts 2 times row 1 from row 3.