Let $G=\operatorname{GL}(V)$ be the group of automorphisms of $V$. Then as a $G$-module, $\mathcal L(V)$ is isomorphic to $V\otimes V^*$, and $\hom_k(\mathcal L(V),\mathcal L(V))$ is isomorphic to $V^{\otimes 2}\otimes V^{*\otimes 2}$. Your question is, in this language:
> what is the dimension of the $G$-invariant subspace of $V^{\otimes 2}\otimes V^{*\otimes 2}$?
You will find this done in treatements of classical invariant theory.If I recall correctly, this particular case is done in detail in Kraft+Procesi's notes on Classical Invariant Theory, which you can find online
Notice that $V^{\otimes 2}\otimes V^{*\otimes 2}$ is isomorphic as a $G$-module to $hom_k(V^{\otimes 2},V^{\otimes 2})$, so its invariant subspace is actually the space of $G$-equivariant maps, $hom_G(V^{\otimes 2},V^{\otimes 2})$. The fact that there are exactly two linear invariants in your sense is the case $m=2$ of the claim stated on page 31 of those notes —this is part of what's called Schur-Weyl duality.