The analogue is known as freeness with amalgamation.
The definition is made in the same way: given an expectation $E: M \rightarrow A$, we say that ($*$-)subalgebras $M_1, ..., M_n$ are ($*$-)free with amalgamation over $A$ if $E(X_{i_1}...X_{i_m}) = 0$ whenever $E(X_{i_j}) = 0$, $X_{i_j} \in M_{i_j}$, and $i_j \
eq i_{j+1}$ for all $j$. We say that a family $\\{X_1, ... X_n\\}$ of $A$-valued random variables are free with amalgamation over $A$ if the $*$-algebras they generate with $A$ are free with amalgamation over $A$. Freeness for classical random variables is then just freeness with amalgamation over $\mathbb{C}$.
You can check out Free Random Variables by Voiculescu, Dykema, and Nica for more, but there are other good references depending on what you're looking for. Speicher has a book called Combinatorial Theory of the Free Product with Amalgamation and Operator-Valued Free Probability Theory that covers the combinatorics in a lot of detail.