Subalgebra Definition If A is an algebra over a field K, and B is a subalgebra, must B be an algebra over K or can it also be an algebra over some subfield of K? For example, if you take $\mathbb{R}$ as an algebra over $\mathbb{R}$, would it be correct to call $\mathbb{Q}$ a subalgebra, since it is an algebra over $\mathbb{Q}$? I would think not, but it seems to satify all the axioms.

No, for the same reason it is not sensible to call a subset of $\Bbb R^n$ which is a $\Bbb Q$ vector space an "subspace of $\Bbb R^n$. Lots of stuff would break down, most obviously dimension theorems.

For both vector $F$ vector spaces and $F$-algebras, it's implied that the subobjects inherit the same operation of scaling with elements of $F$.