Without the 3x3 box constraints, this is just counting Latin squares (Wikipedia, Mathworld).
The number of Latin squares of size $n\times n$ is sequence A002860 in OEIS. See also A000315.
The short answer is that there is no known formula, but the growth is so fast that you won't get far trying to count them by brute force however good your heuristic.