Although $B$ is not symmetric, it is diagonally similar to a symmetric matrix.
Let $P$ be the matrix whose columns are the characteristic vectors of your partition and let $Q$ be $P$ with columns scaled so they are unit vectors. So $Q=PD^{-1/2}$ where $D=P^TP$ (and is diagonal). Your matrix $B$ is $D^{-1}P^TAP$, which is not symmetric in general. But \\[ D^{1/2} B D^{-1/2} = D^{1/2} D^{-1}P^TAP D^{-1/2} = Q^TAQ. \\] So $B$ is diagonally similar to $Q^TAQ$, which is symmetric.