Yes. You know that $P^{-1}AP=U$, where $U$ is upper triangular. Let $J$ be the matrix where the entries of the diagonal which is not the main one are all equal to $1$, whereas all other entries are equal to $0$. Then $J^{-1}UJ$ is lower diagonal; lets call it $L$. Then$$J^{-1}P^{-1}APJ=L$$and therefore$$A=(PJ)L(PJ)^{-1}.$$