In this paper, we present a general method for rank-constrained optimization. We use an iterative convex optimization procedure where it is possible to include extra convex constraints. The proposed approach has potential application in several areas. We focus on the problem of Factor Analysis. In this case, our approach provides sufficient flexibility to handle correlated errors. The benefits of the method are demonstrated via a simulation study.