This paper provides a semiparametric framework for modelling multivariate conditional heteroskedasticity. First, we show that stochastic volatility factor models with possibly cross-correlated disturbances cannot be identified from returns conditional variance structure only, except when strong restrictions on the support of the probability distribution of latent factors volatility are maintained. Second, we provide an alternative way to maintain identifying restrictions through either higher order moments or through a specification of risk premiums based on constant prices of factor risks. In both cases, identification is obtained with conditional moment restrictions which pave the way for instrumental variables estimation and inference. A preliminary step of determination of the number of factors and identification of mimicking portfolios is proposed through a sequence of GMM overidentification tests which encompass Engle and Kozicki (1993) tests for common features.