Is a dimension a parallel world?

Yes. At least, that's more or less one of the meanings of the Word "dimension". Mind you, that doesn't mean that such parallel worlds actually exist (they may, or may not, exist); it just means that the Word is used in this sense.