If identical twins marry identical twins will their children look the same?

No. Nobody looks the same because of environmental factors, even if their DNA is identical.

But: If both of the parent-twins were identical, than the DNA of the children of both married couples would be as similar as the DNA of siblings. The cousins would not look the same, but would look more alike than most cousins look.