Why is calculus so hard?

Calculus in itself is not hard, it is usually remembering the algebra and previous math classes that is hard. New concepts are introduced in Calculus, but isn't it the same with any new subject? For example, many problems in integration, the actual calculus is not the hard part, it is using all of the algebra and other concepts you have used your whole life to simplify the problem so it is easy to solve.