Is there a need of physical security while using cryptography?

Yes. Decrypted or plain text - yet to be encrypted - messages need to be kept securely before they are encrypted and transmitted.