In badminton When is ok to go into neighbors court to retrieve the birdie?

Not when they're still playing - Obviously.
Either allow them to stop their rally partway through, or wait until they've finished it.
In both scenariOS, apologise.