Is it illegal for cyclists to go into a multi story car park if you do not have a car in there?

The owner of the car park might not approve, and can consider it trespassing. But it's not illegal as such. If you have your car in there (giving you a valid reason to go there) it shouldn't be a problem riding your bike to the car.