If you were pushing very hard against a brick wall would you be doing work?

Yes, pushing against a brick wall would require force and displacement, which are the two components of work. You would be exerting energy to apply a force on the wall, but if the wall doesn't move, there would be no work done on the wall.