Is case-by-case hyphenated

Only when it's being used as an adjective. If one says "These requests will be evaluated on a case-by-case basis," there should be hyphenation.

However, if the phrase is used as a noun or an adverb, it will not have hyphenation, like so: "We will review these requests case by case."

Basically, if it's modifying, or affecting the meaning of, a noun, it should be hyphenated. If it's affecting a verb (in the second case, HOW reviewing is done), then it need not be hyphenated.