Is a batter out when he enters the batters box with an illegal bat?

Yes.

According to MLB Rule 6.06(d), a batter is out for illegal action when:

"He uses or attempts to use a bat that, in the umpire's judgment, has been altered or tampered with in such a way to improve the distance factor or cause an unusual reaction on the Baseball. This includes, bats that are filled, flat-surfaced, nailed, hollowed, grooved or covered with a substance such as paraffin, wax, etc.

No advancement on the bases will be allowed and any out or outs made during a play shall stand.

In addition to being called out, the player shall be ejected from the game and may be subject to additional penalties as determined by his League President.

Rule 6.06(d) Comment: A batter shall be deemed to have used or attempted to use an illegal bat if he brings such a bat into the batter's box."