Unless you are talking about business with this woman by cell then you should only keep your casual conversations at work. You already know you are playing with fire as you have absolutely no reason to constantly talk with this coworker and therefore, whether you have an affair with this woman or not you are, in ways cheating on your wife. When in doubt put yourself in your wife's shoes .. how would you feel if she did what you are doing?