Who determines whether or not you pay income tax?

The tax "rules" are actually laws...and enacted as such by your elected members of Congress...with all the give and take, political nonsense and lawyers, and such that it takes to write and pass by vote, anything. They are enforced by the courts and the IRS - with the IRS responsible to make certain operational and explanatory definitions, and things like the forms, to administer the process.