Who were James Madison's election opponents?

In 1808, Charles C. Pinckney, Federalist candidate from SC was his main opposition. George Clinton from NY also ran against him and won 6 electoral votes in his home state.

In 1812, DeWitt Clinton ,from NewYork, was his Federalist opponent.