Mazeppa was platted in 1855, and named in honor of Hetman Ivan Mazepa via a poem by Lord Byron. The city was incorporated in 1877.
The Mazeppa Public School was built in 1858 and further expanded on many times, including after the fire on December 26, 1975 that destroyed the historic elementary school. The newer adjoining high school and 1940s gymnasium were saved from the blaze. The Mazeppa Public School system merged with the Zumbrota Public School system in fall of 1987 to create the Zumbrota-Mazeppa Public Schools.
The city contains one property listed on the National Register of Historic Places: the 1904 Walnut Street Bridge.