A modified version of the double potential formalism for the electrodynamics of dyons is constructed. Besides the two vector potentials, this manifestly duality invariant formulation involves four additional potentials, scalar potentials which appear as Lagrange multipliers for the electric and magnetic Gauss constraints and potentials for the longitudinal electric and magnetic fields. In this framework, a static dyon appears as a Coulomb-like solution without string singularities. Dirac strings are needed only for the Lorentz force law, not for Maxwell\'s equations. The magnetic charge no longer appears as a topological conservation law but as a surface integral on a par with electric charge. The theory is generalized to curved space. As in flat space, the string singularities of dyonic black holes are resolved. As a consequence all singularities are protected by the horizon and the thermodynamics is shown to follow from standard arguments in the grand canonical ensemble.