With the development of smart grid, non-linear load such as power electronic devices has been widely used in electric power system, as a result, it brings power grid with a large number of harmonic, inter-harmonic. Due to inter-harmonics, full-cycle Fourier algorithm applied in theSmeasurementSofSmeters and characteristic quantity of relay protection extraction produces error, as a result, it causes fundamental wave and harmonic detection error, protection mal-operation and missing operation, etc. This article theoretically deduced full-cycle Fourier expression of the fundamental frequency signal containing a single frequency inter-harmonic, extracted the fundamental and second harmonic amplitude from the Fourier transform expression, and drew the fluctuation range curve of fundamental and second harmonic amplitude with the inter-harmonic’s frequency and content,finally, it concluded that general conclusions about the influence of harmonics on extracting the fundamental and second harmonic amplitude by full-cycle Fourier algorithm. At last, taking transformer differential protection as an example, we made simulation and verification of this method. The simulation shows that the inter-harmonic’s influence on full-cycle Fourier algorithm leads to the error of relay protection’s characteristic quantity and further causes protection malfunction or rejecting action.