Wave energy is the most complex renewable energy, which leads to the poor implementation of its control algorithm. Aiming at this problem, this article is based on the dynamic model of a fully enclosed oscillating buoy wave energy converter, a port controlled Hamiltonian with dissipation(PCHD) is derived. Accordingly, the passivity controller can be designed. In order to improve the dynamic performance of the system, the damping is injected into the passivity based controller. Simulation results show that the converter system can rapidly track the best state and have an excellent dynamic performance when the state of wave changes. The system adopts a model of PCHD, which makes the designing process of controller have the characteristics of structural, clear physical meanings and the simple design.