In this paper, we study a resource allocation problem in orthogonal frequency division multiple access (OFDMA)-based Device-to-Device (D2D) communications. To this end, we propose a multi-objective optimization problem (MOOP) framework, which jointly maximizes the sum rate of D2D users (DUs) and cellular users (CUs) in uplink communications and minimizes the total transmit power. The proposed problem formulation takes into account the minimum data-rates and the maximum transmitted power budget for both DUs and CUs. We transform this MOOP into a single-objective optimization problem (SOOP) using the weighted sum method and then propose an approach to solve this SOOP via a monotonic approach yielding an efficient optimal solution. Furthermore, a suboptimal solution based on the successive convex approximation (SCA) is presented to compromise complexity and performance gain. This is done to reveal that the proposed suboptimal solution closely approaches an optimal solution through simulation analysis. Numerical results unveil an interesting tradeoff between D2Ds CUs and demonstrate the superiority of our proposed solution compared to other baseline schemes.