Based on finite element modeling (FEM), the indentation response of graphene with soft substrate is investigated. The effects of the electric modulus ratio between graphene sheet and substrate (E_f/E_s), the thickness of graphene sheet and the indenter tip size on the indentation response are considered. Both substrate and graphene sheet are firstly simplified as linear elastic materials, the contribution of the graphene sheet to the overall indentation modulus of graphene/substrate is identified, and then the relationship between the overall modulus (E_tot), the substrate modulus (E_s) and the elastic modulus (E_f) of graphene is built, based on which the elastic modulus of graphene can be determined. In addition, after considering the viscoelastic model for the substrate and the nonlinear elastic model for the graphene sheet, the indentation response of graphene/substrate is reanalyzed and the contribution of E_f to the E_tot is reevaluated. The present work can provide a useful guideline to understand the mechanical property of graphene.