The mechanical characterization of rocks is important in engineering design and analysis of rock-related structures. In the current researches, rocks are classified as heterogeneous materials with anisotropic behavior, and advanced methods such as combined experimental-numerical approach are developed to characterize the behavior of rocks. In this study, the nanoindentation experiment in conjunction with the generalized means method is used to determine the Young’s modulus and hardness of eight different polycrystalline granite rocks. In the first step, the Young’s modulus and hardness of granites’ constituents are determined through nanoindentation tests on pure granite minerals. Then, the properties of granites are determined using generalized means method by considering the mechanical properties of minerals, their volume fractions and an empirical constant called the microstructural coefficient. Accurate results with less than 3% error are obtained for 62.5% of the granite samples. The generalized means is introduced as a simple and effective method to characterize the mechanical properties of heterogeneous polycrystalline rocks.