Let G be a group and H be a subgroup of G. We say that H is left relatively convex in G if the left G-set G/H has at least one G-invariant order; when G is left orderable, this holds if and only if H is convex in G under some left ordering of G. We give a criterion for H to be left relatively convex in G that generalizes a famous theorem of Burns and Hale and has essentially the same proof. We show that all maximal cyclic subgroups are left relatively convex in free groups, in right-angled Artin groups, and in surface groups that are not the Klein-bottle group. The free-group case extends a result of Duncan and Howie. More generally, every maximal m-generated subgroup in a free group is left relatively convex. The same result is valid, with some exceptions, for compact surface groups. Maximal m-generated abelian subgroups in right-angled Artin groups are left relatively convex. If G is left orderable, then each free factor of G is left relatively convex in G. More generally, for any graph of groups, if each edge group is left relatively convex in each of its vertex groups, then each vertex group is left relatively convex in the fundamental group; this generalizes a result of Chiswell.