Astronomy and Space topics are perceived as holding universal fascination. It is widely considered that exposure to such topics inspires people, changes their perspective and leads to an uptake in science and STEM subjects. But very rarely is the impact of such communication evaluated rigorously and scientifically. There is a need for more rigorous evaluation methods which would reveal the successes and failures of current methods and tools of astronomy communication and whether they might lead to any inadvertent harm. The IAU Office of Astronomy for Development (OAD) and Hosei University together with the South African Astronomical Observatory conducted a randomised controlled trial (RCT) in Cape Town, South Africa to test whether exposure to an astronomy intervention affects empathy and altruism in children. The pilot demonstrated that it is possible to use such methods to evaluate impact of science communication in an inexpensive manner.