Let $X\left( n \right)$, for $n\,\in \,\mathbb{N}$, be the set of all subsets of a metric space $\left( x,\,d \right)$ of cardinality at most $n$. The set $X\left( n \right)$ equipped with the Hausdorff metric is called a finite subset space. In this paper we are concerned with the existence of Lipschitz retractions $r:\,X\left( n \right)\,\to \,X\left( n\,-\,1 \right)$ for $n\,\ge \,2$. It is known that such retractions do not exist if $X$ is the one-dimensional sphere. On the other hand, Kovalev has recently established their existence if $X$ is a Hilbert space, and he also posed a question as to whether or not such Lipschitz retractions exist when $X$ is a Hadamard space. In this paper we answer the question in the positive.