have been to...“曾到过、去过...（并已经回来）”的意思
句子中有many times显然是去过（且回来后又去过）不止一次,所以用have been to.