让我苦恼的是去考虑（想象）我们已经浪费了多少时间.
It做形式主语,真正的主语是to think how much time we had wasted