Yesterday when on the way home,he was caught in the heavy rain.这样吧
不过我觉得When on the way home yesterday,he was caught in the heavy rain.更通顺一点