楼主,有几种说法都对.
I returned home at midnight yesterday.
It was midnight yesterday when I come back home.
It was at mid-night that I went home yesterday.