If I were asking for an English speaker's adress,I would put it in this way:
I wonder if you mind my asking your adress.