完全可以.who are you going swimming with?可以表达为whom are you going swimming with?
如果将介词with提到疑问词的前面的话,只能用whom.