Don't be angry with me.
be angry with somebody是一个固定短语,表示对某人生气.
be angry at something也是一个固定短语,表示对某事生气.