不知道你题目有没有错误,如果按照题目来做：
x^2-4xy-4y^2
= x^2-2*2xy+(2y)^2-8y^2
= (x-2y)^2 -8y^2
= (x-2y)^2 -(√8y)^2
= (x-2y-2√2y)((x-2y+2√2y)