f(x)=(1+1/tanx)sin^2x+msin(x+π/4)sin(x-π/4),
（1）
m=0,
f(x)=(1+cosx/sinx)*sin²x
=sin²x+sinxcosx
=1/2(1-cos2x)+1/2sin2x
=1/2sin2x-1/2cos2x+1/2
=√2/2(√2/2sin2x-√2/2cos2x)+1/2
=√2/2sin(2x-π/4)+1/2
∵x∈(0,π/2)
∴2x-π/4∈(-π/4,3π/4)
f(x)max=√2/2+1/2
2x-π/4=-π/4时,f(x)=0
f(x)值域为(0,√2/2+1/2)
(2)
∵tana=2,即sina/cosa=2
∴ sina=2cosa代入
sin²a+cos²a=1得：
cos²a=1/5,sin²a=4/5
sin(a+π/4)sin(a-π/4)
=sin(a+π/4)sin[-π/2+(a+π/4)]
=-sin(a+π/4)cos(a+π/4)
=-1/2sin(2a+π/2)
=-1/2cos2a
=-1/2(2cos²a-1)
=-1/2(2*1/5-1)
=3/10
∵f(a)=3/5,
∴f(a)=(1+1/tana)sin²a+msin(a+π/4)sin(a-π/4)
=(1+1/2)*4/5+3m/10=3/5
∴3m/10=-3/5
∴m=-2