定语从句Yesterday when I met Mary

定语从句Yesterday when I met Mary
It's yesterday ＿when＿＿ I met Mary
为什么此处用when而不用which?
不是说定语从句后要加介词时才用关系副词吗?
例如：This is the factory __where___he worked last year
因为是work in the factory所以用where
那I met Mary yesterday不用加介词为什么也要用when?