I didn't know you had come back 中的had come 不该用have came back吗

I didn't know you had come back 中的had come 不该用have came back吗
I didn't know you had come back 和I don't know you have come back.意思是不是一样。感觉很少人说I didn't know.能说说在实际口语中的区别吗，还是说表达这个意思时，口语中会用其他的句子。