Can’t we say” il m’a fait peur quand j’étais petit?”

Hi Thekla,

yes, you could say that. It would mean that there was one occasion when you were small that he scared you. If you wanted to express that he did it habitually when you were small you would use the imperfect in both clauses:

Il me faisait peur quand j'étais petit.

-- Chris (not a native speaker).