'I was told if we didn't go back there would be trouble'