‘We’re too frightened to talk about immigration,’ warns top Labour party veteran