Make perm(::Vector{Int}) constructor and application of permutations to integers faster and allocate less

Make perm(::Vector{Int}) constructor and application of permutations to integers faster and allocate less #21522