Some people in England are not allowed to drive a car on the public highway because it would be dangerous for themselves and others if they were to do so. Nobody claims that this is an infringement of their rights. It is just common sense.
Banning unvaccinated people from attending events and entering buildings because if they did so it would be dangerous for other people is also common sense. It would only be an infringement of human rights if the vaccine was deliberately withheld from certain people for non-health-related reasons or if vaccination was compulsory.