For those who do stick to the 20 limit, it will be safer for pedestrians and bicycles but maybe when all the bikes overtake the cars, they might hit a 20 mph car coming towards them.
A lot of car drivers do keep speeds down where people walk or ride but also, many will drive around completely ignoring limits and driving on the rear of the slow cars. I think 30 mph is good enough in Towns and villages.