The fear of bearded dairymen was something that that originated in New York, where the Board of Health banned dairy workers from growing beards. Dr Park, who worked for the board, was quoted as saying:
There is a real menace to the milk if the dairyman is bearded… the milker may be diseased himself…. and the dried sputum may accumulate on his beard and drop from it into the milk…. and you have, no doubt, noticed that men with long beards have a habit of stroking them downwards. That has the effect of brushing off any germs they may contain… The beard, particularly when damp, may become an ideal germ carrier.
Source: Moustaches, Whiskers & Beards, Lucinda Hawksley