It's random based on luck, and probably based on what area you're in too, since one would assume if you're closer to their hideout, you'd see more of them.
Luck is a HUGE factor in encounters though, enough so that I've decided to pretty much always have 8+ luck.