Ghouls don't live off radiation, they simply are able to survive in it without the penalty of death. Ghouls do have to both eat and drink water, as shown in Fallout 1 with the city of Necropolis and even in the Bethesda games themselves.
Mutants are not hyper intelligent unless they were never exposed to radiation before or were born underground. In other words, only a vault dweller or a member of the Enclave would actually create an intelligent super mutant. Literally everyone else would become a "dumb dumb" as Tabitha puts it in FNV. Thus they are extremely rare. Fallout 1 only has one smart super mutant, for example, the Lieutenant. Fallout 2 only has a handful of them. Same with New Vegas.
The psychic mutants aren't really mutants. They're humans that had FEV injected into their brains, thus why they look normal. Their brains were the only thing that was mutated and gave them powers. They are also extremely rare, and we only see 4 of them in the original game. After that the only other Psyker we've ever seen is the Forecaster in New Vegas. This pretty much means the Psykers are all but extinct unless the Forecaster proves this particular strain of FEV can be passed down through reproduction. Even then the Forecaster is the last of his kind. The Psykers have all but died out. Once he passes, unless he reproduces, there will be no more.
Mutants have already developed societies. Prospering societies in fact. They're called Jacobstown and Broken Hills. (ironically both founded by Marcus, an intelligent Super Mutant) There's also Black Mountain which is run by one of only 3 intelligent Nightkin in the game. (I count Lily because even though she has mental disorders with Leo, she can get over it by taking medicine and otherwise seems fairly smart). The only example of a ghoul society we've really seen is Underworld in FO3 or whatever it's called, but this is usually because ghouls intergrate into society with other people fairly well. (for example, Freeside) There's some ghoul racism, but for the most part ghouls and people intergrate fairly well unlike Super Mutants.