I was confused by the fact that, even in the encapsulating armor, Mrs. Wright could somehow still tell I was a tribal... Although it did lead into the very funny line about how you *got* the armor, but nonetheless... Charisma is sort of a combo of how slick you are and how goodlooking you are. For an extreme example, take Adolph Hitler. Perhaps the most brilliant orator in the century, but he was a short, ugly little man, and not particularly intelligent, yet he managed to convince a lot of people that his fucked up ideas were actually *good*.
If it were up to me I'd put somewhere on the character screen an "appearance" figure, taking into consideration things like if you're a ghoul, a mutant, what perks you've taken (things like Sex Appeal, you know the stuff), what you're wearing, etc. I'd also make separate slots for the armor items like a "head" slot for gas masks, mirrored shades, riot helmets, etc. I'd also have two separate torso armor slots, so you could wear a bullet-proof vest under your leather jacket, but the bullet-proof vest's modifiers would only apply to torso shots. Anyway, so as to make it simple, a suit of power armor, and environment suit, or anything else contiguous like that, would just be dropped into the "outer" torso slot and it would automatically fill in all the slots it applied to, and dump anything else into the main inventory. Anyway the "Appearance" number would affect your applied charisma rolls, and it could be NPC-specific; I.E. a scientist won't really care if you're spiffy looking, he'll care if you are eloquent, while a simple farmer may be swayed by your well-kept leather coat (with two sleeves, even!) and cowboy boots.