The BOS have had 200 years to build up their numbers. It's the strongest faction in California after NCR
The brotherhood doesn't build its numbers though. Even after they were crushed under the NCR's feet, their philosophy didn't change ; just ask Veronica how to join them. "You're born into it. My parents, their parents, so on.". Until the war, they never needed numbers to be strong. Just look how many soldiers Maxson sends to attack the Unity's HQ. Three men. And they come from the most populated, active and advanced chapter of the brotherhood, not from a minor chapter like the Mojave's. Yes, three men IS an army, in the elder's eyes. An army he actually had to beg for in front of the council of elders.
Considering that they needed three times less resources to fuck the NCR in the butt when they bombed Redding's gold mines, his philosophy made sense... Until they had to face an actual army in a real war. A brotherhood could beat anyone in combat, true. But in war ? That's a totally different scenario.
I'm judging the size of Elder Lyons chapter by the fact they are able to fight a war against Super Mutants
Without any kind of success for twenty years. Their main positions could be overrun in a single day if their luck runs out, and they backed away from several of them because they couldn't hold them. Their intel couldn't tell them anything about the location of the enemy for twenty years, they have no backup, virtually nobody to recruit, half of their men have abandoned them and they don't have a single supply line. It's a miracle they were still alive after a month in this situation, actually.
They didn't engage in any kind of contact before activating Liberty Prime, because they would have been exterminated. Sarah Lyons herself admits it. Just like they didn't attack the Enclave by themselves in Fallout 2, they were backed by the NCR when they finally took action. And at this point, the Enclave was already pretty low.
and more even AFTER having half of their number break away to form the Outcasts. We've got something like 50 Brothers of Steel "on camera" and that's assuming you assume the population numbers of Fallout 3 are literally true.
Hell, 50 people is about 15 times the resources Maxson agreed to allocate to attack the Unity's HQ, and he had all the resources of the Brotherhood central headquarters, which included freaking airships. And he had to BEG these three men from the council of elders. The entire bunker's only entrance is protected by two knights only...
50 people is a huge number for the brotherhood. Helios one was one of the biggest battles the brotherhood ever took part in, and it cannot hold much more than a few dozen people. Let's assume that they had 50 people in there, aka, an entire chapter's militant branch. The NCR outnumbered them 15 to 1. That makes 750 NCR Soldiers besieging Helios One. That's huge numbers when it comes to Fallout, even for the NCR. If anything, I doubt that there were more than 20-30 Brotherhood soldiers holding Helios, really.
We also see this again in Fallout tactics. In their eyes, four men is more than enough to attack an entire raider fortress, evacuate resources from a hostile city or liberate a city. They never operated with a war mindset, they were always in skirmish numbers and tactics. Always.
It made sense in the first decades after the bombs, since there was no nation to fight, no borders to defend. And they stayed in that mindset, refusing to accept that the world changed, and they needed to change as well. Their refusal to change is their demise, just like it was Japan's demise when they refused to take interest in the outside world, until the outside world came knocking on their door with gatling guns.
Hell, Elijah could easily rally loyalists, but he doesn't. He's an old timey brotherhood member, with old brotherhood mindset : with the right tools, you can cripple a country with one man.
Redding is the proof of that. Graham is the proof of that. Ulysses is the proof of that. Elijah could have been another proof of his own philosophy, too bad he decided to fuck with THE mailman.
The brotherhood never needed to have an army. Hell, when they exiled every single rebel, these guys were so few in numbers they had to actually recruit auxiliaries, just to stay relevant.
The Mojave Chapter has over a hundred members "on camera" and that's after being massacred in a war and fleeing another war.
A hundred ? Hell, I only count about twenty, at most, plus a handful of patrols. Remember that the canon states that the courier CAN exterminate them if he has the element of surprise. A single man. And it's the one and only time in Fallout where the defeated faction is not only beaten, but actually exterminated. Every single one of them.
It wouldn't really be believable if the chapter was made out of a hundred people. The courier may be tough, but not THAT tough.
New Vegas is probably the only title where what you see on camera is not an approximation, or something that requires you to tell yourself "alright, but there's plenty more people in reality than the ones we see". Except for the battle of Hoover Dam,