In pondering the question "Fallout vs Arcanum," I think one must clarify-- "versus" in what context? According to what criteria?
I think Fallout is a better game, but Arcanum is a better role-playing game.
A key criteria for assessing role-playing games is the extent to which the player, rather than the designer, influences the narrative. A good role-playing game should resemble a playground or toy room in a kindergarten-- presenting a wide variety of options for engagement. In a school, some children will prefer to draw pictures with a finger-painting set, others will entertain themselves by building castles with wooden blocks, while yet others prefer to shoot marbles, and so forth. The point here is less that different play options are available-- the point is that different children will have very different reasons for wanting to play any game at all. The variety of options (paint set, blocks, marbles) is an attempt to encourage all the children to explore their individual motivations (e.g. "I like to ponder aesthetics by painting rather than compete in a contest of hand-eye coordination by shooting marbles") rather than to teach them Newtonian physics (through marble-shooting) or the engineering significance of load-bearing structures (by arranging wooden blocks).
Any thoughtful game should strive to offer numerous opportunities for emergent gameplay-- play in which the player engages in activities satisfying to himself but which were likely not considered as fundamental features or functions by the designer. A well-made role-playing game, on the other hand, should strive for a much more difficult standard-- providing a maximum flexibility for emergent narratives to give meaning or purpose to the player-character avatar's activities as well as emergent gameplay representing the actions of the avatar in the game-world. For example, if I play Super Mario I may find opportunities for emergent gameplay in that I may strive to kill all Koopas or collect all possible coins, but the game's narrative solicits very little player input. Mario must rescue Princess Toadstool-- that's what Mario is doing, and that's all he'll ever do. A better example would be an adventure game in which one must gain access to a guarded room; a well-made adventure game would provide opportunities to sneak past the guard, overpower the guard, bribe the guard, or charm the guard, all at the player's discretion. Good computer role-playing games, on the other hand, should insofar as feasible resemble table-top games with human participants exercising their imaginations. For example if the game-master in a table-top game has prepared a "dungeon" scenario but the players are not interested in entering the dungeon and prefer to explore a nearby wilderness, there's no problem (at least for a well-prepared game-master)-- though in a sense the narrative will turn out very differently than the game-master imagined, since the ultimate objective of the game is to entertain the participants then the table-top game provides, with its flexibility, maximum opportunities for accomplishing that objective. Mario, on the other hand, can never decide he's had it with Princess Toadstool, and would instead prefer to move to New Zealand and marry Xena the Warrior Princess.
With its inexorably linear plot and stifling time-limits, Fallout fails in several ways to offer the flexibility required of a truly outstanding role-playing game. The game is simply not open-ended enough; opportunities for the player to disregard the provided narrative in favor of a new, private narrative more satisfying that particular player are inadequate in comparison to Arcanum. Yet Fallout is an outstanding game, in that its mechanics are well-designed and optimized to provide the desired experience (flexible characterization and development of the player-character, turn-based combat, and so forth).
This is incidentally why I believe Fallout 2 is a better role-playing game than Fallout. The larger, more diverse world offers more opportunities for the player-character to simply "wander the wasteland with nothing but a shotgun, getting into random adventures like Mad Max" as one once commonly-expressed sentiment in NMA days of yore put the matter. Arroyo? GECK? Who cares. It's my game and my character is going to live whatever life he or she wants (within at least some limits, of course).
In contrast to Fallout or even Fallout 2, Arcanum has a stipulated plot so minimal that the core narrative is nearly incoherent. The pathetic coward-god, Nasrudin, has grown too lazy to exercise his power and would like the player-character to help him. In return, he offers-- nothing, except perhaps a "pretty please, with sugar on top." Yet this nonsensical plot works to accentuate the virtues of Arcanum, by nearly insisting that the player construct one or more private narratives of his own in order to make sense of any particular playthrough.
Arcanum certainly has profound problems in terms of mechanics and the implementation of mechanics-- for example, the oft-mentioned and quite significant inadequacies of the combat system, which avidly embraces mediocrity rather than striving to be either a first-rate turn-based system or a first-rate real-time system. Yet I think other aspects of the game's mechanics are often wrongly maligned. For example, we often hear that "magic is overpowered and unbalances everything" during Arcanum play. This simply isn't true-- I regularly play technologist characters in Arcanum, as much as possible simply ignoring the magical side of the game, and each time I play Arcanum I enjoy a highly satifsying game with such an approach. This, then, is in fact a virtue of Arcanum's mechanics: Don't like magic? Don't use it, and have a fine playthrough anyway.
In an important sense, the truest test of a good role-playing game is the extent to which players may comprehensively subvert the intentions of the designers yet have a jolly good time anyway. The most vital wisdom divined by philosophers since the time of Immanuel Kant is: all is interpolation. We see something azure-- yet no object is azure because there is no "azure," there is only light which changes from one frequency to another as it is partially absorbed and partially reflected by some object and then refracts again in our eyeballs. "Azure" describes an experience of ours-- nothing more. All we truly perceive is light, and that one or more objects exist which may have refracted that light is but a theory (albeit a well-supported theory) of ours. Likewise when we read a book, we can never truly know what ideas the writer meant to express-- we can only know what ideas the book inspired in our own thoughts. For example, we may well admire and identify with the character who, to the writer, was unambiguously the scorned villain of the piece, yet enjoy the book all the same without feeling scolded ourselves. More fundamentally, unless we suppose we can communicate via ESP or gnosis then both the medium of words and the act of perceiving those words in a discrete circumstance interpolate the transference of ideas between writer and reader. Three participants-- writer, medium, and reader-- are simultaneously "Author." A less subtle way to express this idea may be: If The Matrix is sufficiently well-constructed, we never become aware of its existence-- but is then the meaning of our lives what we experienced, or instead what we never experienced? Good role-playing games embrace and facilitate this insight in practical ways. Good rpg design encourages player initiative, which is but a another way of saying good rpg design encourages player subversion. Engaging rpg gameplay is interpolation, not revelation. Another way to express this idea may be: many of us (here at a crpg forum) are more likely to enjoy a conversation than a lecture.
The primary weakness of Arcanum is that it does not go far enough in providing the flexible framework which is in fact the fundamental design principle of the game. Consider the vast expanses of nearly empty land on the northern portion of the world map. A better Arcanum (one provided more development time and a larger budget) would have filled that area with several towns and adventure areas completely unrelated to the core Bates-Nasrudin plot for player-characters who wish to ignore such lofty events and prefer instead to simply seek a new, interesting life for themselves in the profound game-world. In most playthroughs of Arcanum, my characters end up as artisans trying to build a fortune using the game's technology-crafting system rather than spending any time at all worrying about charging into the Void to most likely (according to Nasrudin's vagaries in the most commonly-expressed expositions players are likely to encounter) suffer ignominous damnation.
My point, then, is that if we ask "Is game X good?" what we should really be asking instead is: "Is game X good at doing Y? And if game X clearly intends to offer experience Z, how well is that intention expressed and received?" If Fallout intends to offer a somewhat different experience than Arcanum, then comparisons become more difficult, and subjective considerations of player temperment and personal taste must of course enter into such matters.
There's a reason bigwigs at Acme Megagames Inc, with their talk of how "our players prefer designed experiences to unstructured free association" make the big bucks-- they do indeed have their fingers on the pulses of a significant portion of the gaming medium's audience. Furthermore, players who prefer more pervasively designed experiences to the alternative have nothing of which they ought be ashamed. The point of playing games at all is to have fun-- so whether one prefers Fallout to Arcanum or vice-versa is in the end much less important than the fact that both games exist and thereby provide a diversity of options for variegated fun.