I'd actually say that mechanics have improved a fair amount, though not always as much as they should. For example, the last Tales game I played was Tales of the Abyss and I was a bit disappointed that they hadn't done more to build on Tales of Phantasia's battle system, they seemed to have pretty much copy and pasted it. Not that it's a bad system, I just feel like they could have done more to improve it is all.
I'd say that the bigger problem is that some archaic design is still used, like random encounters. There is no reason for any modern JRPG to not have encounters be visible and avoidable (Lufia) and/or completely scripted (Chrono Trigger, which actually does both).
I also think that they have probably become overly commercialized, Final Fantasy in particular seems to have too many character design decisions handed down from upper management/marketing. I'm not sure how bad XIII was but XII added in Vann and Penelo in order to have characters of those stereotypes which apparently are popular with Japanese audiences.
I'd also say that too many JRPGs seem to think that the story they are telling is the most important thing and makes mediocre or bad gameplay irrelevant. I just don't think that the overarching design is as good as it should be. I think technology has played a part in that and I also think that consumers haven't sent a firm message that they aren't willing to put up with it (or the companies aren't listening).
I hate sounding like a broken record but Chrono Trigger is where it is on JRPG lists (at least mine) because it's a concise and deliberate game. The content all seems hand-placed and controlled, making the pace of the game exactly what the developers wanted and making the player's party be at very predictable levels throughout most of the game. The plot is always advancing and advances at a good and consistent pace, without really long pauses between plot events, minor or major. The player is also always rewarded for what they do, usually through plot/world events (such as changing the attitude of the mayor of the town or planting trees which turn into a forest). Really the worst side content in the game is the stuff that doesn't do that, fighting stuff for crafting materials in 10,000BC and the minigames at the millennium fair. It's a fairly short game but I'd rather play a Chrono Trigger than a FFXII that takes at least five (I'd say closer to ten) times as long to do all of the side content.
I just feel like they don't design games with enough attention to detail as they should. I like games to be of a reasonable length but it's about quality before quantity and I feel that too many games sacrifice the former for the latter. If that means that the game is only twenty hours of content instead of one hundred, so be it.
I think part of the thing is is that the SNES games felt like they were moving the genre forward and the later ones like they were really pushing the platform to it's limits. The most recent game that really excited me was the beginning of FFX, up until you land in Besaid.
I need to get around to playing the Persona games, I've heard nothing but good things about them. Actually, Persona 4 sounded like it incorporated visual novel elements into it's story telling.