I'd say the question of flex mostly comes down to personal preference. I skied a 90-flex boot for my first two years and it worked fine, but as I got better and started skiing with a more traditional (forward, driving the shovels of my skis) stance, I found myself wanting a stiffer boot. Now I typically ski 120-130 flex boots, and couldn't go back to a softer boot and still ski like I want to. But then I know some really good skiers with park backgrounds how can rage on 90-110-flex boots.
What skis do you use and would you say you tend to ski more centered / neutral, or with more of a forward, traditional stance? If it's the latter, it might be worth looking into stiffer boots as they can help you drive skis more powerfully (particularly longer, stiffer, and / or heavier skis) and they can provide more support when skiing really fast (since softer boots can "fold up" at speed and buck you forward). Some of the main potential downsides with a stiffer boot that I've noticed are that you might find it too "harsh" (since the boot isn't absorbing impacts as easily) or you might not be able to drive your skis because you can't flex the boot enough. Those are the things I notice, but I'm sure other people have other things that they notice with stiffer / softer boots.
And if you are interested in stiffer boots but don't want to buy a whole new pair, Pulse's ProFlex
attachment is worth a look. I've had good experiences with it so far — the ramp up in stiffness is noticeable, without notably negatively affecting the boot so far.