unless its windy and the mic is interfering with the stability of your camera, there is no reason to not always have a mic attached. all those times you think you don't need the audio, well I bet you will find a use for it later and be kicking yourself when you dont have it.
People overemphasize the benefit of on-camera external mics because mic'ing from on-camera is an inherently poor place to mic...well anything. Proper mic placement will be far more effective than using a nicer microphone. The main advantage I see to them is to narrow the pattern (e.g. if you're filming by a river but don't want the rapids to interfere with dialoge), but aside from changing the pattern and having a different EQ response, it won't have much of a positive effect. And if you're using a glidecam, it could be a complete tradeoff, at which point you have to ask yourself if its worth it.
That's not to say you should just ignore audio, since its role is incredibly important. I don't use on-camera external mics because I record sound separately and superimpose in post. Given the choice between using the built-in mic v.s. using a RODE Videomic Pro, obviously the RODE is a better option of the two; my point is simply that it isn't optimal, contrary to what many will tell you.
It isn't difficult, nor does it require a boom. I use a little handheld condenser recorder and in my spare time I record things like people taking off, grinding, slashing, rain, etc. Then in post I have a library of sounds I can use for any particular edit. I do it for the Boys Club edits and those are as half-assed as they come.
I hope it doesn't seem like im stealing the answer from you @pandysloo but i have passed experience with this if the audio for a clip isn't good, what i would do is find the audio that fits the best and slow it up or speed it up to mach the rail. However if you change the speed to much it sounds shitty.