I am 5'8 and I got a Medium and it was slightly big. Not too bad, but the cuffs were past my knuckles and I have long arms. It doesn't fit tall. I think you should be a large if you want a baggier fit.
"im not a woman but i sure am interested" - Hickles
I'm 6'1 and I have a XL inverse jacket. It's bigger than a standard fit but not a super tall fit. Actually it fits pretty much exactly the same as my XL Oakley Fort jacket. Zipper length is aboout 33.5". Really high quality stuff though.