Depending on how baggy or tight the jacket fits you and what look you want you can go a couple of ways fit wise. But the general rule of thumb for most when it comes to color combos is to have one brighter piece and one more mellow piece to balance it out. So look for some muted or darker colors for pants.
At the end of the day though its up to you. If you dig how something looks don't hold back.