I used to do this same thing many, many years ago. It is scary, as you sometimes feel like you're going to fall off the trampoline, and sometimes you do. The trick is to jump straight up and don't start flipping until you're near the apex.
To practice this, jump straight up and then rotate backward and get really super comfortable falling on your back. Practice the fall to back and spring up, again and again. Make sure you don't start rotating to your back until you're at or near your apex. This will help transition into a backflip.
You will essentially be re-teaching yourself to backflip. Before you know it, you'll be backflipping off everything...rope swings, ski jumps, the toilet...