Windows has a much larger user base then Mac and therefore it's not as lucrative to make viruses for Mac also because unlike Windows, Mac OS X applications don't share a common registry. Mac OS X applications use individual preference files, thus the types of global configuration changes which enable so much of Windows malware is simply not as feasible on a Mac. Further, root access is needed in order for malware to interact with other programs (i.e. steal passwords, intercept transmissions, etc.).
I'd say like most people that you would be fine without antivirus on your Mac.