There is a subtle trick!
To show off your animated GIFs you must upload them to WordPress using the add media tool (see the documentation on uploading images). A subtle trick in doing this is knowing that when you upload images, WordPress automatically makes several different copies, in different sizes, for use in your themes (e.g. a thumbnail a small sized version).
The problem with animated GIFs is that copies WordPress makes loses all of the frames (and hence animation) from for file. Have no fears! The thing you have to make sure you do is to select the full sized version when you get the option, you want to insert the original file. For this reason, you want to make sure your source GIF is not too wide to fit in the space your theme allows for your posts- 500px is a safe bet if you are unsure.
This screencast will walk you through the steps.