WP themes usually have the following code in header.php
<link rel="alternate" type="application/rss+xml" title="RSS 2.0" href="<?php bloginfo('rss2_url'); ?>" />
<link rel="alternate" type="text/xml" title="RSS .92" href="<?php bloginfo('rss_url'); ?>" />
<link rel="alternate" type="application/atom+xml" title="Atom 0.3" href="<?php bloginfo('atom_url'); ?>" /
Remove above when you are using the following code
add_theme_support('automatic-feed-links');
This automatically adds the relevant feed links everywhere on the whole site. Standard feed, comments links, category and tag archives, everything as it should be.