People who claim "I don't believe in labels" tend to strike me as rather pretentious. Generally it seems to me that this is a disingenuous attempt to avoid associations with an idea or categorization that is unpopular, rather than standing up an owning it.
In this vein, I find the speech by Sam Harris that Andrew Sullivan is quoting rather unimpressive:
Attaching a label to something carries real liabilities, especially if the thing you are naming isn't really a thing at all. And atheism, I would argue, is not a thing. It is not a philosophy, just as "non-racism" is not one. Atheism is not a worldview—and yet most people imagine it to be one and attack it as such. We who do not believe in God are collaborating in this misunderstanding by consenting to be named and by even naming ourselves.
Ah, but atheism is a thing.
It is a category.
While atheism is not a worldview or a philosophy, it is certainly a legitimate category of worldviews or philosophies. Atheism is a category of beliefs that share one common element: the denial of the existence of God. Not labelling "atheists" as such is a rather Orwellian way of trying to define the terms of the debate.
Certainly there are other categories comparable to atheism that, while not a philosophy or worldview, nonetheless do exist within clearly recognizable borders.
Monotheism and polytheism come to mind. Sam Harris himself offers one: non-racism. For that matter, fatalism, belief in reincarnation, pragmatism, and a hot of other things can also be called categories, even if they do not represent the entirety of a worldview or philosophy.
I am not sure where this aversion to naming one's self came from. But unlike Andrew, I find it intellectually dishonest to eschew labels (although it is perfectly sound to be opposed to a particular taxonomy of labels, e.g. the current understaning of "conservative" and "liberal").
That is all.