Do you have to believe in God to be an atheist?

Yes. The word "Atheist" comes from the word "theist" or a believer in god. The addition of the prefix "a" specifies singularity. So an atheist is someone who believes in only one god. A nontheist is a nonbeliever in god, while a moretheist is a believer in multiple gods. Sometheists believe in a tightly restricted pantheon of a few gods and uhtheists are unsure about the existence of God.

There are multiple examples of this formulation in modern English. For example, a person who is amoral follows only one ethical principle (He's amoral; all he cares about is his reputation). A person who is asexual only has sex with one person at a time (That makes him asexual; he doesn't want people to think he's sleazy). In medicine, a person is asymptomatic if they have only one symptom of a disease (Given his asexual, amoral nature, the one symptom of herpes is more easily explained in other ways).

There are several corruptions of this formation. For example, an advertisement is a single vertisement, or an inducement to buy one specific thing. The name Alexander comes from al-Exander the Great, who modified his name to emphasize his uniqueness. Similar modifications can be seen in the Muslim world (al-Nassir, for example).