Neither atheists nor agnostics believe in gods. Whether or not this lack of belief is itself a belief has long been debated and forms part of the disagreement between atheists and agnostics. The atheist confidently proclaims "God does not exist" while the agnostic admits uncertainty and believes nothing (for now). On what do atheists base their claim of nonexistence? Can atheists prove God does not exist? No. They cannot. In fact, it is impossible to prove any nonexistence claim. The reason why is well illustrated by James Randi's 'flying reindeer' experiment.
In American culture there exists a popular story of a man traveling the world one night every year delivering presents to children. This man makes the trip in a sleigh pulled by 8 flying reindeer. Because they lack wings and there is no other known way by which they could fly, flying reindeer appear to violate the laws of nature. What sort of experiment can I do to prove "Reindeer can't fly"? Let's take a bunch of reindeer to the top of a tall building and start pushing them off. How many reindeer must be pushed and killed before we consider the statement proven? After seeing the first few fall to their death I'm willing to accept the postulate without further slaughter but I would not have absolute proof. The possibility still remains that the reindeer we tested were freaks in their lack of ability to fly. Or, they could fly but either chose not to or were prevented from doing so by the features of the experiment like the building not being tall enough or it not being Christmas Eve and hitched to Santa's sleigh. Every reindeer on the planet could fail the test under any number of varying conditions without eliminating all uncertainty. It will always be possible to come up with explanations after the fact as to why the reindeer didn't fly while keeping the central tenet. Eventually as the failures build one must start doubting the entire set of hypotheses and wonder how the idea got started if it is so difficult to find evidence for it. It becomes easier to believe that someone in the past either imagined seeing a reindeer fly or made it up. Regardless, it caught the imagination or met the needs and desires of people and the story was passed on to subsequent generations. So I reject flying reindeer without rigorous proof.
I also cannot prove that ESP and UFOs don't exist, that a god doesn't interfere with the universe, nor even that there is no place on Earth where conservation of energy doesn't hold. There is enough evidence consistent and none inconsistent with these statements in my opinion to consider each highly probable. Therefore, until there is evidence to the contrary I'm willing to accept them as true while fully acknowledging the lack of absolute certainty. The believer, on the other hand, must accept flying reindeer without evidence, i.e. it becomes blind faith. The agnostic is left in the quandary of having to allow for the possibility of anything existing regardless of how outrageous it may be.
I feel justified in accepting or rejecting these claims because they all fall in the realm of the knowable. The reverse of each claim could easily be proven if true. It takes only one reindeer pulling out of its nose dive and soaring through the sky to prove reindeer can fly; only one 10-mile high cross hovering over Ithaca with the words "Believe or Burn!" to convert even the most dedicated atheist. Even without such extraordinary events some level of proof can be obtained through the many claims of religion (special creation of humans, world-wide flood, prophecy, etc). If any of these claims could withstand scientific scrutiny and not be more likely explained by delusion or deception, theists would have a large leg up on atheists. The level of scrutiny, however, is admittedly very high and some theists do cry foul. They would have us judge their claims based on credence of character rather than the weight of evidence. But by their very nature divine interventions violate the laws of physics. Therefore, claims of interventions carry the same demand for evidence as claims of violations of the conservation of energy. Theists' failure to verify or substantiate their claims and the clear disproof of many of them lead people to start questioning the whole theist paradigm. How could a god capable of creating the universe out of nothing be so incompetent when it comes to making its presence known? And why is the creator's handiwork so obviously absent from its creation? Atheists consider the failures of religion to be so complete as to justify rejecting the claim that a deity is influencing the workings of the universe and demanding our obedience and worship. This decision is not made with 100% certainty but rather a certainty near that for accepting the conservation of energy. So until I'm shown a perpetual motion machine that works only when a priest is praying nearby I'm confident a godless universe is so highly probable as to be quite certainly true. And, I'm willing to stake my imaginary, immortal soul on it.
On the other hand when asked how the universe came to be, the most intellectually honest answer is "I don't know" because there is absolutely no evidence upon which to make a judgement. There is no test that can prove, disprove, or even distinguish the different proposed origins. Cosmology can follow the universe back in time to a fraction of a second after creation but might never reach the actual instant of creation. Therefore, to claim as true an origin with or without a god is to step into the unknowable and to base one's beliefs on faith instead of evidence. The First Cause argument works its way back to creation and then defines God as "that which created the universe." One could just as easily define this as Uncle George or Proposition 3 of the Wilson Cosmological Principle. As such the First Cause god is purely definitional and meaningless. Likewise, a god that creates the universe and then sits around but never interferes -- a roi faineant, a do-nothing king -- is meaningless because it is indistinguishable from the First Cause god. The agnostic holds this unknowable god as the hole in the atheist's argument. While a roi faineant is completely consistent with all the evidence (or lack of it), this is not the god in which unbelief defines the atheist. Disbelief in this god is an extra step made on faith that the atheist need not make.
Atheists and agnostics differ in where they draw the line between probable but uncertain and sufficiently probable to be very certain. Because the atheist's position can never be proven, this division is unavoidable. For the theist, however, the two may as well be one since both demand the theist provide evidence for God's existence. The burden of proof is clearly on the theist.