Why did it take so long for oxygen to build up in the Earth's atmosphere?

Because there wasn't much dry land, so there wasn't much runoff sediment to bury carbon.
http://www.newscientist.com/article/mg20126882.600-ancient-earth-was-a-barren-waterworld.html