This survey is perhaps more instructive. It asked both whether respondents were certain of God's existence, and whether they were atheist. Japan tops the charts for uncertainty; but it is actually the lower half of the table for atheism. What does this suggest? A bet-hedging disinclination to commit one way or the other? A reluctance to be rude even to non-existent beings? Or perhaps the monotheistic bias implicit in the question about "God" skewed things somewhat?
The strange thing is that these Japanese friends who aren't at all religious generally nevertheless go to pray at a Shinto shrine at least once a year; they have a Buddhist priest come to pray for the souls of their departed at ordained intervals; and they have a little altar to those same ancestors in their houses. This is apparently a matter of culture and custom, not religion. That's a pretty blurry line in many countries, I suppose, even without translation issues, but it does make it hard to take seriously the kind of worldwide surveys linked above, since their questions must mean such different things to different people.