Fly-half Yu Tamura kicked three penalties but Ireland were 12 to 9 up at half-time.
But Japan showed more mettle in the second half. Replacement winger Kenki Fukuoka struck the killer blow, wriggling over the line on the hour mark to score the winning try of a frenetic Pool A clash.
Japan coach Jamie Joseph hinted that the result had not come as a surprise.
“You’ve just got to be careful before the game. You don’t want to come across too arrogant and cocky,” the ex-All Black said, praising his forward pack for holding Ireland in the set-piece.
“We obviously had a lot of belief in our gameplan and in what we wanted to do. We knew how good Ireland were and how strong they were.
“We’re very proud of the players. We’ve been thinking about this game for quite a long time. Obviously Ireland were only thinking about it for the last six or seven days. But for us, it’s been a great performance.”
Ireland captain Rory Best said his team were unsurprised how tough the game was. “Anyone that is utterly shocked hasn’t seen how good they are. We were on the wrong side of the penalty count. But Japan played really, really well. They posed a lot of questions to us and unfortunately we couldn’t come up with the right answers.”
In the earlier game, a first-half hat-trick by hooker Julian Montoya was enough to give Argentina a bonus-point win against Tonga as they beat the spirited Pacific islanders 28-12 at the Hanazono Stadium. (AFP)