The seven-time Cincinnati champion, who turned 38 last week, was unable to make an impression as he played only his second match since losing the Wimbledon final to Novak Djokovic.
Rublev, a 21-year-old Russian qualifier who missed months last season with injury, was shell-shocked at his win over the 20-time Grand Slam champion.
“It’s an amazing feeling when you play such a legend as Roger,” he said. “At least 99 per cent of the crowd was supporting him.
“Today I was just trying to do my best. I told myself I need to play every point to the end. No matter what the situation, I kept fighting.”
Federer went down to the world number 70 in barely an hour, losing for only the 10th time here after winning 47 matches.
World number one Novak Djokovic glided into the quarter-finals by dismissing Pablo Carreno Busta 6-3, 6-4.
The Serbian star is the only top-five seed remaining in the US Open warm-up event.
Women’s top seed Ashleigh Barty held off Anett Kontaveit as the Estonian served for victory before the top seed scraped a 4-6, 7-5, 7-5 triumph into the WTA event quarter-finals.
The win keeps the Australian in the chase for a possible return to the WTA number one ranking, which she lost to Naomi Osaka on the eve of this US Open build-up event.
Barty needs to reach Sunday’s final in order to regain the ranking summit. Barty will play on Friday against Maria Sakkari of Greece, a 6-7 (4/7), 6-4, 6-4 winner over ninth seed Aryna Sabalenka. (AFP)