The hospital got the idea from a similar 'baby hatch' in Germany
The child was said to be old enough to tell medics at the hospital in southern Japan that his father had left him.
The drop-off at Jikei Hospital, which opened last week, has been criticised by some, including the government.
The Catholic-run hospital has said it is aimed at helping new mothers who would otherwise resort to abortion.
Abortion rates are relatively high in Japan, while adoptions have traditionally been rare.
Police in the city of Kumamoto said the child was left at the "baby hatch" on the same day it opened.
Japanese media said the boy was aged between three and four. He was in good health and able to tell hospital staff his name and the fact that his father had dropped him there.
The hospital would not comment.
The report provoked a strong reaction from Chief Cabinet Secretary Yasuhisa Shiozaki.
"Parents have the obligation to raise their children. We find this extremely regrettable," he said.
Jikei Hospital announced its plans for a "baby hatch" at the end of last year after learning of a similar idea in Germany.
The idea is that a woman can place a newborn in a small window in an outside wall, which opens on to an incubator bed - and sets off an alarm bell to alert staff.
Hospital director Taiji Hasuda said at the time he hoped it would lead to a reduction in abortions.
But the plans were heavily criticised by government officials, including Prime Minister Shinzo Abe who said parents must bring up children themselves.