The practice of having all SCOTUS cases heard by the whole court (en banc) rather than by individual justices or panels is nowhere specified in the US Constitution. Nor is it mentioned by The Federalist which is often a good source on the intentions of the Framers.
It seems to have been adopted by the first session of the court with little discussion -- at least none that has come down to us.
I strongly suspect that English models were in the minds of those early court members, but I do not know exactly which models. The practice of the various state Supreme courts may also have been influential. This practice seems to have been confirmed and definitely settled by Chief Justice John Marshall, who did so much to define the Court's procedure. He also strongly encouraged the court to issue a single option in each case, discouraging both concurring and dissenting opinions. He felt that the Court should always speak with a single voice. During the first fifteen years of his tenure, he himself wrote close tom 90% of the court opinions, according to an academic study of the Marshal Court I read about a year ago.
It is worth recalling that in the early years there were not many SC cases, and there was no need to split up the work of the court. It is also worth remembering that at that tine there were no separate appeals courts. Rather, each Justice "rode circuit" holding court in several different cities, in each sitting with one or two District Court judges as a panel court. Thus when a case reached the SC on appeal, it had already been passed on by a single Justice. The justices, through the end of the Marshall Court in the 1830s spent more than half of each year on circuit.
Addition: One should also remember thst from the start through the 1830s the Justices lived in boarding houses during the Court term, and in most terms most or all of the Justices lived in the same house, where the work of the Court was discussed over dinner and in the evenings.
None of which Really answers "why do we do it this way?" I do not know of any document that first laid out this practice, much less gave reasons for it.