Not sure what you mean by "had become acting President and then sworn in as President". Only the Vice President can actually become President. Everyone else on the line of succession can only act as President, not actually become President, according to Article II, Section 1, Clause 6 of the Constitution (and presumably if they were sworn in, it would be as Acting President).
If the Vice President were the designated survivor, and the President's fate is unknown, rather than assume that the President is dead, he (in combination with a majority of the Cabinet) could simply declare the President to be unable to discharge the duties of his office according to the 25th Amendment, section 4. In that case what happens is clear; the Vice President is rightfully Acting President until the President transmits that he is able to discharge the duties of his office again (and possibly have to wait 4 days after that in case the VP and Cabinet wish to counter that declaration). Even if the President reappears, the powers of the Presidency would still rightfully belong to the VP until the President makes this declaration.
If the President and Vice President's fates are both unknown, then it's tricky. Article II, Section 1, Clause 6 of the Constitution provides that when the President and Vice President are both unable to discharge the duties of their offices, Congress provides by law (the Presidential Succession Act) who acts as President. The problem is that unlike for the President, there is no constitutional provision specifying a way to declare the Vice President unable to discharge his duties. So unless the Vice President is known to be dead or the Vice Presidency is vacant, there will be questions about whether someone later in the line of succession can rightfully act as President or not. Assuming that this person can rightfully act as President due to disability of the President and Vice President, Article II, Section 1, Clause 6 clearly states that they act only until the disability is removed (of the President or Vice President). So if the Vice President or President shows up, presumably they take over, though again in the case of the Vice President, there is no constitutional provision specify how to declare that a Vice President is able to discharge the power of his office again.
This problem with lack of a provision to declare that the person is unable to discharge the powers of their office, or that they can discharge the powers of their office again, also exists for everyone further in the line of succession (provided by the Presidential Succession Act), so let's ignore it for now.
If the President pro tempore of the Senate is acting as President, the Speaker of the House becoming able to discharge the powers again doesn't bump the President pro tempore, because section (c)(2) of the Presidential Succession Act specifies that the Speaker or President pro tempore continues to act as President unless the President or Vice President's disability is removed.
If a Cabinet member is acting as President, section (d)(2) of the Presidential Succession Act specifies that they continue to act as President unless someone higher on the list can act, but not counting if a higher Cabinet member's disability is removed. This means that if the Speaker of the House or President pro tempore of the Senate can discharge the powers of their office again, they will bump the Cabinet member currently acting as President, but if another Cabinet member higher on the list can discharge the powers of their office again, they will not bump the Cabinet member currently acting as President. Some scholars believe that the "bumping" provision may be unconstitutional as the Article II, Section 1, Clause 6 of the Constitution says Congress provides by law the single officer who will act as President, and that this officer "shall act accordingly, until the Disability be removed, or a President shall be elected", with no allowance for passing the powers between different officers prior to the removal of disability of the President or Vice President.