An act to regulate the time and manner of administering certain oaths
First law containing a crime
The renewal of the Northwest Ordinance, which authorized the executive to adopt state law within the Northwest Territory
i.e. An Act to provide for the Government of the Territory North-west of the river Ohio (1 Stat. 50). However, as best I can see this gives the Governor of the territory and, initially, judges (later a legislature) the authority to make laws, without specifying any crimes.
First solely criminal law
An Act for the Punishment of Certain Crimes Against the United States (1 Stat. 112)
First law containing a crime, still in effect
Possibly the above. Certainly, this article on U.S. Law Enforcement Abroad: The Constitution and International Law describes (in reference 11) "18 U.S.C. §2381 (1982), adopted originally as Act of Apr. 30, 1790, ch. IX, 1 Stat. 112." The wording of 18 U.S. Code § 2381 does seem to closely mirror that of the 1790 Act:
Whoever, owing allegiance to the United States, levies war against them or adheres to their enemies, giving them aid and comfort within the United States or elsewhere, is guilty of treason and shall suffer death, or shall be imprisoned not less than five years and fined under this title but not less than $10,000; and shall be incapable of holding any office under the United States.
That if any person or persons, owing allegiance to the United States of America, shall levy war against them, or shall adhere to their enemies, giving them aid and comfort within the United States or elsewhere, and shall be thereof convicted, on confession in open court or on the testimony of two witnesses to the same overt act of the treason whereof he or they shall stand indicted, such person or persons shall be adjudged guilty of treason against the United States, and shall suffer death.