The case is Bivens v. Six Unknown Fed. Narcotics Agents, 403 U.S. 388 (1971).
Liability under Bivens applies to federal agents who violate a direct constitutional right in a similar way that state officials, or others acting "under color of any statute, ordinance, regulation, custom, or usage, of any State or Territory" may be sued under 42 USC § 1983 for any action that causes "the deprivation of any rights, privileges, or immunities secured by the Constitution and laws". Section 1983 does not apply to persons acting under federal laws, and thus Bivens becomes a counterpart.
It is true that Bivens applies directly to the Fourth Amendment. But it quotes favorably Bell v. Hood, 327 U. S. 678 (1946), specifically the passages:
where federally protected rights have been invaded, it has been the rule from the beginning that courts will be alert to adjust their remedies so as to grant the necessary relief [at page 684]
and
it is . . . well settled that, where legal rights have been invaded, and a federal statute provides for a general right to sue for such invasion, federal courts may use any available remedy to make good the wrong done. [also at page 684]
Neither of these passages singles out the fourth amendment. Nor does the logic of Bivens depend on anything specific to the Fourth amendment, as opposed to the Fifth, the Sixth, or the First.
I don't know if a case has been brought under Bivens alleging violation of these or other federal constitutional provisions granting specific rights to individuals. The logic of Bivens would require a firm and clear grant of rights, but, for example the Due Process Clause of the fifth is such a grant.
If the courts were to hold that a particular claim could not be brought under Bivens because the Supreme Court has not authorized such an extension of Bivens, then Bivens itself describes an alternate procedure. The opinion points out that:
[at Pages 390-391] petitioner may obtain money damages to redress invasion of these rights only by an action in tort, under state law, in the state courts. In this scheme, the Fourth Amendment would serve merely to limit the extent to which the agents could defend the state law tort suit by asserting that their actions were a valid exercise of federal power: if the agents were shown to have violated the Fourth Amendment, such a defense would be lost to them, and they would stand before the state law merely as private individuals. Candidly admitting that it is the policy of the Department of Justice to remove all such suits from the state to the federal courts for decision, respondents nevertheless urge that we uphold dismissal of petitioner's complaint in federal court and remit him to filing an action in the state courts in order that the case may properly be removed to the federal court for decision on the basis of state law.
This procedure the Court rejected in Bivens as unneeded, but it did not say that such a procedure could not be followed.
I would think that the logic of Bivensshould extend to the invasion, by federal officials (other than judicial officers), of any right explicitly and specifically granted to individuals under the constitution. But one can never tell how the courts will react when asked to extend a previous decision.
The Bivens did not createa cause of action, or at least it did not purport to do so. It simply held that the longstanding rights grantef under the Fourth Amendment could result in money damages even n the absence of a specific statute providing for such damages. The Bivens Court wrote:
[at Page 403 U. S. 402] The contention that the federal courts are powerless to accord a litigant damage for a claimed invasion of his federal constitutional rights until Congress explicitly authorizes the remedy cannot rest on the notion that the decision to grant compensatory relief involves a resolution of policy considerations not susceptible of judicial discernment. Thus, in suits for damages based on violations of federal statutes lacking any express authorization of a damage remedy, this Court has authorized such relief where, in its view, damages are necessary to effectuate the congressional policy underpinning the substantive provisions of the statute. J. I. Case Co. v. Borak, 377 U. S. 426 (1964); Tunstall v. Brotherhood of Locomotive Firemen & Enginemen, 323 U. S. 210, 323 U. S. 213 (1944). Cf. Wyandotte Transportation Co. v. United States, 389 U. S. 191, 389 U. S. 201-204 (1967)
The OP wrote in a comment:
What's the point of Bivens if causes of action still exist for amendments like the 1st Amendment, that are not part of Bivens? What's the difference between suing somebody under the 4th amendment through Bivens, and suing somebody under the 1st amendment?
This is merely a matter of terminology. If the US Supreme Court now recognizes a right to sue for money damages directly under the First Amendment, that will be called "an extension of Bivens". If it rules that such a suit would need a new law from Congress to authorize it (as Sec 1983 authorizes suits against state officials), that would be cal;led "refusing to extend Bivens. All suits for damages directly under a constitutional provision where there is no specific federal law authorizing money damages are now called Bivens suits, because that is the case that specifically held that such damages were permitted without such a law.