Your question is honestly pretty long and complex. For the first part, i.e. why were the §242 charges dropped... (without 2nd guessing the prosecution here), that's actually (by far) the most common outcome, statically speaking. Here are some (nation-wide) stats; basically §242 charges are only pursued in about 3% of the cases referred to the DOJ (by the FBI):
Charges under § 242 are most often used in cases involving excessive use of force by police. Yet they are also relevant in cases broader than police, such as cases of excessive violence by a prison guard, and cases where an individual acting "under the color of law" commits a robbery, theft, or sexual assault. Most § 242 cases are first investigated by Federal Bureau of Investigation (FBI) agents or another federal agency, then referred to U.S. attorneys within the Department of Justice who have discretion over whether to prosecute. [...]
Although charges under § 242 are rare, they often form the legal basis in cases where police use of force is in question. For instance, after the police officers involved in the arrest of Rodney King in 1991 were acquitted by a Los Angeles County jury, federal prosecutors charged and ultimately convicted the officers with the federal crime of depriving Mr. King of his civil rights while acting "under the color of law" (i.e. in their capacity as police officers). [...]
In fact, in the twenty-year period between 1990 and 2019, federal prosecutors filed § 242 charges about 41 times per year on average, with as few as 19 times (2005) and as many as 67 times in one year.
Historically very few referrals under § 242 are actually prosecuted by the Department of Justice. Between 1990 and 2006, the percent of referrals that federal attorneys prosecuted never rose above three percent. A driving force behind the declination rate appears to be not the standards used by federal prosecutors for filing prosecutions, but the standards used by the FBI in deciding whether to refer a case to federal prosecutors. Between 2006 and 2019, as the number of referrals declined, the percent of referrals that were prosecuted increased. Yet this did not result in a marked increase in the total number of prosecutions each year. The highest year on record, 2008, saw 67 total § 242 prosecutions, but no year since has come close.
As discussed in another article successfully prosecuting such 242 cases is hard because of the Screws standard.
For example, the Kent State shooting criminal charges (stemming from National Guard firing on student protesters/rioters during the Vietnam-war era) were dismissed by the judge mid-trial. In a summary of that case:
During the course of the protests, the demonstrators
ignored an order to disperse, and the national guardsmen who had
been sent to maintain order began to advance with bayonets. A melee
ensued with many students heaving rocks and insults at the guardsmen.
In the chaos, the guardsmen’s line broke, and the guardsmen
were suddenly surrounded by students. Without an order to fire,
twenty-nine guardsmen shot at least fifty-four rounds into the crowd
killing four students and wounding nine.
The issue presented in federal court [in Shafer] was whether the guardsmen had the
requisite intent to willfully deprive the students of their constitutional rights. Based upon
both intent and federalism concerns, the court held that the officers
had not acted with the requisite specific intent.
An important facet of Shafer is the way the opinion emphasized
that a federal prosecutor needs to show a prior animus of the officer
toward the victim in order to prove a § 242 violation. The court
concluded that acting out of fear, anger, or frustration did not, in and
of itself, amount to the specific intent to violate constitutional rights.
This interpretation of specific intent requires the officer to formulate a
bad purpose prior to the deprivation of the right. In turn, the court
tacitly rejects any use of a reckless disregard standard, keeping the
intent requirement narrow.
In more general academic terms, from a CRS report summarizing the issue with 242:
By its text, Section 242 applies only to violations that are committed “willfully.” The Supreme Court
stringently construed the willfulness standard in the 1945 case Screws v. United States (the main opinion
in Screws was joined by only four justices, but binding opinions of the Supreme Court have since adopted
its analysis). In Screws, a defendant convicted of violating the statute now codified as Section 242 argued
that the law was void for vagueness—that is, it violated the Fifth Amendment’s Due Process Clause
because it did not give potential defendants clear notice of the conduct it proscribed. The Supreme Court
rejected that argument by interpreting “willfully” to require the government to show that a defendant
acted with a “specific intent to deprive a person” of constitutional rights or with “open defiance or in
reckless disregard of a constitutional requirement.”
The Screws plurality recognized that its interpretation of Section 242 differed from the usual mental state
standard in criminal cases. To obtain a conviction for a crime, the plurality explained, the prosecution
usually must show that the defendant intentionally performed some action, and the action was prohibited
by law; but prosecutors ordinarily need not show that the defendant knew the conduct at issue was illegal
or specifically intended to violate the law. However, Section 242 imposes criminal liability for
constitutional violations, and courts examining the “broad and fluid definitions of due process” may
interpret the Constitution to protect rights not expressly enumerated in the Constitution or prior court
decisions. [...]
Lower federal courts vary in how they apply the willfulness analysis in Screws. The U.S. Court of
Appeals for the Fifth Circuit has held that “willfully” means “that the act was committed voluntarily and
purposely with the specific intent to do something the law forbids. That is to say, with a bad purpose
either to disobey or to disregard the law.” By contrast, the U.S. Court of Appeals for the Third Circuit,
while remarking that “Screws is not a model of clarity,” upheld a jury instruction stating both that “an act
is done willfully if it is done voluntarily and intentionally, and with a specific intent to do something the
law forbids,” and that the jury could “find that a defendant acted with the required specific intent even if
you find that he had no real familiarity with the Constitution or with the particular constitutional right
involved.” Overall, however, the Supreme Court’s interpretation of the willfulness requirement has
resulted in what some view as a significant hurdle to bringing Section 242 claims.
I think the highlighted sentence says it all: not only is the Screws bar pretty high, but it's also been interpreted somewhat differently by various courts... That probably gives rise to even more prosecutorial discretion in such cases...
As for why they didn't go for the manslaughter charges [instead], IANAL, but the manslaughter text you linked to is clearly inapplicable in case the shooting is legally justified. "Manslaughter is independent of intent" as you say, but it's not without any other burden of proof (besides the killing):
(a)Manslaughter is the unlawful killing of a human being without malice.
It is of two kinds:
Voluntary—Upon a sudden quarrel or heat of passion.
Involuntary—In the commission of an unlawful act not amounting to a felony, or in the commission in an unlawful manner, or without due caution and circumspection, of a lawful act which might produce death.
Just imagine what it would mean if any shooting is "an unlawful killing" by definition. Then it's simply up to a prosecutor to decide who goes to prison or not when shooting someone (i.e. prosecution = guaranteed conviction) under such an assumption.
The federal wording is pretty arcane, but in some states simpler wording is used. E.g. the Minnesota policewoman who warned the suspect of imminent tasing but then shot her firearm (killing the suspect) was indeed charged with 2nd deg. manslaughter, but while that does not require intent, it still requires proof of "culpable negligence" under Minnesota law. In fact, in this case, the policewoman's own word show that she [probably] did not mean to shoot, which ironically probably precludes a defense of legally justified shooting.
Furthermore, you put in bold that the officer need to have "exhausted other options", but that wording doesn't actually appear as such in the federal law. As phoog points out, that's in the DC law.
In the DC code you quoted the "trier of fact shall consider" bit. I don't know who that is supposed to be exactly. Judge and jury are generally included, but it's not clear to me if the prosecution is held to the same standard. (Might make a good separate question.) Also " The totality of the circumstances, which shall include [...]" doesn't seem to preclude other considerations, not spelled out. So while a deadly weapon in the hands of the subject is clearly a spelled out consideration, some suspect strangling someone else with their bare hands, or more obviously, about to throw somebody off a rooftop, is not spelled out as such, but clearly would normally be considered a justifiable reason to shoot.
I suppose the most debatable part here is whether the officer should have tried "using non-deadly force prior to the use of deadly force" in this case.
From Law & Crime, the officer's justification was that he
subjectively believed shots had been fired, that officers were down, and that officers were being overrun. Most of that was true. Byrd’s assessment that what he was hearing from fellow officers on his police radio was either true or close to the truth was likely reasonable as well given the totality of the circumstances of the Jan. 6th breach of the Capitol.
I guess that was considered enough by the prosecution. He also said he did engage in verbal de-escalation measures (another consideration in the DC statue you've linked):
I had been yelling and screaming as loud as I was, “Please stop. Get back. Get back. Stop.” We had our weapons drawn.
He also said that he didn't know whether she was armed or not, but that under the circumstances that didn't matter to him. So, yeah, that's what we know. As it turns out, the article's author repeats what I observed above, that possession of a weapon is a factor, but not dispositive:
Byrd is correct on the law: nothing in the justification statute cited above requires the person being shot (here, Babbitt) to be armed with a weapon. As the statute itself points out, it is helpful to the analysis: if Babbitt hypothetically had displayed a weapon when she climbed through the window, it would have been easier to justify Byrd’s actions. But the lack of a weapon is not dispositive. The law is best restated like this: the police can meet “threat of serious bodily injury or death” with deadly force. Hundreds upon hundreds of angry people crashing into the halls of Congress — some of them wearing tactical gear, some allegedly carrying or beating officers with deadly weapons, and some allegedly attacking officers with chemical irritants — can hypothetically cause death or serious bodily injury by the sheer force of their numbers. An angry mob of people screaming and shouting about an election can still present a serious threat if they outnumber the assembled law enforcement forces and resort to destructive violence. Though many conservatives — including Babbitt’s husband — have drawn considerable attention to Babbitt’s lack of a weapon, it is not completely dispositive of the threat posed by the situation in its entirety. It weighs against Byrd but is only part of the “totality of the circumstances” analysis contained within the statute.
In the officer's stated judgement:
there was imminent threats and danger to the members of Congress
I guess the prosecution bought that, at least to the extent that they thought they could not prove otherwise beyond a reasonable doubt.
Of some interest, the CRS document linked in answer says that indeed there are agency-specific policies that matter, but that generally speaking there's nothing in Federal law itself besides the 4th amendment and its case law that governs use of [deadly] force:
At the federal level, there is no generally applicable statute that governs the use of force by law
enforcement. [...]
That said, the Supreme Court has announced some general principles that lower courts employ to guide
their assessment of the reasonableness of both lethal and other uses of force by police officers. First,
reasonableness is judged “from the perspective of a reasonable officer on the scene, rather than with the
20/20 vision of hindsight.” In other words, the calculus must allow “for the fact that police officers are
often forced to make split-second judgments—in circumstances that are tense, uncertain, and rapidly
evolving[.]” Second, the reasonableness inquiry is an objective one; that is, the appropriateness of a use of
force is gauged by what is “‘objectively reasonable’ in light of the facts and circumstances confronting”
an officer. As such, the officer’s “underlying intent or motivation” is irrelevant. The Court has also noted
several factors to be included in the assessment of the reasonableness of a particular use of force: (1) “the
severity of the crime at issue,” (2) “whether the suspect poses an immediate threat to the safety of the
officers or others,” and (3) whether the suspect “is actively resisting arrest or attempting to evade arrest
by flight.” And with respect to lethal force specifically, the Court has said that use of such force is
permissible where “the officer has probable cause to believe that the suspect poses a threat of serious
physical harm, either to the officer or to others[.]” Thus, “if the suspect threatens the officer with a
weapon or there is probable cause to believe that he has committed a crime involving the infliction or
threatened infliction of serious physical harm, deadly force may be used if necessary to prevent escape,
and if, where feasible, some warning has been given.”
You'll be perhaps pleased to read above that (officer's) intent indeed does not matter, but otherwise it's down to a fairly flexible standard on deadly force that “the officer has probable cause to believe that the suspect poses a threat of serious physical harm, either to the officer or to others”.
Slightly off topic, but on that angle, for example, the DOE regulations (a kind of policy) allow use of deadly force if some kinds of nuclear material are being stolen--presumably the implication being that their falling into the wrong hands presents that kind of threat of physical harm.
As for "judicial review", I doubt it's possible to have the prosecutors' decision(s) reviewed (anytime soon), but as e.g. in the Kent State case and others, it's possible to bring separate civil lawsuits in such cases. Quoting from the very first source in my answer:
Although federal prosecutors can bring criminal charges under § 242, individuals can also bring civil lawsuits against the government under 42 U.S.C. 1983, which similarly covers deprivation of civil rights by individuals acting under the color of law
Sometimes, e.g. in Kent State, these civil lawsuits had different outcomes than the state prosecution in criminal track.
Also of some note, in the Kent case, the DOJ more recently (2012) declined to reopen the case, so such request can be made in light of new evidence.
