Civil litigation involves general causes of action that are available to anyone, including both private parties and the government. Civil courts are designed primarily to provide restitution to the injured party.
Criminal courts exist for governments to exercise their police powers: specific, limited authority held only by the government. The principal purposes of criminal law are...well, they're debated, but broadly speaking, it's to punish and/or rehabilitate the criminal and deter future criminals.
Because criminal conviction can result in jail time and even the death penalty, there are more stringent procedural protections accorded to criminal defendants than there are to civil defendants. So when the government's goal is to recover damages, it's easier for them to use the less burdensoome civil procedures, just the same as anyone else would.
Let me give you an example. Someone steals your identity and runs up $100,000 on your credit card.
You call the police, and they find someone they think is the guy. To convict him, the police must convince an entire jury panel that he did it, beyond a reasonable doubt--a high standard. He pleads the fifth, and without his testimony, the police may not succeed. If they do, he will be sent to jail, and he may also be ordered to give part or all of your money back. If the case is weak, however, the police may not want to spend their limited time on it--and that's their call, not yours. (This also applies to government agencies; only law enforcement can bring a criminal case, not any government agency.)
However, you can also file a civil lawsuit. In that lawsuit, depending on the jurisdiction, you may only need to convince some of the jury--civil verdicts don't always have to be unanimous. You may even just face a judge, with no jury. And the legal standard is a "preponderance of the evidence," which in layman's terms just means "more likely than not"--a much easier thing to prove. Because the Fifth Amendment doesn't apply to civil litigation, you may even be able to argue before the jury that the defendant's refusal to testify suggests he's guilty.
In summary: civil lawsuits use different rules and procedures, which may make it easier to recover money (or get other civil relief, such as an injunction) in cases where that's the goal. These courts are open to anyone, including the government. But if the government wants to use its special police powers to put someone in jail or get other criminal relief, they have to use the stricter criminal rules and procedures.