united-states
This answer is based upon general principles of criminal procedure in the United States which are quite similar in most states, although not necessarily exactly identical. The answer is likely to be different in other jurisdictions.
If inadmissible evidence is offered by the prosecution and admitted at trial, this is a ground upon which the defense can move for a mistrial (which due to double jeopardy amounts to an acquittal), or upon which the defendant can appeal the conviction of the jury, if convicted.
Acquittals of a defendant at trial cannot be appealed by the prosecution.
But, to appeal, the error in admitting inadmissible evidence must be contemporaneously objected to by the defendant's lawyer (or the defendant if the defendant is not represented by counsel), or the admission of the evidence must be "plain error" (which is very rarely met on evidentiary issues). If the defendant's lawyer doesn't promptly say "I object" that ground for contesting a conviction is usually lost.
Even then, the standard of review on appeal is whether the judge abused the judge's discretion in admitting the evidence, not whether the appellate court would have ruled the same way if presented with that evidentiary question.
Also, even if there is an error, a conviction on a particular count will not be overruled if the error was "harmless", which is to say that there is a reasonable possibility that admitting the inadmissible evidence caused the defendant to be convicted of that count. Often, if the evidence is overwhelming, or the inadmissible evidence wasn't that prejudicial, a conviction will be affirmed notwithstanding the admission of inadmissible evidence.
Sometimes harmless error is evaluated considering all of the errors at trial as a whole, rather than individually, in addition to all of the other evidence admitted at trial.
If an evidence issue can be foreseen and is central to the case (e.g. suppression of evidence of possession of drugs in a drug possession case), the issue of the admissibility of the evidence will often be resolved in a pre-trial hearing and subject to appeal then, prior to trial, rather than being resolved in the trial itself where the prosecution has no right to appeal, and the defendant risks conviction if the ruling goes against the defendant.
If the conviction is overturned on appeal, the usual remedy is to remand the case to the same judge to conduct a new trial with a new jury, in a manner consistent with the appellate court's rulings.