I own a patent on a text search and retrieval method. The examiner had cited a nebulous concept as prior art, but my attorney was able to discard that, and the patent was issued. I'm now concerned whether that nebulous concept could be used as a defense by an infringer.
You can ask your attorney about the ground the prior art was overcome or look up the case in Public PAIR. All of the written back-and-forth between the attorney and the examiner are public record. One key is on what grounds did the reference get defeated. It might have been non-analogous art, not actually contained one or more elements of a claim or been non-enabled (a good bet if it was nebulous). It is also possible that the attorney amended to get around it.
One thing to keep in mind is the phase “it’s good for what it teaches”. A seemingly nebulous reference can still be a teaching, suggestion or motivation to combine references. The references must be enabled but the hint to combine might not be.
It is the case that some newly unearthed prior art might be better at getting your patent invalidated but the courts are not giving much deference to the USPTO these days. A court or PRAB board looks at it with fresh eyes.
Technically, a patent is presumed valid but it is now a very weak presumption so something you got past in examination can still come back in the hands an actual adversary and be convincing to a court. You might have dependent claims that are narrower that survive attack.
An infringer can obviously go to court and say “This patent shouldn’t have been issued because of reasons”, and then the court would decide if this is true.
You have the advantage that the patent examiner already knew about this nebulous idea and decided it didn’t matter, but the court can overrule that.
The real question is whether this idea does constitute prior art in the opinion of the court, and that would exactly depend on the idea. Would it be obvious to me, knowing the area of this patent, AND the nebulous idea, to come up with this patent?
But the principle is: If your lawyer convinced the patent examiner that this idea doesn’t matter, then it is more likely that they can convince the court as well. An infringer has likely a better chance to invalidate the patent with things that the patent examiner has never seen.