If you consent, the evidence can almost certainly be used against you. Florida v. Bostick, 501 U.S. 429 (1991) ("Even when officers have no basis for suspecting a particular individual, they may generally ask questions of that individual, ask to examine the individual's identification, and request consent to search.")
If you refuse consent, it is not clear whether the evidence can be used against you, as we dont' know why the officer is asking to frisk you. A stop-and-frisk must be supported by a reasonable and articulable suspicion that you have just committed or are about to commit a crime, and that you are at that moment armed and dangerous. If they reasonably suspect you have just committed a crime but do not reasonably suspect you are armed and dangerous, the police may stop you, but they may not search you.
That point is worth emphasizing because several other answers are incorrectly assuming otherwise. For one example, in Thomas v. Dillard, 818 F.3d 864 (9th Cir. 2016), the police responded to a report of a domestic violence incident. Based on their reasonable and articulable suspicion that the suspect had committed that crime, the police detained him for an stop and frisk. Because they had could reasonably explain why they thought he had committed a crime, but could not reasonablly explain why they thought he was armed and dangerous, the court said the stop was legal, but the frisk was not:
Whereas the purpose of a Terry stop is to further the interests of crime prevention and detection, a Terry frisk is justified by the concern for the safety of the officer and others in proximity. Accordingly, whereas a Terry stop is justified by reasonable suspicion that criminal activity may be afoot, a frisk of a person for weapons requires reasonable suspicion that a suspect is armed and presently dangerous to the officer or to others. A lawful frisk does not always flow from a justified stop. Rather, each element, the stop and the frisk, must be analyzed separately; the reasonableness of each must be independently determined.
Even then, the search is basically limited to a minimally intrusive patdown to ensure you don't have any weapons on you, and the officer is generally not permitted to actually search inside pockets or the like, though the search may escalate based on what the officer is able to feel during the patdown. Minnesota v. Dickerson, 508 U.S. 366, 375–76 (1993) ("If a police officer lawfully pats down a suspect's outer clothing and feels an object whose contour or mass makes its identity immediately apparent, ... its warrantless seizure would be justified by the same practical considerations that inhere in the plain-view context.")
The refusal to respond generally operates as a refusal to consent. The police are therefore free to conduct whatever search they could have conducted without your consent. If they have a reasonable, articulable suspicion that you're carrying a gun, they may frisk you to see if that's the case. If they have a warrant to search your pockets, they can search your pockets. If they don't have any of that, they need to keep their hands to themselves.