When you refer to customs, that necessarily denotes travel to a foreign county, such that each county will have their own laws, rules, and regulations that govern these issues. It is more than likely that if you refuse to answer the questions of customs officials in ANY country, you will be denied admittance. The same is true if you refuse or balk at being searched (personally or your possessions), and keep in mind that this is without reasonable suspicion or probable cause. The best thing to do is to answer the questions honestly and accurately, but also as narrowly as possible to completely answer. Trying to argue with them will only send up red flags and you will be there longer. Remember it is a privilege, not a right, to enter a sovereign nation of which you are not a citizen.
For example, in the U.S., customs reserves the right to detain for questioning, search you, your car, your children, your bags, packages, purse/wallet, or any other travel item with full legal authority to do sol they can even examine your electronics (content and hardware). You place your stuff on the exam station and open it. (After the exam is completed, you will be asked to repack and close the baggage.) If you are unhappy with the way you are being treated, you do have the right to ask to speak to a CBP supervisor, but I cannot see anything good coming of it, unless they were super rude without provocation or broke something of value.
The authority to delay and speak with travelers derives from the United States Code (section citations below) enables CBP to prevent the entry of persons who are inadmissible under the Immigration and Nationality Act, and to prevent the smuggling of merchandise, including narcotics and other contraband items, into the United States.
Speaking with travelers and examining merchandise coming into or leaving the United States is just one of the mechanisms used to identify illegal or prohibited items, and to determine whether or not someone is trying to enter the U.S. for unlawful or fraudulent purposes. Unless exempt by diplomatic status, all travelers entering the United States, including U.S. citizens, are subjected to routine Customs examinations. At times, people make the mistake of thinking their civil rights are being violated by being asked questions about their trip, personal background and history, etc. That is not the case. Supreme Court decisions have upheld the doctrine that CBP's search authority is unique and does not violate the fourth amendment's protection against unreasonable searches and seizures.
U.S. Customs website has a detailed Q&A section. Most modern countries do as well.