In Common Law Countries
In common law countries, Dale M. is right (the vast majority of the time).
Without negligence there is usually no liability for damage to property in an accident. The general rule in the common law rule is that "shit happens" and no one is responsible for the damage, when no one was negligent and everyone defendant acted reasonably under the circumstances. The "shit happens" defense is generally a valid defense to a claim for property damages or personal injury in common law countries.
"No negligence despite causation" findings are rare in auto accident or plane crash cases (absent truly extraordinary weather conditions or freak intervening causes like meteors falling from the sky that cause car accidents), but are common, for example, in professional malpractice cases.
For example, suppose a reasonably competent surgeon does surgery on you in a fairly high risk case. The best surgeon in the hospital could have saved you. You didn't get that surgeon and died. There is no basis for a suit for medical malpractice against the surgeon since the surgeon was not negligent, even though the surgeon was not perfect.
Also, sometimes the victim is the person primarily or totally at fault. For example, suppose a drunk driver T-bones your WellsFargo armored cash delivery car, which is built like a tank, in violation of a red light, with their SmartCar, and the SmartCar is totaled, while your armored car doesn't even have a scratch. Your car was a cause of the damage to the drunk driver's vehicle, but you have no liability for the drunk driver's losses, because you weren't negligent.
Strict Liability Exceptions To Negligence Based Liability
There are some exceptions if you have provided a warranty, guarantee, or insurance that the property won't be damaged (since contractual liability is generally strict liability unless otherwise provided by agreement). For example, you might be liable under a lease for any damage in excess of reasonable wear and tear from any cause other than the landlord's negligence.
Sometimes there can be negligence by someone your are responsible for even if you aren't personally negligent (which is called "vicarious liability").
There is strict liability for accidents caused by defective products even in the absence of negligence, and for accidents caused by ultra-hazardous activities (e.g. explosives).
U.S. states are divided about liability for animals that roam free with the main divide being between fence in states (mostly in the eastern U.S. and other more urbanized places) and fence out states (mostly in the west and more wild frontiers).
There are a few other exceptions, but they are rare and somewhat inconsistent between jurisdictions.
Comparative Fault
Also, the modern trend in common law countries is to allocate liability for accidents based upon comparative fault or modified comparative fault. So, everyone who was negligent (including the victim), or would otherwise have had strict liability for the damage, is assigned a percentage of fault (adding up to 100%) that is their share of responsibility for the overall loss. If no one is negligent at all, there is no recovery. In modified comparative fault, if the victim is at least either 50% or 50%+ at fault (depending upon the state), there is no recovery.
Some systems of comparative fault make negligent people who owe money jointly and severally liable with a right to contribution if you pay more than your fair share of the loss. Other systems limit each person's liability to their percentage of fault with no implicit guarantee of other negligent parties who are judgment proof.
Who Determines Liability And Damages?
In the U.S., and a small number of other jurisdictions, liability and damages are frequently decided by juries (and there is a right to a jury the vast majority of the time even if it isn't always elected). In most common law jurisdictions, liability and damages are always or almost always decided by judges (exceptions apply in perhaps 1% or less of cases).
In Civil Law Countries
In civil law countries (continental Europe, Quebec), the standard of liability for accidents in the absence of special cases like the ones discussed in the common law is that you are liable for damage that is your "fault." See, e.g. this article discussing civil law tort liability under Central American civil codes. It states:
Much like in Europe (think of art. 1382 of the Napoleonic Code), in
the Central American civil codes, the concept of tort ordinarily rests
on a general clause imposing fault-based liability, though it is
possible to identify among these civil codes some interesting
variations. For example, only in Honduras (art. 2236) and Panama (art.
1644) tort liability is characterized using elements such as: action
or omission, fault or negligence, and obligation to compensate.
Whereas, Costa Rica (art.1045) and Nicaragua (art. 2509) add other
elements to the characterization, such as fault and imprudence, and in
the Nicaraguan text, the notion of malicious acts is also included.
The Salvadoran Civil Code (arts. 2065 and 2080) is rooted in the
classic construction of delict, quasi-delict or fault, although it
also adds features such as malice and negligence. The exception to
this trend will be the Guatemalan Civil Code (art. 1645). Although it
uses terms such as intention, carelessness or recklessness as defining
criteria, this provision is not really describing a fault based
liability model, because it contains a rebuttable presumption of fault
or negligence.
The civil law concept of tortious fault is not spelled out in great detail in civil codes, although there are a few specific situations that are covered (e.g. bailments when your property is in the possession of another person and collapsing buildings). Civil law countries have instead been developed in legal doctrine in those countries that is not apparent from the civil code text.
Basically, civil law tortious fault involves some culpability greater than mere "but for" causation of an accident, but involves a lower threshold of wrongfulness than the concept of "negligence" in the common law, which is a failure to act as reasonable person would have to prevent harm to others under the circumstances. You can have civil law tortious fault even if you were acting as a reasonable person, but you still have to have at least done something slightly wrong. So, there is still a "shit happens" defense in civil law countries, but it tends to be much harder to establish.
Who Determines Liability And Damages?
In civil law countries, the call is always made by judges (or by panels of judges in cases involving larger damages).