Under Cal. Civ. 1950.5(f)
Within a reasonable time after notification of either party’s
intention to terminate the tenancy, or before the end of the lease
term, the landlord shall notify the tenant in writing of his or her
option to request an initial inspection and of his or her right to be
present at the inspection
Since the law also specifies the form of the notice, it is not a reasonable inference that writing can be dispensed with if a tenant seems to have knowledge of the option. So the landlord has violated the law. The tenant is supposed to either decline the option (discharging the landlord's obligation), or request an inspection. Then comes the scheduling at a mutually acceptable date and time, with at least 48 hours' prior written notice of the date and time of the inspection (the 48-hour requirement can be modified with a written waiver signed by both parties). Then the inspection shall proceed, and the landlord presents a list of deficiencies to the tenant. It seems that you're at that stage.
Right at this very minute, it's not clear that you have been damaged: you've simply been notified of the deficiencies that could result in some of the security deposit being retained. The first solution is therefore to remedy those deficiencies, as you would have done, had things followed stipulated procedure. Various reasons come to mind why that might not be a reasonable option (landlord already cleaned it up and charged you; your work schedule precludes timely implement of clean-up, or the fact that you moved far away).
It's not clear that the landlord avoided the inspection – it sounds like he did inspect the place, just not in your presence (there is also a requirement for the form of giving notice about deficiencies: was that law followed?). The thing to focus on is that the law gives you the right to an inspection at a reasonable time before move-out, so that you can remedy any deficiencies. See para 3:
The tenant shall have the opportunity during the period following the
initial inspection until termination of the tenancy to remedy
identified deficiencies, in a manner consistent with the rights and
obligations of the parties under the rental agreement, in order to
avoid deductions from the security.
The law then allows you to maintain an action in small claims court against the landlord for the amount retained by the landlord.