I'm not a lawyer, and I'm not even a professional photographer, but using a photo you took to promote your own business is the very definition of commercial use. And commercial use opens you up to being sued by someone who can show they're affiliated with the image but do not endorse your business or business practices. Or by the owner of the work being ticked off at your violating their right to privacy (i.e., they would probably prefer choosing whether pics of their bathroom are splashed about the interwebz advertising your business). This is why professional photographers get location and model releases when shooting commercial work. You may also want to look up in your contract with your ex-employer whether it was work-for-hire, in which case they could own the copyrights to images you took while on the job.
Given that the work you're advertising as yours was done on someone else's payroll, presumably using their contacts, materials, and tools, and that you're setting up in direct competition with them, I think they may have a point, but you'd have to ask a lawyer.
Personally, I'd say it's not worth it to possibly antagonize ex-customers as well as an ex-employer in the same industry. Word gets around. I say drop the images from your time with your ex-employer, and start asking customers if you can use photos of the work you're doing for them, and use those. If you're good enough to stay in business, you should well be able to get the customers you're doing work for to endorse you with permission to use images of the glass you did for them.