Finanzamt will not help too much with this question. It is clear already that if a person lives in Germany, then usually he/she need to submit tax declaration in Germany (of course, there could be different circumstances also including a tax agreement between two countries, which is relatively simple to find online).
@Badle: there are definitely ways to declare it and in many cases you should do it. If unsure, please ask tax consultant.
However, the original question is not about taxes, it is about if a person can work as a freelancer while he/she is not allowed to "work as freelancer in Germany".
Here are my 2 thoughts:
1) If you are allowed to work in another country (e.g., your country of origin) then of course you can work there when, for example, you go there for a travel. However, if you are physically in Germany while working, then I'm not sure how to prove that German laws does not apply here.
2) You can simply get a permission for a freelance job, please see last page of this document
P.S. I'm not a lawyer.