You can write anything you want (basic First Amendment protections): the question is whether one would be liable for damage that arises from what you're written, or whether you can distribute what you've written. Distribution may be restricted, thanks to the Commerce Clause (hence FDA regulations, which figure prominently in the analysis).
Most software writers are not medical practitioners, and vice versa, and what the software writer does is implement something described by a competent medical practitioner. It is logically possible that a med. practitioner might also try to write software (no problem) and distribute it (possibly a problem); or, a software writer might read up on something on Wikipedia and try to implement it (again, no problem) or distribute it (possible problem).
Damages can be sorted into two categories, implementation errors and scientific errors, and having ruled out deliberate sabotage, we are left with negligence. If the software writer failed to use suitable care in writing code that sums a set of numbers, the software writer has been negligent. If the person purporting to have the qualified scientific knowledge mis-states the formula, that person has been negligent. It is possible for many parties to be negligent (the programmer failed to be diligent in understanding the software requirements; the medical professional failed to adequately explain what was required of the program).
The FDA does regulate medical devices, and "device" is construed pretty broadly (condoms are non-exempt class 2 medical devices). There is FDA guidance on medical software which makes it clear that the device manufacturer shoulders the regulatory burden. Anytime you manufacture a tangible thing (which is within the scope of FDA regulation for medical devices), you have to have the thing approved. Many (most?) medical devices implement software, and are thus within the scope of FDA scrutiny. It is not illegal to write software that ends up being implemented in an unapproved medical device, but the unapproved medical device itself is illegal.
It is reasonably likely that purported medical software (not a gadget with software build in) would be held to be a non-exempt device. To know if something is exempt, you would look at the exemptions list, and determine that your program (or thing) is not on or implied by anything on that list. The visual acuity eye chart is a class 1 exempt item, as is the manual toothbrush. There is no way to specifically look for things that are only software, or that contain software, so the search through the list would have to be guided by knowledge of the subject area.
Canada is helpful in explaining when software is a regulated medical device. This is non-probative w.r.t. US law but gives you an idea what is likely to be considered a "device". Under Canadian law, the software would be clearly a regulated medical device. The FDA has a power-point that attempts to say something about the matter which warns you that you are on your own and "You will need to go back to study and use the source regulatory documents" to make the determination. Eventually, slide 13, they hint that if you intend the software to be used in diagnosis, prevention, or treatment, then it is a device (so, yes, the aforementioned software would be a device, and probably not exempt from regulations). General purpose software (word processors, web browsers, communications software, etc.) are not indented to be used for a regulated purpose, although they can be so used, and thus they are probably not subject to FDA regulation.
There is a murky relationship between regulatory approval and liability. Being approved by the government does not convey immunity to negligence suits (see Wyeth v. Levine), but being approved can have weight in determining whether a party was negligent, since regulatory scrutiny ostensibly filters out errors that could have been caught. Federal approval does not preempt state tort law, as the court ruled.
Regulations pertaining to medical devices hold for anything that qualifies as a device, and is not defined (negatively) in terms of disclaimers. If you sell a medical device but label it saying "this is not a medical device, it is not created by a competent medical practitioner", that doesn't make it not a medical device. Unfortunately, what counts as a regulated device is based on intended use, and there is an obvious connection between disclaimers and intentions. Taking MS Word as an example, MS does not as far as I know say that "Word is not intended to be used as a medical device". It can certainly be used to diagnose, teat, and prevent medical conditions, but so too can a screw driver or pretty much anything else. The number of non-medical uses vastly outweigh the medical uses, so it would be deemed not to be a regulated device. A program which prescribed a set of prayers to be uttered in case of illness would be subject to First Amendment override of any FDA regulations. Quack medical devices are prohibited (that's why there is regulation of devices in the first place), but discerning the fine line between permitted actions based on nutty beliefs and forbidden actions is not easy. The FDA also has guidance on the distinction between "Complementary and Alternative Medicine", which does not clearly state that, for example, a software reading of your cakra-energies based on a computer program's questions is not a medical device. They do say of mind-body medicine (mentioning yoga, biofeedback and tai chi as examples) that "CAM practices in this domain would not be subject to our jurisdiction under the Act or the PHS Act", but then say "any equipment or other products used as part of
the practice of mind-body medicine may be subject to FDA regulation, depending on the nature of the product and its intended use" (hence a yoga-enabling program may be subject to regulation).