Is there a short enough sample length (1 second, 1/10 second, 1/100 of
second?) where the distribution of the audio clip isn't infringement?
There is no arbitrary rule.
Audio samples (and opposed to the underlying compositions which audio samples are a performance of) are held to the most strict standards of infringement (particularly when it comes to establishing a prima facie case of infringement prior to considering whether the fair use defense applies) of pretty much any kind of work protected by copyright. Even short samples will generally be infringing.
The leading case is Heim v. Universal Pictures Co. 154 F.2d 480 (2nd Cir. 1946) in which one of the intermediate U.S. Courts of Appeals held that infringement could be proven if "a single brief phrase, contained in both pieces, was so idiosyncratic in its treatment as to preclude coincidence," and is not instead a copy of an indistinguishable snippet of music that is in the public domain.
The lower threshold is probably that the sample needs to be long enough for a human listener to discern that it is a sample of some particular recognizable copyrighted work, if the listener is familiar with the original work.
The speed of the sample and the distinctiveness of the sampled material would influence this determination.
A single generic middle C on an ordinary piano held for a whole note at an allegro pace might not qualify as an infringement. A very distinctive three note sequence with a very distinctive vocal or sound quality in a fast tempo piece might be only a fraction of a second long and still be infringing.
The specific use case I'm interested in involves build a database of
audio samples for a musical algorithm (e.g., detection of genre or
mood) with the intention of later distribution of the database.
The hard and close question, in my mind, is not the size of the individual audio samples, but whether the form in which the samples are used is sufficient to constitute an infringement of one of the specific sticks in the bundles of rights that come with a copyright (e.g. reproduction, distribution, performance, derivate works) that is protected by a copyright. In other words, is an audio clip that is buried as part of training set data for an app that only an IT professional can access with a device other than the one upon which it would be typically used really distribution of a copy, or a performance, or a derivative work.
The implication of this description of the proposed activity is that a user of the software can't actually listen to any of the audio samples, which simply constitute a training data set for a machine learning process.
While copyright is broader than simply a right to prevent someone from commercially appropriating the value imparted by the creative innovations of the original work (e.g. copyright can be used to prohibit a charitable or free distribution of a copyrighted work), copyright law is also interpreted in light of its purposes. And, the primary purpose of copyright law is to prevent the commercial or non-commercial appropriation of the original work without permission from the author (or a mandatory license to perform one's own cover version of an original musical composition for a fee set by an administrative agency), and is interpreted in that light.
The value of this software to its users doesn't seem to meaningfully implicate this interest since the user doesn't benefit from any one particular individual work, so it isn't obvious to me that audio samples used in this manner are infringing.
The safer course of action, however, would be to do the machine learning process from the audio samples before distributing that software, and to include only the sound processing model that flows from that machine learning process, rather than the training set data of audio clips themselves, in the distributed product. To allow users to understand the training set and its influence on the end machine learning product, the software could be distributed with a bibliography citing to all of the works from which audio clips were taken that were used in the training data with pinpoint citations to the portions of the cited works from which audio clips were used.