As touched on above, there are some boundaries between notes on a full range where the harmonic structure noticeably changes from one recorded note's structure to another's. It is not possible simply to apply a polynomial fitting algorithm to the harmonics as was done in the power contribution case. For example, consider a set consisting of two recordings of C1 and C2 (which are an octave apart). Both recordings have as their most powerful frequency (first peak) their base frequency. The lower recording has as its second most powerful frequency (second peak) twice the base frequency (one octave or above the base), as would be expected from a simple harmonic series. The higher recording, however, has as its second peak triple the base frequency (two octaves or above the base). If a simple linear interpolation is applied to determine the second peaks, the results between C1 and C2 will evenly distribute between and above the base, which frequencies are not in the harmonic series, and thus will not sound correct. For example, taking one note above the lowest recording, C1, this incorrect interpolation dictates the second peak should be above the base. D1 similarly has a second peak above the base, continuing up to B1 at , and ending at C2 at above the base, or triple the base frequency.
In general, the harmonics above the base are taken from a set of often-used harmonics, that tend to be integer multiples above or integer divisors below (e.g., ) the base frequency. Thus, a solution is perhaps a somewhat discretized list assigning a certain probability to each harmonic. Other algorithms involving optimization and genetic algorithms may suggest other models. These models, when applied to data from a recording set over a full range of notes, could yield methods to generate individual harmonic structure for all frequencies.
Of interest to sound engineers and others interested in vocoders is the possibility of applying this method on a recording set not consisting of all one instrument. For example, a recording set of a bass drum, oboe, and voice could produce unique results with varying components of each at all ranges. This application, however, is not worth while until the harmonic structure is a function of frequency, as discussed in the proceeding paragraph.
The method described herein does not model many parts pipe speech: (1) the beginning of pipe speech, called chiff, is an important part of all pipe organs that is specifically modified (not removed) by the builder during installation; (2) the end of pipe speech, which is of less importance; (3) sibilance, the sound of air moving over a pipe mouth during nominal speech. Without these it is clear that the produced sound does not come from a genuine pipe. The above method works well on sounds that have a few significant frequencies making up much of the sound. These three parts of pipe speech have no such property, and so are perhaps modeled better in the time domain, instead of the frequency domain.