Skip to content

failure to converge when several bands are specified. #5

@jlwehle

Description

@jlwehle

Attach is a test case based on a CIC compensation filter where
fquant succeeds if the number of frequency bands is reduced
below a certain threshold, otherwise it fails. The code is
built like so:

clang++ -I/usr/local/include -O2 -march=amdfam10 \
  -o fquant_tc fquant_tc.cxx -L/usr/local/lib -lfquantizer -lmpfr

and running it results in:

Parks-McClellan delta = 0.017878909509322568616
Naive rounding error    = 0.84135723114013671875

Reduction = 600.09815000000003238 ms
LLL initial error       = 0.85398578643798828125
Band 0 error = 0.85398578643798828125
Band 1 error = 0.24652576703809714331
Band 2 error = 0.029929713393439527636
Band 3 error = 0.13246696585733630513
Band 4 error = 0.41661805504400462041
Band 5 error = 0.21240771528807655955
Band 6 error = 0.018072934649282501596
...
The exchange algorithm did not converge.
TRIGGER: Not enough alternating extrema!
POSSIBLE CAUSE: Nmax too small
The exchange algorithm did not converge.
TRIGGER: numerical instability
POSSIBLE CAUSES: poor starting reference and/or a too small value for Nmax.
Parks-McClellan delta = nan
[fquant_tc.cxx.txt](https://github.com/sfilip/fquantizer/files/3823011/fquant_tc.cxx.txt)

It would be useful for more cases to work with fquant.

-- John

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions