Skip to content
This repository was archived by the owner on Apr 6, 2019. It is now read-only.
This repository was archived by the owner on Apr 6, 2019. It is now read-only.

fixed_point compare uses too great a range #257

@johnmcfarlane

Description

@johnmcfarlane

In

auto a = fixed_point<int8, 32>(0x100000000LL);
auto b = fixed_point<int8, 0>(1);
ASSERT_GT(a, b);

expression overflows. But only 16 (maybe 15) bits of range are required to scale operands such that comparison can be performed. See issue 123 for a discussion.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions