[XERCESC-2241] Fix integer overflows in DFAContentModel class

[rouault]

On .xsd files like the following ones (generated by ossfuzz, so broken), integer overflows can happen in DFAContentModel::countLeafNodes() and DFAContentModel::buildDFA() which can later cause out-of-bounds access.

Found in https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=52025

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
           xmlns:myns="http://myns"
           targetNamespace="http://myns"
           elementFormDefault="qualified" attributeFormDefault="unqualified">

<xs:element name="main_elt">
  <xs:complexType>
     <xs:sequence>
        <xs:group ref="myns:mygroup" minOccurs="32767" maxOccurs="1"/>
      </xs:sequence>
  </xs:complexType>
</xs:element>

<xs:group name="mygroup">
  <xs:sequence>
      <!-- related to https://issues.apache.org/jira/browse/XERCESC-1051 -->
      <xs:element name="elt" maxOccurs="33333">
        <xs:complexType>
            <xs:sequence>
 ame="x" type="xs:int" maxOccurs="1"/>
            </xs:sequence>
        </xs:complexType>
      </xs:element>
  </xs:sequence>
</xs:group>

</xs:schema>

[rouault]

CC @rleigh-codelibre This should be relatively safe to apply

[rleigh-codelibre]

The logic of the change all looks fine, but a query regarding the type used for numeric_limits::max().

Thanks,
Roger

[rleigh-codelibre]

Thanks for the changes, this all looks to be correct to me. I'll merge it once the CI has passed (for the CI build which isn't broken).

[scantor]

@rleigh-codelibre If you can apply this to master that will make it easier for me to cherry-pick back to the branch.

[scantor]

(My only concern re: compatibility was the reference to size_it in the max function possibly causing compatibility issues, but I'll take the risk.)

[scantor]

I applied this to both branches.