HReddy118194 (Community Member) asked a question.
I am getting an integer over-flow issue for the following code. Please help me fix this error.
The errors on the lines :
- size_t bytesPerRow = inWidth << 2;
- size_t dataSize = bytesPerRow * inHeight;
where size_t is unsigned long long datatype.
PLRGBAImageRef _Nullable PLRGBAImageCreate(size_t inWidth, size_t inHeight)
try
{
constexpr size_t kSizeLimit = numeric_limits<size_t>::max();
if (inWidth <= (kSizeLimit >> 2) / ((0 == inHeight) ? 1 : inHeight))
{
size_t bytesPerRow = inWidth << 2;
size_t dataSize = bytesPerRow * inHeight;
unique_ptr<unsigned char[]> data(new unsigned char[dataSize]);
return new RGBXImage(std::move(data), {inHeight, inWidth, bytesPerRow}, 0, 0);
}
return nullptr;
}
.png){kind=logo}
Hi @HReddy118194 (Community Member),
The concern here is that these calculations could result in values greater than would fit in a size_t, which can have unintended consequences. We recommend that you verify the result of the calculation does not wrap around, and if it does to return an error. For example, inWidth should not exceed `SIZE_MAX / 4` since the left-shift operation multiplies by four.
Kind regards,
Duncan