MDEV-35639 func sha2/char - handle OOM

[grooverdan]

• The Jira issue number for this PR is: MDEV-35639

Description

The SQL functions sha2 and char perform string allocations without returning
a SQL error if these allocations fail.

Corrected so SQL errors are returned if the reallocation failed.

Release Notes

Handle sha2/char function out of memory errors on the chance reallocation of memory fails.

How can this PR be tested?

With difficulty. The error handling is pretty straight forward.

If the changes are not amenable to automated testing, please explain why not and carefully describe how to test manually.

Basing the PR against the correct MariaDB version

• This is a new feature or a refactoring, and the PR is based against the main branch.
• This is a bug fix, and the PR is based against the earliest maintained branch in which the bug can be reproduced.

PR quality check

• I checked the CODING_STANDARDS.md file and my PR conforms to this where appropriate.
• For any trivial modifications to the PR, I am ok with the reviewer making the changes themselves.

[cvicentiu]

Hi Daniel!

Good catch for sha2. You can test this (at least manually) by just faking an OOM for str->alloc. Like so:

 if (str->alloc((uint) digest_length*2 + 1) || true) /* Each byte as two nybbles */

For the other realloc in Item_func_char, however, at least for this function you need to fix append_char too. Here's an idea how I would optimize the function:

What if we prealloc: arg_count * 4 + 1 at the start of val_str, as that's the maximum length the str can end up storing. In that case you just need to check that function's error code.