Update deprecated mem_ops.h function calls to use std::span-based versions and resolve some TODOs

[KaganCanSit]

Hello,

This draft PR includes updates to modernize the codebase by replacing deprecated mem_ops.h function calls with their std::span-based counterparts in line with C++20 standards. Additionally, I also discussed some TODO comments that I thought would be quick fixes.

My reference points the C++ Core Guidelines and CppReference. Since I am not actively developing in C++20 at the moment, I am marking this PR as a draft to review and feedback.

Changes made:

• Replaced xor_buf calls with std::span-based equivalents.
• Handled several smaller TODOs identified in the codebase.

Confirmed that builds and test suites pass with:

ninja clean && ./configure.py --without-documentation --cc=clang --compiler-cache=ccache --build-targets=static,cli,tests --build-tool=ninja && ninja && ./botan-test

Check code format with:

python3 src/scripts/dev_tools/run_clang_format.py --clang-format=clang-format-17 --src-dir=src

---

Outstanding considerations:
I intentionally did not modify the following function, as it touches multiple areas and could introduce bugs without extensive review:

// TODO: deprecate and replace, use .subspan()
inline void xor_buf(std::span<uint8_t> out, std::span<const uint8_t> in, size_t n) {
   BOTAN_ARG_CHECK(out.size() >= n, "output span is too small");
   BOTAN_ARG_CHECK(in.size() >= n, "input span is too small");
   xor_buf(out.first(n), in.first(n));
}

Files affected by this function;

src/lib/modes/aead/eax/eax.cpp
src/lib/mac/cmac/cmac.cpp
src/lib/pubkey/dlies/dlies.cpp

---

Additional notes:
There is a change in TLS, so additional checking may be required.

Thanks in advance for your time and guidance.

[coveralls]

coverage: 90.422% (+1.2%) from 89.189%
when pulling 1e9606e on KaganCanSit:feature-handling-some-todo-messages
into f2257be on randombit:master.

[reneme]

Looks good to me now. Thank you for taking this on!

For the record: The mem_ops.h header is marked as "future internal header". So, technically, we don't even need to mark the typecast_copy and xor_buf overloads as "deprecated", because for library users the entire header is going to go away in a future major release.

Personally, I still think your change in mem_ops.h makes sense as-is, for documentation purposes.

@randombit, over to you. :)

[reneme]

Thanks for your patience with this. I left a few suggestions for simplification (mostly unnecessary explicit construction of std::span).

Regarding the changes in xts.cpp: I think we should postpone those until some more top-down modernization has landed. Most notably #4880 and similar effort to spanify process_msg and start_msg. Once that is done, the abstract interface of all cipher mode implementations will be purely span-based and ready for such modernizations.

[KaganCanSit]

I have addressed all feedback from the review:

• Changes to src/lib/modes/xts/xts.cpp have been rolled back as suggested to avoid conflicts with upcoming modernizations (Refactor: Cipher_Mode::finish_msg() using std::span #4880, Chore: BufferTransformer and Blowfish Refactor #4151)
• Retained deprecation warnings in headers to prevent new ptr-length style additions.
• The existing branch has been re-baselined to the parent branch.
• Implemented other requested fixes and improvements. (Open typing is a bit my style, sorry.)

roughtime std::span modernization proposal BufferSlicer is noted for future work - I may address this in a separate PR once it is merged. Comment

I will follow the results of the CI tests and then we can merge if you see fit. I see you're quite busy with clang-tidy today, I'll rebase it from time to time if needed, just FYI.

[reneme]

Looks good to me. Thanks for your efforts.

[KaganCanSit]

Looks good to me. Thanks for your efforts.

Thank you for your time and for sharing your expertise. Your guidance has been very helpful throughout this process.

I’ll need to spend some additional time getting more familiar with the span construct. Since I'm currently use the C++11 standard, I've not using it in job.

@randombit, this development is ready to be merged at your discretion.

Best regards.

[randombit]

Sorry for the wait here. Changes are generally fine but the issue is mem_ops.h is currently a public header but I don't think the library should be in the business of providing general utility functions like this, which is why the header is deprecated and will be removed in Botan4. Any new functionality along these lines should go into the (internal) mem_utils.h header. Ideally we can update all of the callers within the library and then guard the declarations here, which are no longer used internally but which we must continue to provide for now for compatability, with !defined(BOTAN_IS_BEING_BUILT) as is done with various other functions, so it's not possible to add new code to the library which uses them.

[KaganCanSit]

Sorry for the wait here. Changes are generally fine but the issue is mem_ops.h is currently a public header but I don't think the library should be in the business of providing general utility functions like this, which is why the header is deprecated and will be removed in Botan4. Any new functionality along these lines should go into the (internal) mem_utils.h header. Ideally we can update all of the callers within the library and then guard the declarations here, which are no longer used internally but which we must continue to provide for now for compatability, with !defined(BOTAN_IS_BEING_BUILT) as is done with various other functions, so it's not possible to add new code to the library which uses them.

I recall we previously discussed removing the header file here. Actually, adding a new function for ‘mem_ops.h’ in this branch is not currently on the agenda. I was only focused on the TODO message; my goal was to deprecate the use of existing function calls and update them with std::span-based overloads.

Within the project, the use of functions marked with the declaration BOTAN_DEPRECATED(‘This function is deprecated, use the range-based version instead’) has now ended. These can be completely removed. However, since there may still be users who use them, I thought it would be prudent to declare them BOTAN_DEPRECATED. But additionally, to ensure they are not reused within the project, they can be wrapped with the !defined(BOTAN_IS_BEING_BUILT) macro, as you suggested.

I thought moving them to the (internal) mem_utils.h header would be a separate task. I'm not sure, maybe I misunderstood.

Some time has passed, so I need to review it again, but if you guide me step by step, I can work on it as soon as possible and apply the update you want.

[reneme]

Given that this PR doesn't add any functionality to mem_ops.h, I would suggest the following steps :

1. Let's change the deprecation messages in mem_ops.h from "This function is deprecated, use range-based version instead" to something along the lines of "This function is deprecated without a replacement". That wording doesn't suggest that we're planning to provide support for this function post-Botan4.
2. Merge this PR (because the comment history is becoming unwieldy)
3. In a follow-up PR: Move the declarations and internal usages of the relevant functions over to the internal mem_utils.h, guard mem_ops.h with !defined(BOTAN_IS_BEING_BUILT) for backward compatibility, and eventually remove it in Botan4

[KaganCanSit]

Given that this PR doesn't add any functionality to mem_ops.h, I would suggest the following steps :

1. Let's change the deprecation messages in mem_ops.h from "This function is deprecated, use range-based version instead" to something along the lines of "This function is deprecated without a replacement". That wording doesn't suggest that we're planning to provide support for this function post-Botan4.
2. Merge this PR (because the comment history is becoming unwieldy)
3. In a follow-up PR: Move the declarations and internal usages of the relevant functions over to the internal mem_utils.h, guard mem_ops.h with !defined(BOTAN_IS_BEING_BUILT) for backward compatibility, and eventually remove it in Botan4

Thank you for informing me and guiding me @reneme. I would like to take on this task in this thread and in the pull request that will be opened later. Is @randombit suitable for you?

---

Updates:
To avoid further clutter in the comments, I will update this comment.
I am implementing the solution proposed by @reneme here. If deemed appropriate and these changes are merged, I am ready to take part in developing the remaining changes. For this, I will wait for this branch to be merged. (I don't want to repeatedly rebase a new branch based on this branch with every change.)

• 09.11.25: Rebased to main branch.
• 01.01.26: Rebased to main branch.

[KaganCanSit]

Hello @randombit, @reneme,

First, I wish you a healthy, peaceful, and successful New Year. I hope to contribute more to the project in 2026.

I wanted to share a note regarding this long-standing development branch. Some of my branches have been open for a long time and for some I’ll even need to revisit the details to fully recall what I did (there are ones approaching a year). For this reason, we could follow the approach @reneme outlined in this comment.

Another idea I have is: if avoiding changes to the header file (mem_ops.h) is important for you, then there aren’t actually any new functions added here. This PR only introduces deprecation annotations. If preferred, we could update the usages within the project to use std::span without making any changes to the header. That way, future modifications will be easier and will be future for Botan4 only change header file.

I would really like to contribute to this development. Unfortunately, I don’t have much practical experience with std::span yet, as I haven’t been using the latest standard extensively. However, I am always open to your guidance and feedback.

Kind regards.