Brodes/seh flow phase1 throwing models

[bdrodes]

Part of a phased approach to an overhaul of how exceptions are modeled and handled in IR, in order to handle both SEH (structured exception handling) and ordinary C++ exceptions. These two types of exceptions have subtly different mechanics and uses. Consequently, users cannot treat all exceptions as the same. E.g., a call to memset can raise an SEH exception but it would not raise an exception with C++ exceptions.

In this PR, I start by creating a new Throwing.qll library to provide metadata for any kind of exception, and a predicate to indicate if it is SEH or CXX. This change impacts models for NonThrowingFunctions, such as (as previously mentioned) memset. This PR, however, does not fully provide support to differentiate SEH and CXX exception handling in the IR. This will be addressed in future PRs.

[jketema]

You cannot just delete files that are not in non-implementation or internal directories. These will need to go through a deprecation period. Could you discuss with @MathiasVP what the correct approach should be here.

[MathiasVP]

Indeed, we have a 1-ish year deprecation period for non-internal QL things. So we need to:

• Mark this class as deprecated
• Keep the functionality. i.e., extending NonThrowingFunction should do something equivalent to what it did before

And then it'll be deleted by someone in a PR a year from now.

[bdrodes]

I was thinking about that after I submitted, fixed.

[jketema]

Can't we just do:

private import Throwing as T

deprecated class NonThrowingFunction = T::NonThrowingFunction;

and write in the change note that (a) the class is deprecated, and (b) requires a new member to be implemented.

[jketema]

Or is that too dirty?

[MathiasVP]

We can definitely do that, yes!

[bdrodes]

We can do this, but I had avoided changes like this because any existing use of the old NonThrowingFunction would require the member predicate be defined, breaking any existing queries. I thought the point of deprecating vs deleting was to not completely break existing builds. If you are actually ok with that @jketema I'm ok with it.

[bdrodes]

I suppose another option is to just make the deprecated version extend the new throwing function mechanic and set the exception type predicate to be any type (matching the prior intended behavior). I can do whatever, just please advise what is more acceptable to github standards.

[jketema]

Similarly, the mayThrowException predicate will need to be kept for now and marked as deprecated.

[bdrodes]

Need to rethink this generally for backwards compatibility given the original thing being deprecated is abstract. Stand by...

[bdrodes]

Ok I decided to move back to the old mechanic. I switched because SEH doesn't really throw an exception per se. But it's fine, and easier to just keep it the way it was. The changes have been made, just waiting for the checks to pass.

[bdrodes]

I'm seeing failures with some security/CWE tests, but I cannot recreate that locally. @MathiasVP is that a common discrepancy? can you maybe run the test locally as well? Thye all passed for me.

[paldepind]

Why do we need to deprecate NonThrowingFunction and add a new NonThrowingFunction that extends ExceptionAnnotation? To me it seems that the things in ExceptionAnnotation are not relevant for NonThrowingFunction? E.g., having a getExceptionType on a function that doesn't throw exceptions.

Perhaps the stuff in ExceptionAnnotation could be moved inside ThrowingFunction, the ExceptionAnnotation could be removed, and NonThrowingFunction could be kept as is. Would that not work?

[MathiasVP]

Why do we need to deprecate NonThrowingFunction and add a new NonThrowingFunction that extends ExceptionAnnotation? To me it seems that the things in ExceptionAnnotation are not relevant for NonThrowingFunction? E.g., having a getExceptionType on a function that doesn't throw exceptions.

This is because we want to distinguish between the kinds of exceptions that cannot be thrown. For example, memcpy cannot throw any C++ exceptions. However, when running on a Windows machine it may throw a structured exception (for example, if memcpy dereferences an invalid pointer). So we need to be able to distinguish between "a function that cannot throw a C++ exception, but may throw a structured exception" and "a function that may throw a C++ exception".

I don't care specifically about how we distinguish between those two concepts. I agree that it's a bit unintuitive that you implement a getExceptionType predicate when extending NonThrowing. The interpretation is that this models that the function cannot throw an exception of this specific type (where "type" here refers to whether it's a structured exception or a C++ exception).

Other API suggestions could be:

1. 4 classes: NonThrowingSehException, NonThrowingCppException, ThrowingSehException, ThrowingCppException (where, for the last two classes, you'd need to implement a predicate that specifies whether this is a "must" or a "may" throw).
2. Get rid of NonThrowing and Throwing and simply have one class ExceptionFunction (similar to the one in this PR which is named ExceptionAnnotation) where you implement a predicate like hasException(Exception e, Option<Boolean>::Option unconditional) (or any other three-valued type) with the specification that:
   • A function may throw a C++ exception if hasException(TCxxException(), Option<Boolean>::some(false)) holds
   • A function may throw a SEH exception if hasException(TSehException(), Option<Boolean>::some(false)) holds (this one isn't really needed since any function should be assumed to may throw a SEH exception inside a try block)
   • A function must throw a C++ exception if hasException(TCxxException(), Option<Boolean>::some(true)) holds
   • A function must throw a SEH exception if hasException(TSehException(), Option<Boolean>::some(true)) holds
   • A function will not throw a C++ exception if hasException(TCxxException(), Option<Boolean>::none()) holds
   • A function will not throw a SEH exception if hasException(TSehException(), Option<Boolean>::none()) holds (I don't imagine this will ever be necessary as this guarantee cannot really be made about any function)

[jketema]

I find this confusing. Together with the NonThrowing this seems to say that:

This function is non-throwing, but only in the case of C++ exceptions. So it may still throw a SEH exception.

Is that the correct reading?

[bdrodes]

That's right. That's what it is saying. It is no longer sufficient to say a function doesn't throw, you have to say how it doesn't throw (which kind of exception doesn't it throw). If they want to say it doesn't throw any you can just return the parent exception type.

The issue we got into with making memcpy nonthrowing is that it is true that it doesn't throw a C++ exception, but it absolutely throws a SEH exception. The mechanics in this PR force users to think about what it is they really want when they say a function throws or doesn't throw.

[jketema]

Thanks.

I wonder if this is all somewhat overly complicated. In my understanding the following cases are interesting:

• C++ exceptions:
  • Functions that may throw (this is the default, but are modelling of this in the IR is limited)
  • Functions that do never throw a C++ exception, i.e., functions marked either as `noexcept or that are C functions (that we model)
• SEH exceptions:
  • Functions that may throw (this is the default, and we want to improve the modelling in the IR here)
  • Functions that always throw

Is this correct?

[jketema]

We have a brief meeting. The conclusion was that my assessment above is correct. The proposal is do something simpler:

• Rename, NonThrowing to something like NonCppThrowingFunction and introduce a deprecated NonThrowing alias.
• Introduce a new class AlwaysSehThrowingFunction which is used to model functions that always throw an SEH exception.
• Deprecate the Throwing class.

We should also remove the use of the Throwing class in the models, but we can only do that in the next phase when we update the IR (otherwise the IR breaks).

[bdrodes]

Overhauled the PR, let me know if that works.