How to avoid deserialization of untrusted data attacks with Hyperion serializer?

[jonnydee]

We are currently using Akka 1.4 with Hyperion serialization configured for all user messages sent between Akka systems. We want to make sure Hyperion cannot be tricked into deserialization of untrusted data (see CWE-502 for details about that matter).

I know about Hyperion's akka.actor.serialization-settings.hyperion.disallow-unsafe-type configuration option which can be used to enable a built-in blacklist for dangerous types. This approach doesn't suffice in our case, however, because we need to trust that the built-in blacklist currently is complete and also will remain complete in the future. Note that being forced to upgrade Akka to some newer release just because new/missing types might have been added to the blacklist is not an option for us. Being able to configure types on the blacklist ourselves would surely be an improvement but the general problem of an outdated blacklist would still remain. It would just be a shift in who is responsible / trusted to have the most recent and complete list of types on the blacklist.

We would like to have an option for a whitelist approach where we list all types that are allowed to be deserialized explicitly. I looked into Hyperion's "Pre-Configured Types" mechanism which involves an IKnownTypesProvider. I hoped it would allow us to configure known types explicitly while at the same time ruling out unknown ones. Please correct if I am wrong, but it seems like this is not the case. Instead, known types would be replaced by some id (which is good) but "unknown" types would still be serialized with type information (which is bad). Furthermore, Hyperion's current implementation for mapping ids to known types seems to depend on the order of the types returned by IKnownTypesProvider.GetKnownTypes() method. (The id corresponds to the type's position in the returned IEnumerable<Type>.) This makes an approach where known message types are collected by some automatism (e.g. a reflection-based approach) difficult, because all involved Actor Systems would have to see exactly the same list (size/items/order) of known types even though some of them would never see all messages in that "overall" list.

We know that using some schema-based serialization technology (e.g. Google's Protocol Buffers) doesn't suffer from such attack vectors but, unfortunately, we cannot use it in our product. Also securing communication between Actor Systems using a VPN is not an option for us.

So the questions are:

• Is there some other way to defend against untrusted data attacks when using Hyperion?
• Do you have any idea on how to implement a whiltelisting machanism for Hyperion in an easy and future-proof way?

[Aaronontheweb]

@jonnydee

First, sorry for the delay in getting back to you.

So right now it looks like our unsafe types list is static when it really should be configurable. @Arkatufus on our team will open an issue for that and we'll work on some solutions.

I like the idea of taking the IKnownTypesProvider and passing in a setting that says "only allow these types to be deserialized" - that seems like a robust way of handling this. Alternatively, maybe we should allow users to define a construct similar to a types provider but for the purposes of filtering what can and can't be serialized - i.e. maybe a predicate filter. We do have to balance performance with however we do this, so maybe trying something simple with the IKnownTypesProvider would be a good compromise.

[Aaronontheweb]

Furthermore, Hyperion's current implementation for mapping ids to known types seems to depend on the order of the types returned by IKnownTypesProvider.GetKnownTypes() method. (The id corresponds to the type's position in the returned IEnumerable.) This makes an approach where known message types are collected by some automatism (e.g. a reflection-based approach) difficult, because all involved Actor Systems would have to see exactly the same list (size/items/order) of known types even though some of them would never see all messages in that "overall" list.

FYI, this also seems like a bad Hyperion design if this is true - we should probably move to an explicit K/V system, where the developer must specify the "key identifier" for the type, so known type providers can have their lists extended. We should provide a portability bridge for existing IKnownTypesProvider implementations to convert to this without damaging existing wire formats.

[Arkatufus]

As of now, the return value of IKnownTypesProvider.GetKnownType() is the one being passed to the underlying Hyperion serializer, so it can be considered as immutable after the serializer starts.

Expanding IKnownTypesProvider with another method that returns a list of allowed/disallowed types is a viable solution, but it will break compatibility with previous versions of Akka; a safer solution would be to provide another interface to pass in the allowed/disallowed types. It shouldn't matter coding wise, since you can use one class that implements both interface. This implementing class can then be passed in in the HOCON settings just like the known types provider setting.

Adding a toggle in HOCON to let Hyperion to use the list provided by IKnownTypesProvider.GetKnownType() as a whitelist can also work, though there might be a case where user would want some flexibility and have other types to be allowed while not listed as a known type, most probably for backward compatibility.

Another option would be to pass a class that contains a delegate callback, but that means we will couple Hyperion and Akka and that just seemed wrong to me,

[to11mtm]

@Aaronontheweb my comments:

We're not going to check subtype / marker interface, gobbledygook - if you want to secure your Hyperion serialization to the maximum degree possible, build this class to include all of the verbatim type names you will allow

Frankly if you wanted such functionality it would be fairly trivial to implement as an extension method. However it is a footgun if people don't understand the implications of using such a thing (i.e. you'd only want it on interfaces within your own private codebase).

One plus side of the builder pattern alongside using strings, is frankly it makes implementation somewhat clean: If we are in 'explicit' mode and specifying all types up front, that means we could potentially in that mode, branch on loading type by name; explicit mode could get to use a normal dictionary string-type cache, instead of the existing concurrent dictionary (since all types are preadded, don't have to worry about the 'add' case being concurrent.) So you'd basically pay a heavier penalty for creating the serializer, but overall deserialization could actually be -faster-!

[jonnydee]

@Aaronontheweb Thank you very much for your response and suggested solutions. Personally, I like the string-based filtering approach together with the 'TypeFilterBuilder'. However, I'm not sure if my initial 'whiltelist' idea was a good one, because it might be tedious and error prone to list all known message types. Your suggested 'ITypeFilter' interface with 'Exclude' method seems to go for the blacklisting approach which is indeed better, I think. Correspondingly, the 'TypeFilterBuilder' should provide an '.Exclude' method which would be used to build up the blacklist in a type-safe manner.

Of course, being able to either backlist OR whitelist types would allow for greatest flexibility.

[Aaronontheweb]

@jonnydee with that exclusion system you could still implement "whitelist"-style filtering by simply taking everything in your filter you want to include and then just !-ing it inside the boolean logic whereever that goes. "Anything that is not one of these types gets excluded"

This would allow you to specify other conditions too, such as "anything that is not on this list" AND "anything on this list that doesn't belong to an Assembly with name X" or whatever.

It will be tedious - you'd have to pre-scan the types you'd want to include using some assembly scanning on the builder, but that way the reflection is controlled by you when you're setting up the filter, not by the filter itself when it's evaluating messages.

[jonnydee]

@Aaronontheweb You're right, of course. Thanks again for your help. The chosen approach is brilliant! :-)

[Aaronontheweb]

No problem! We're getting ready to merge it into Hyperion and release it soon: akkadotnet/Hyperion#281

With any luck that should be available by the end of the week and we'll also adapt Akka.Serialization.Hyperion to use it.

[Aaronontheweb]

@Arkatufus has posted a full spec for this here: akkadotnet/Hyperion#275 (comment)

[Aaronontheweb]

This is now merged into Akka.NET v1.4.32 #5510

This change will be available in our nightlies beginning tomorrow: https://getakka.net/community/getting-access-to-nightly-builds.html

And we'll release 1.4.32 soon.

[Aaronontheweb]

cc @jonnydee

[Aaronontheweb]

@jonnydee this was released as part of Akka.NET v1.4.32 yesterday.

[jonnydee]

@Aaronontheweb thanks for this great support and for letting me know 😀

[Aaronontheweb]

No problem. Let us know if you run into any issues with it.