To technical blogs
26 May 2021

What is (de)serialization?

We see serialisation and deserialization a lot when we implement applications and services. Serialisation is a way of converting data objects into a format which can be used to transfer the object. Some common known examples are YAML, XML and JSON but there are more ways to serialise data. When we serialise we transform our objects into a transferable format and during deserialization we transform the special format back into objects we can use in our software.

When is deserialization insecure?

Deserialization can be insecure if the input is out of our control and we don’t validate the data. For example the input of a user will be deserialised in our application into a object which we are using in our software. We assume the user provides a correct serialised version of the object, but we don’t know for sure. Besides that there are a lot of libraries which can be used for (de)serialisation and some of them have ‘features’ we’re not always aware of.

Insecure deserialization in Java

The basics of serialisation in Java

In Java objects can be serialised and deserialised when the class implements the java.io.Serializable interface. Let’s check this out by an example.
The following Message class is meant to be sent to another service. I use Java’s internal way to serialise and deserialise the instance of the class and write the serialised object to a file called my_message.ser.
The Message class:
Let’s create an instance and write it as file:
The instance myMessage is serialized using Java’s ObjectOutputStream and the result is written to a file using the FileOutputStream class in Java. Of course, this can also send to a network connection, etc. There are a lot of implementations of this OutputStream.
If we want to deserialize this message we can read the file again and restore the Message object.
If we check the hexadecimal content of my_message.ser we’ll see the following:
If we change the content of my_message.ser in a hex editor we can manipulate the object. For example, we can change ‘John’ to ‘Jack’. If we read the file and restore our Message object again, you will see that the author is not John Doe anymore but Jack Doe.

Using serialization in Java

In the example above we write the serialized data to a my_message.ser file. Another usage of serialized data can be found in message brokers using the JMS protocol.

Let’s checkout an example by publishing our Message to a Artemis queue using the JMS protocol. The following Java code will quickly publish the message to the queue:


When we sniff the traffic to the Artemis instance using Wireshark we’ll see the following packet data is sent:

Well, the following part looks familiar:


We can see that the actual content of the serialized object starts with the hexadecimal characters AC ED 00 05.

In Java you will find serialization in a lot of implementations and features.

Insecure deserialization in Java

In Java you can implement a readObject or writeObject method in the class to execute some extra code during deserialization and serialization. An example:


If a Message object is serialized, the writeObject method will be called. The readObject method will be called when the object is deserialized.

Let’s create a new Answer class:

We end up with two files; my_message.ser and my_answer.ser.
What will happen if we expect a Message object during deserialization but we provide serialized Answer data?
If we run the code we see the following output in the console:
So what does this mean? We see that we got an exception about our cast, we tried to cast a Answer to a Message which threw an error. But.. we also see that the readObject method for the Answer class is executed, even if we didn’t do anything with Answer in our reading code.
This can be dangerous if we have classes implementing the Serializable interface and the readObject method but run some code which can lead to problems.

Gadgets

Classes which implement the Serializable interface and execute some code in the readObject method which can lead to problems are called “gadgets”. There are a few common known gadgets which are mostly part of some commonly used libraries, for example Apache’s Commons Collections library, Spring Core, Spring Boot and Hibernate. They caused vulnerabilities in some big applications, like Jenkins, JBoss and Websphere.

Exploit it!

A great tool to generate a payload is ysoserial. This tool is released as part of a talk at the AppSecCali conference in 2015. I used ysoserial to generate a serialized instance of the java.util.PriorityQueue queue which uses the TransformingComparator and InvokerTransformer class of the commons-collection4 library. Eventually, after some invokes and reflection magic, the payload will execute the Linux command touch powned, which will create a new file called powned to the file system.
I saved the serialized object as my_message.ser file and if I deserialize it with my reading example, the powned file is created on the file system.

But what if our application is using JMS and we want to run our malicious code on the server?
Let’s say our server consumes messages from a Artemis queue and prints them to the console.

If we have access to the Artemis queue, we could create our own client application and publish our malicious payload to the messages queue.

Once the server will pick up our malicious object in the queue the file powned is created on the server. Our malicious code is executed on the server.

 

What about other ways of serialization?

JSON

Serialization is implemented in almost every programming language and there are a lot of ways and libraries available.
An example of serialization we see almost every day is JSON. With JSON it’s possible to create a structured and readable presentation of data objects. JSON is most of the time used in HTTP requests and responses but you can also find it in configuration files and internal communication between services. An example of our Message object in JSON will look like this:

JSON in Java with Jackson Databind

A famous Java library used for JSON serialization is called Jackson. With the extension jackson-databind it’s possible to quickly translate your Java objects into JSON and JSON back into objects. Since 2017 there were found a lot of “gadgets” which could lead to problems using Jackson Databind with some particular setup.
Jackson can be insecure if you are using enableDefaultTyping or @JsonTypeInfo combined with global types like Serializable or Object.
Let’s extend our Message class with a collection of answers.

You see that I used a Collection of Serializable objects here. Probably not the first idea if you have to implement it, but it might be possible that you don’t know the type of the items and you have to be a bit general.
We use Jackson with the enableDefaultTyping option to serialize this to JSON:

The JSON representation will be:

You probably noticed the problem here; the Java class ends up in the JSON because Jackson can’t guess it when we want to deserialize it.
Let’s use one of the famous gadgets and build our own JSON structure:

In this case we use a famous gadget com.sun.rowset.JdbcRowSetImpl and use a URL to our malicious LDAP JNDI service running at ldap://attackers-host.com:1389/obj.
To let the server (my laptop in this case) run our custom malicious code we create a new class:

This class will start the Calculator app on my MacBook when a new object is constructed.
Now we compile this class and host it on our server. We make sure that the compiled .class file is available with http://<our host>/exploit/JNDIExploit.class.
We now can use the tool marshalsec to host a malicious LDAP JNDI server.
I used localhost:8000 which is a webserver on my local machine.
Once we send in the crafted JSON with ldap://localhost:1389/obj as dataSourceName Jackson will try to construct the JdbcRowSetImpl object which will call our JNDI server on construction. The JNDI server will redirect back to http://localhost:8000/exploit/JNDIExploit.class and the class hosted on our webserver will be constructed. In the constructor of our class we implemented the malicious code with Runtime.getRuntime().exec() which will be executed directly. So, the Calculator application pops up!
In this example I used a older version of Jackson. Most of these gadgets are fixed. Jackson implemented a blacklist of classes which are blocked and the use of enableDefaultTyping is deprecated. This blacklist of gadgets is growing during the last years, you can check GitHub for the full list and history. Make sure you always use the latest versions of libraries!

More serialization

In this article I gave a few examples for Java but serialization is also used in other programming languages.

PHP

In PHP you can use the serialize and unserialize methods which converts objects to a text notation. The type ends up in the textual notation of the object. Make sure you avoid the use of serialize and unserialize for objects which implement vulnerable code in magic methods like __destruct, __sleep, __wakeup, etc. These magic methods will automatically be called when you serialize or deserialize objects.

NodeJS
In NodeJS you have to be careful with the node-serialize package. With this package you can also serialize functions. An example of a serialized Javascript object is:

So if you unserialize user input without any validation, a malicious user can inject functions which can be executed by your NodeJS server.

YAML
Another commonly used format is YAML. Like JSON there are a lot of libraries which can serialize and deserialize objects into YAML. Some of these libraries have some interesting features.

For example the Java library SnakeYAML. You can use a special notation to create custom Java objects during the YAML parsing.

This will create the list with actual Answer objects. Like with Java serialization and Jackson Databind, there is a list of known “gadgets” which execute code during construction and are part of a lot of other libraries and frameworks.
You can find more about this in the marshalsec GitHub project.
SnakeYAML is the Java implementation of the Python library PyYAML. So the same features also exist in PyYAML.

About the author

I’m Jordy and I’ve worked at Sqills for 6 years now. I’m a software developer in the platform tools development team and a certified ethical hacker in the ‘Sqills Red Team’. As the Sqills Red Team we’re trying to improve the security of our software, infrastructure, and processes by thinking like a hacker. We try to attack our own software and network and spot potential weaknesses.

 

Do you want to know more? Contact Jordy here!

 

Other technical blogs