How to Write a Custom Protocol for Gatling?

I like to try out new tools and new things, so I decided to loadtest my newly written “simple chat” application to see how AkkaStreams handles load. Starting with load testing, there are some tools out there. The top three are JMeter, Gatling and Tsug. Tsug is written in erlang, and my half year long erlang course was nothing to show off, so I skipped it immediately. I used JMeter before, but I didn’t want to write a new plugin to it. Gatling stayed. It’s a nice loadtesting tool, written in Scala, and uses netty and akka. It has a not so hard DSL and I read somewhere that it’s easily extendable with self-written custom protocols. That last point was questionable because there is really minimal documentation. (I still have some questions but I reverse engineered the bare minimum for a new and simple protocol.)

writing a custom protocol for gatling

The docs and the sources

As I said, minimal documentation:

So I think this mini tutorial will be filling a gap (smile)

The preparations

I’m using IntelliJ, and sbt, so let’s start a new project, with sbt and scala version 2.11 (I tried it with 2.12 first but I think Gatling is not compatible with it yet).

We now init a simple testrunner class like this:

And a simple simulation like this (this is the test from the official documentation):

At this point, you can run the GatlingRunner’s main method from the IDE and this will run a simple test for you.

We will do one more class. A simple tcp server for testing our protocol.

It’s a really basic tcp server. Write some info to the output, and convert the incoming byteStrings to its upper counterpart for every incoming connection. (If you don’t understand the above code you can learn more from the AkkaStreams doc or the activator template.)

 

So we have a project where we have a testserver now, with basic implementation, and a GatlingRunner which can run (and debug) tests from our IDE.

gatling custom protocol

Our protocol

So this protocol will be the bare minimum. We will define our server’s endpoint, and one action called connect. It will connect, send, and recieve some data.

Our simulation will look like this:

(If you copy & paste this code it will be all red, but it’s fine (smile))

So start with the Predef.

If you read the code carefully, you can notice that this is the file which we import in our new simulation. This file is the definition for our protocol DSL. The val and the function will lead us to the protocol and action builder, while the implicit builderToProtocol is needed for the framework.

Our next stop is the Protocol:

I left these classes and objects in one file due to couple-line implementations, and because they are sticking together. However, it’s reasonable to split it into 2-3 files and take them to a package.

  • The class UpperProtocol is a minimal implementation of Protocol (we just add some parameters and define the type).
  • The UpperProtocol object has a factory method, and a ProtocolKey. The Protocol key is also a basic implementation. (No default value, the class is our ProtocolClass, and the newComponents is from the JMS implementation.)
  • I have no idea what the Components is for, but it’s needed for the ProtocolKey, and the JMS has the same implementation.
  • The ProtocolBuilder class build method (used by the predef) creates a new Protocol object
  • The ProtocolBuilder object summarizes the builder method

When you say upper.endpoint(…) in the simulation, the calls come from the Predef.upper (which is the ProtocolBuilder). The builder creates an object from the Builder class, which “build” method is implicitly given in the Predef. That’s why the system knows how to build and get the inicialized Protocol.

The Upper is a small class, it collects the reachable actions:

It has one action, connect, which creates a new ConnectActionuilder.

 

Our next (and almost last) stop is the Action:

The ConnectActionBuilder extends from the Action builder, and because of that, it needs to implement the build method. The builder should call the action itself. It’s a bit complicated at first, but because the protocol holds the connection data, the connect action needs to be able to read the actual protocol. From the builder, we can search our component, and can give – from the protocol – it down to the action. The statsEngine is basically the data holder and aggregator, if you want to log responses you must give it down too. The action itself could be anything. You can create actors, send messages, write or read values from-to the session, log responses, and so on. Our action creates a client, run it, measure the runtime, and log it down. The last line in the execute function is again a bit of blackmagic (smile) I think this replies back to the runner actor that the actual execution is ended, and this is our new session object.

The Client is a minimal implementation for feeding the server:

(It’s from the activator template, a basic tcp connector stream.) I think the Await is an antipattern in this context, but if I would write it with onComplete, I would need to give down the full parameter list of the statsEnine.logResponse + the StatsEngine itself + force the simulation to wait for the responses, so much noise for a simple piece of code. This is a blocking request for now, only for illustration (smile).

If you copypasted all the code from above to a project, you can run the server, then the simulation, and see some result on the output. Congrats! You just wrote your first custom protocol for Gatling!

 

The whole source is avaiable on github.

 

Some things I am still playing with:

  • What are the names? –  (smile) I have really no idea why I need to generate a name in the ConnectAction.
  • What is one file and what is not. – I think the Action and the ActionBuilder belong to one file, but if you have more than one Action, they need to be separated and archived to a package. I think the Protocol, ProtocolBuilder, and ProtocolComponent fit in one file too.
  • What if I build an actor system at the top of the Client, – I’m experimenting with a chat-like protocol, with async statsEngine logging.

 

Conclusion

If you want to write a new custom protocol to Gatling, you need to dig deep, and do some guessing, but I think this short tutorial will bring light to some questions. I still have some questions with the inner workings, but I think this post can save some hours for other people… like yourself. (smile)

 

Closing words

If you have any advise or proper knowledge in this field, please write a comment, and I will edit this post to became a better source. The code above is just an example and most of it is reverse engineered.. or is well educated guessing on my side. Any help or correction is appreciated.

 

 

Gergő Törcsvári

Gergő Törcsvári

Software Developer at Wanari
I would love to change the world, but they won’t give me the source code (yet).

  • sygour

    Thanks for the post, very helpful!
    Built a small and efficient (even if not optimized) RabbitMQ protocol with your help. Will share it at some point I think
    Just so you know, I built it with scala 2.12.1 and it worked properly

  • Pingback: steroids online()