Writing Node.js Unit Tests With Recorded Network Data
Automated unit tests are a wonderful thing. They help give you the confidence to make difficult changes and are your first line of defense against regressions. Unfortunately, however, unit tests typically only validate code against the same expectations and pre-conceived notions we used to write the code in the first place. All to often we later find our expectations do not match reality.
One way to minimize this problem is to write your tests using real world data. In the case of network code, this can be done by recording traffic to a pcap file and then playing it back as the input to your test.
- The pcap module provides a wrapper around the native libpcap library.
In both cases, however, the data is provided in a raw form which still includes the Ethernet frame, the IP header, and the TCP header. The code to be tested, however, probably expects data as provided by net.Socket with all of these headers stripped. This makes it awkward and tedious to write tests against pcap files.
To address this problem I’ve written a socket compatibility wrapper around pcap-parser called pcap-socket. This wrapper implements just enough logic to parse and strip the network headers. Data is then provided to your test code using the same API provided by net.Socket.
1 2 3
The IP address is required in order to tell pcap-socket which end of the recorded session you would like to pretend to be. Any data sent to this address in your file will be treated as data to be delivered by the socket.
This handles the incoming side, but what about writing data out?
In this case the packets originating for your configured address in the pcap file will be ignored. We are less interested in how the real server responded during your recording than how the code under test responds.
Therefore, data written to the pcap-socket gets placed in a separate
output stream. This lets you write tests that examine and validate
responses for correctness.
This might be more clear with some pictures.
The following diagram represents the logical flow of data using a real net.Socket object.
In contrast, the pcap-socket configuration looks like this:
Here is an example of using the
output stream to validate your code’s
response. Note, this uses the new streams2 API, but you can also use
the more traditional
on('data') API as well.
1 2 3 4 5 6 7 8 9 10 11 12 13
Record Your Network Data
Now that we’ve covered the basic pcap-socket concepts, the next step is to record some network data in the pcap file format. If you are already comfortable with network monitoring tools, you may wish to skip to the next section.
The easiest way to do this is either with the UNIX command line tool tcpdump or with the graphical wireshark application. In either case, you first need to set up the monitoring tool with a filter matching the type of data you want to collect.
For example, in tcpdump you might do the following to record HTTP sessions from a particular host.
sudo is required since putting the network interface
into promiscuous mode requires administrator privileges. Also, you
typically must specify the correct network interface using the
option. Here I am specifying my MacBook’s wireless interface.
Once the filter is running, perform whatever actions you need to in order to trigger the network traffic. This could be using the browser to hit a web page, executing a query against a database, etc.
If you end up with more data in your file then you would like, you can specify a more strict filter and write the data out again to a second file.
Ideally you should aim to have the minimum amount of data in your file required to represent a real-world instance of the situation you want to test.
Write Your Unit Test
Your unit test will typically need a few standard sections:
- Create the pcap-socket from the pcap file.
- Write some response validation code that reads from the
- Pass the pcap-socket to your code in some way. Hopefully there is an easy way to introduce the pcap-socket in place of net.Socket. You may need to get creative here or refactor the code to support this.
As an example, lets test everyone’s first node.js program; the simple hello world web server:
1 2 3 4 5
Most of this test is actually shown in snippets above, but I will repeat them here for clarity.
First, create the pcap-socket:
Here the pcap file is stored in the file
'./data/http-session-winxp.pcap'. I used
tcpdump to examine the file
and determine that the web server was bound to the IP address
Next, write code that validates the response that should occur.
1 2 3 4 5 6 7 8 9 10 11 12 13
Finally, to supply the pcap-socket to the HTTP server we take advantage
of the fact that it internally listens for the
'connection' event in order
to begin processing a new session.
All together the test looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
A More Complex Example
The HTTP example is a good start, but ideally it would be nice to test a more complex case that spans a longer TCP session.
The following code tests the netbios-session module against a pcap recording between my scanner and a Windows XP virtual machine. It validates the request, positive response, and subsequent message stream.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44