Wanderview

Composable Object Streams

In my last post I introduced the pcap-socket module to help test against real, captured network data. I was rather happy with how that module turned out, so I decided to mock out dgram next in order to support testing UDP packets as well.

I almost immediately ran into a few issues:

  1. The dgram module does not implement a streams2 duplex API. It still provides an old-style “spew-stream”.
  2. I wanted to share code with pcap-socket for parsing ethernet frames and IP headers.
  3. I also wanted to implement some missing features such IP fragment reassembly. These types of features require operating across multiple packets and therefore fit the streaming model better than the simple utility function approach.

Using the basic rebuffering byte streams provided by the new streams2 API seemed problematic. For one, UDP packets are distinct and shouldn’t be summarily rebuffered. Also, I needed a way to extract packet header information and pass it along with the byte stream.

I was considering a couple ways to proceed when Raynos was kind enough to implement object streams.

This seemed to solve a lot of my problems. I could now turn off rebuffering and I could pass arbitrary objects around.

The new object mode, however, did create one new issue.

Now that streams are not just ordered bytes, how can I write general purpose composable streams other people could easily use? If every person uses their own object structure then it could be very difficult to put together separate stream modules in a useful way.

Over time I came up with an approach that seemed to work well for the network protocol domain. Essentially, I structured messages like this:

1
2
3
4
5
6
7
8
9
10
var msg = {
  data: buf,
  offset: 14,
  ether: {
    src: '01:02:03:04:05:06',
    dst: '06:05:04:03:02:01',
    type: 'ip',
    length: 14
  }
};

Each message is the combination of some binary Buffer data and additional meta-data. In this example I have an ethernet frame that’s been parsed off the start of the msg.data buffer.

After a full parsing chain:

1
2
3
4
5
6
var ipstream = new IpStream();
var udpstream = new UdpStream();
ipstream.pipe(udpstream);

ipstream.write(msg);
var out = udpstream.read();

The resulting out message might look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
var out = {
  data: buf,
  offset: 42,
  ether: {
    src: '01:02:03:04:05:06',
    dst: '06:05:04:03:02:01',
    type: 'ip',
    length: 14
  },
  ip: {
    src: '1.1.1.1',
    dst: '2.2.2.2',
    flags: {
      df: false,
      mf: false
    },
    protocol: 'udp',
    protocolCode: 17,
    offset: 0,
    id: 12345,
    length: 20
  },
  udp: {
    srcPort: 5432,
    dstPort: 52,
    dataLength: 500,
    length 8
  }
};

This lets us inspect all of the extracted information at the end of the processing pipeline.

This approach also lets different stream implementations work together. For example, the IpStream can inspect the msg.ether.type property provided by EtherStream to see if msg represents an IP packet or not.

This approach also allows streams to avoid stepping on each others toes. Both the EtherStream and IpStream produce src properties, but they don’t conflict because they are namespaced under ether and ip.

To help solicit feedback on this approach I started a gist that outlines the approach. If you’re interested or have an opinion please check it out. I’d love to know if there is a better, more standard way to build these sorts of object streams.

Oh, and I did finally implement the dgram mock object. See the pcap-dgram module for examples. Both it and pcap-socket are built on top of the new ether-stream and ip-stream. And ip-stream does indeed now support fragmentation reassembly.

All of these composable object streams are implemented using a new base class module called object-transform. It makes it fairly easy to write these kinds of transformations.

Of course, this still leaves me with my first issue. The dgram core module still does not provide a streams2 API. If this message structure makes sense, however, it should now be possible to provide this API without losing the rinfo meta-data provided in the current 'message' event. UDP wouldn’t benefit from the back pressure improvements, but this would allow dgram to easily pipe() into other composable stream modules.

Again, if you an opinion on if this is useful or how it can be improved, please comment on the gist or send me a tweet.

Thank you!

Writing Node.js Unit Tests With Recorded Network Data

Automated unit tests are a wonderful thing. They help give you the confidence to make difficult changes and are your first line of defense against regressions. Unfortunately, however, unit tests typically only validate code against the same expectations and pre-conceived notions we used to write the code in the first place. All to often we later find our expectations do not match reality.

One way to minimize this problem is to write your tests using real world data. In the case of network code, this can be done by recording traffic to a pcap file and then playing it back as the input to your test.

In this post I will discuss how to do this in node.js using the pcap-socket module.

Module Overview

There are currently a couple of options for accessing pcap data in node.js:

  • The pcap module provides a wrapper around the native libpcap library.
  • The pcap-parser module provides a pure JavaScript implementation.

In both cases, however, the data is provided in a raw form which still includes the Ethernet frame, the IP header, and the TCP header. The code to be tested, however, probably expects data as provided by net.Socket with all of these headers stripped. This makes it awkward and tedious to write tests against pcap files.

To address this problem I’ve written a socket compatibility wrapper around pcap-parser called pcap-socket. This wrapper implements just enough logic to parse and strip the network headers. Data is then provided to your test code using the same API provided by net.Socket.

Creating a pcap-socket requires the path to the pcap file and an IP address:

1
2
3
var PcapSocket = require ('pcap-socket');
var file = path.join(__dirname, 'data', 'http-session-winxp.pcap');
var psocket = new PcapSocket(file, '10.0.1.6');

The IP address is required in order to tell pcap-socket which end of the recorded session you would like to pretend to be. Any data sent to this address in your file will be treated as data to be delivered by the socket.

This handles the incoming side, but what about writing data out?

In this case the packets originating for your configured address in the pcap file will be ignored. We are less interested in how the real server responded during your recording than how the code under test responds.

Therefore, data written to the pcap-socket gets placed in a separate output stream. This lets you write tests that examine and validate responses for correctness.

This might be more clear with some pictures.

The following diagram represents the logical flow of data using a real net.Socket object.

In contrast, the pcap-socket configuration looks like this:

Here is an example of using the output stream to validate your code’s response. Note, this uses the new streams2 API, but you can also use the more traditional on('data') API as well.

1
2
3
4
5
6
7
8
9
10
11
12
13
psocket.output.on('readable', function() {
  var chunk = psocket.output.read(156);
  if (chunk) {
    var str = chunk.toString();

    test.ok(str.match(/HTTP\/1.1 200 OK/));
    test.ok(str.match(/Content-Type: text\/plain/));
    test.ok(str.match(new RegExp(msg)));

    test.done();
  }
});
psocket.output.read(0);

Record Your Network Data

Now that we’ve covered the basic pcap-socket concepts, the next step is to record some network data in the pcap file format. If you are already comfortable with network monitoring tools, you may wish to skip to the next section.

The easiest way to do this is either with the UNIX command line tool tcpdump or with the graphical wireshark application. In either case, you first need to set up the monitoring tool with a filter matching the type of data you want to collect.

For example, in tcpdump you might do the following to record HTTP sessions from a particular host.

1
sudo tcpdump -i en1 -w data.pcap port 80 and host 10.0.1.12

Note that sudo is required since putting the network interface into promiscuous mode requires administrator privileges. Also, you typically must specify the correct network interface using the -i option. Here I am specifying my MacBook’s wireless interface.

Once the filter is running, perform whatever actions you need to in order to trigger the network traffic. This could be using the browser to hit a web page, executing a query against a database, etc.

Make sure to use save the results to a file using the -w data.pcap option in tcpdump or Save As in wireshark.. You can then replay the data at any time with this command:

1
tcpdump -r data.pcap

If you end up with more data in your file then you would like, you can specify a more strict filter and write the data out again to a second file.

1
tcpdump -r data.pcap -w data2.pcap host 10.0.1.6

Ideally you should aim to have the minimum amount of data in your file required to represent a real-world instance of the situation you want to test.

Write Your Unit Test

Your unit test will typically need a few standard sections:

  • Create the pcap-socket from the pcap file.
  • Write some response validation code that reads from the output stream.
  • Pass the pcap-socket to your code in some way. Hopefully there is an easy way to introduce the pcap-socket in place of net.Socket. You may need to get creative here or refactor the code to support this.

As an example, lets test everyone’s first node.js program; the simple hello world web server:

1
2
3
4
5
// Setup an HTTP server to test
var server = http.createServer(function(req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
});

Most of this test is actually shown in snippets above, but I will repeat them here for clarity.

First, create the pcap-socket:

1
2
var file = path.join(__dirname, 'data', 'http-session-winxp.pcap');
var psocket = new PcapSocket(file, '10.0.1.6');

Here the pcap file is stored in the file './data/http-session-winxp.pcap'. I used tcpdump to examine the file and determine that the web server was bound to the IP address 10.0.1.6.

Next, write code that validates the response that should occur.

1
2
3
4
5
6
7
8
9
10
11
12
13
psocket.output.on('readable', function() {
  var chunk = psocket.output.read(156);
  if (chunk) {
    var str = chunk.toString();

    test.ok(str.match(/HTTP\/1.1 200 OK/));
    test.ok(str.match(/Content-Type: text\/plain/));
    test.ok(str.match(new RegExp(msg)));

    test.done();
  }
});
psocket.output.read(0);

Finally, to supply the pcap-socket to the HTTP server we take advantage of the fact that it internally listens for the 'connection' event in order to begin processing a new session.

1
server.emit('connection', psocket);

All together the test looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
var PcapSocket = require('pcap-socket');

var http = require('http');
var path = require('path');

module.exports.http = function(test) {
  test.expect(3);

  var msg = 'Hello World\n';

  // Setup an HTTP server to test
  var server = http.createServer(function(req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end(msg);
  });

  // Configure the pcap socket to provide real, recorded network data
  var file = path.join(__dirname, 'data', 'http-session-winxp.pcap');
  var psocket = new PcapSocket(file, '10.0.1.6');

  // When the server sends back a response, validate that it makes sense
  psocket.output.on('readable', function() {
    // Read the full response; length determined by looking at pcap file
    var chunk = psocket.output.read(156);
    if (chunk) {
      var str = chunk.toString();

      test.ok(str.match(/HTTP\/1.1 200 OK/));
      test.ok(str.match(/Content-Type: text\/plain/));
      test.ok(str.match(new RegExp(msg)));

      test.done();
    }
  });
  psocket.output.read(0);

  // Supply the pcap socket to the HTTP server as a new connection
  server.emit('connection', psocket);
};

A More Complex Example

The HTTP example is a good start, but ideally it would be nice to test a more complex case that spans a longer TCP session.

The following code tests the netbios-session module against a pcap recording between my scanner and a Windows XP virtual machine. It validates the request, positive response, and subsequent message stream.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
var Session = require('../session');

var NBName = require('netbios-name');
var PcapSocket = require('pcap-socket');
var path = require('path');

var FILE = path.join(__dirname, 'data', 'netbios-ssn-full-scanner-winxp.pcap');
module.exports.server = function(test) {
  test.expect(7);
  var psocket = new PcapSocket(FILE, '10.0.1.12');

  var session = new Session();

  // validate expected request
  session.attach(psocket, function(error, request) {
    test.equal(null, error);
    test.equal('VMWINXP', request.callTo.name);
    test.equal('PRINTER', request.callFrom.name);
    request.accept();
  });

  // validate that we receive all the expected bytes in the session
  var length = 0;
  session.on('message', function(msg) {
    length += msg.length;
  });

  // validate positive response
  psocket.output.on('readable', function() {
    var chunk = psocket.output.read(4);
    if (chunk) {
      test.equal(0x82, chunk.readUInt8(0));
      test.equal(0, chunk.readUInt8(1));
      test.equal(0, chunk.readUInt16BE(2));
    }
  });
  psocket.output.read(0);

  // validate session completes properly
  session.on('end', function() {
    test.equal(438, length);
    test.done();
  });
};

Making Progress on Personal Projects

Sometimes I wonder how many people consider personal projects, but never even get started because they feel they don’t have the time? How many amazing and interesting ideas lie unrealized due to day-to-day commitments?

While my project is neither amazing nor all that interesting, I too spent a period of time thinking I just didn’t have the time to work on it. My wife and I happily welcomed our daughter into this world just last September. Ever since, our lives have been a whirlwind of activity that is very rewarding, but also leaves us exhausted at the end of the night. Clearly I could not take on a new endeavor now.

I began thinking, however, about something I heard Robert Egger say during a talk at Georgetown. He has been a local fixture in the DC non-profit community for years through his DC Central Kitchen. He spoke about effecting change through “relentless incrementalism”. Every day do a little bit better.

While my project is not as ambitious as his, I thought this might help me get started. To that end, I came up with the following rule for myself:

Make at least one productive commit each night.  This commit does not need
to be large or substantial.  Even something as small as cleaning up stale
comments or the TODO list is adequate.

On the face of it, this may seem like a trivial and insignificant rule. After all, you’re not going to make much progress tweaking comments for days on end.

My experience, however, has been that this rule is just the right amount of motivation to get me to open the editor at the end of a long day. Then, once I’m in the code, I will typically end up working on a larger feature instead. My fifteen-minute commit turns into a two-hour coding session that makes significant progress.

So far, I feel this has been quite successful for me. Looking at my local git log, I made the initial commit on November 17, 2012. For over two months now I’ve made slow, but real progress each day.

Of course, has spending all this time writing code at night increased my stress or taken away from my family time?

After thinking about it, I don’t believe so. For the most part I work at night after both my wife and daughter are asleep. If anything, it has replaced my time spent watching reruns of Buffy the Vampire Slayer and playing Diablo 3.

In terms of stress, I’ve actually found the personal project to be rather energizing. This should not be surprising given the research that creative workers feel the most satisfaction from a sense of progress. Daniel Pink also does an excellent job describing this effect in his book Drive.

The feeling that I’ve accomplished something far outweighs the temporary entertainment of TV or the false treadmill of most video games.

So the next time you feel like you don’t have the time for that cool project you want to build, consider just making one small commit each night. You’ll be surprised how far it can take you.

Naming the Project: FileShift

My last post described some basic NetBIOS operations using node.js. This was the first step towards my larger project goal of integrating my scanner with DropBox. At the end of the post, however, I realized I didn’t have a good place to keep that final script pulling all my npm modules together.

To resolve this, I’ve decided to go ahead and give the overall project a GitHub repository. As a side benefit, the project also has a name besides “the project”.

From now on it will be called FileShift.

While I plan to build out the underlying infrastructure in separate npm modules, the core logic will end up here. While its obviously very early in the effort, ultimately I’d like to see this project support translating between as many different file system protocols as possible.

If nothing else, this should keep me busy for a while.


The NetBIOS proxy script from my last post can be found here.

Working With NetBIOS in node.js

Lately I’ve been caught in the middle of a dispute between my Xerox scanner and Mac OS X. The Mac only wants to use modern versions of SMB to share files and is giving the scanner the cold shoulder. In an attempt to mediate this issue I’ve turned to hacking on ancient protocols in node.js.

So far I’ve tackled NetBIOS and thought I would share some of the code I’ve come up with.

The first step to finding or advertising a NetBIOS name is to start up a netbios-name-service instance.

1
2
3
4
5
6
var NBService = require('netbios-name-service');

var nameService = new NBService();
nameService.start(function() {
  console.log('NetBIOS name service started');
});

Then, if you want to search for a service, like say a printer, you can execute the following code.

1
2
3
4
5
6
var NBName = require('netbios-name');

var queryName = new NBName({name: 'PRINTER'});
nameService.find(queryName, function(error, address) {
  console.log('Found NetBIOS name [' + queryName + '] at [' + address + ']');
});

Note, you must use the netbios-name module in order to properly define names in order to search, advertise, or perform other operations. This module provides a simple class that combines the simple NetBIOS name with additional information such as the scope ID (aka domain name) and a suffix byte indicating the what type of node is being represented.

For the problem at hand, however, I don’t need to search for a name. Instead I want to advertise the node.js server via NetBIOS so that the scanner can push files to us.

1
2
var myName = new NBName({name: 'XYKON-2'});
nameService.add({nbname: myName});

This causes the service to monitor UDP port 137 for broadcast queries looking for the name XYKON-2. If a query occurs, then the service will automatically respond with the IP address of the server.

So this allows the scanner to find our server, but what about handling the NetBIOS traffic containing the actual file operations?

To deal with this part of the problem we need to use the netbios-session module. Here is code to receive incoming NetBIOS sessions.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
var net = require('net');
var NBSession = require('netbios-session');

var server = net.createServer(function(socket) {
  var sessionIn = new NBSession({autoAccept: true});

  sessionIn.on('message', function(msg) {
    console.log('NetBIOS session message with [' + msg.length + '] bytes');
  });

  sessionIn.on('connect', function() {
    console.log('New NetBIOS session from [' socket.remoteAddress + ']');
  });

  sessionIn.attach(socket);
});

server.listen(139);

The netbios-session class is essentially a wrapper around a TCP socket. After calling new to create a new instance, you need to call connect() or, in this case, attach(). Here we are using attach() to associate the session with a new TCP socket.

Once the session is ready to send and receive data it will emit the 'connect' event. At this point 'message' events will occur whenever a message is received from the remote peer. Messages can be sent using the write() method.

So, now that we are receiving data from the scanner, we need to forward this on to my Mac’s SMB service listening on port 445.

Now, it turns out, direct SMB connections to port 445 actually use the NetBIOS session header to implement message framing, but it skips all of the initial session negotiation that normally occurs.

So, to achieve our forwarding we want to create a second session to port 445 using the 'direct' constructor option.

1
2
var sessionOut = new Session({direct: true});
sessionOut.connect(445, '127.0.0.1');

After this, we can take messages we receive from sessionIn events and pass them straight to 'sessionOut.write()'. Since SMB is bidirectional, we also need to pass messages in the reverse direction. With a bit of back-pressure logic sprinkled in, this code looks like the following:

1
2
3
4
5
6
7
8
9
10
sessionIn.on('message', _forward.bind(null, sessionOut, sessionIn));
sessionOut.on('message', _forward.bind(null, sessionIn, sessionOut));

function _forward(dst, src, msg) {
  var flushed = dst.write(msg);
  if (!flushed) {
    src.pause();
    dst.once('drain', src.resume.bind(src));
  }
}

Putting all the pieces together we end up with the following.

netbios-fwd.jsSource File
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
'use strict';

var NBService = require('netbios-name-service');
var NBSession = require('netbios-session');
var NBName = require('netbios-name');
var net = require('net');

var NAME = 'XYKON-2';
var SCOPE_ID = 'example.com';
var FWD_PORT = 445;
var FWD_HOST = '127.0.0.1';

var server = net.createServer(function(socket) {
  var sessionIn = new NBSession({paused: true, autoAccept: true});

  sessionIn.on('connect', function() {
    var sessionOut = new NBSession({direct: true});

    var endHandler = function() {
      sessionIn.end();
      sessionOut.end();
    };
    var errorHandler = function(error) {
      console.log(error);
      endHandler();
    };

    sessionIn.on('end', endHandler);
    sessionOut.on('end', endHandler);

    sessionIn.on('error', errorHandler);
    sessionOut.on('error', errorHandler);

    sessionIn.on('message', _forward.bind(null, sessionOut, sessionIn));
    sessionOut.on('message', _forward.bind(null, sessionIn, sessionOut));

    sessionOut.on('connect', sessionIn.resume.bind(sessionIn));

    sessionOut.connect(FWD_PORT, FWD_HOST);
  });

  sessionIn.attach(socket);
});

function _forward(dst, src, msg) {
  var flushed = dst.write(msg);
  if (!flushed) {
    src.pause();
    dst.once('drain', src.resume.bind(src));
  }
}

server.listen(139);

var nameService = new NBService();
nameService.start(function() {
  var myName = new NBName({name: NAME, scopeId: SCOPE_ID, suffix: 0x20});
  nameService.add({nbname: myName});
});

We now have a fully functional proxy server for connecting devices that only speak NetBIOS up to modern, direct SMB servers.

Of course, this assumes that the underlying SMB protocol is compatible with the destination. It turns out, the new Mac OS X SMB server has some additional restrictions. While I can connect through the proxy using net use on Windows XP, my scanner still fails to connect.

Looking in the log I found this:

1
smbd[91973]: 127.0.0.1 SMB client not supported - Unicode strings are required

Unfortunately this implementation is not even a temporary work around for my problem since the scanner doesn’t know how to talk Unicode.

So, it looks like I will be diving into the SMB protocol next so that I can intercept the file operations directly in node.


All the modules used in this post are available on GitHub and npm:

The end script is available on Github in the FileShift project.

Time Machine and Npm

Do you develop npm modules on Mac OS X? Do you also use Time Machine to backup your work?

If you answered yes to both those questions, then there is a good chance you will be looking at this dialogue box at some point.

I ran into this problem a couple weeks ago. At first I thought my disk was going bad, but all the Disk Utility checks ran fine. Eventually I ended up looking in the log and found this gem:

Jan 15 22:25:37 xykon-2 com.apple.backupd[38289]: Error: (-36) SrcErr:NO Copying /Users/bkelly/Dropbox/devel/node-netbios-session/node_modules/netbios-name/name.js to /Volumes/xykon backup/Backups.backupdb/xykon (2)/2013-01-15-222534.inProgress/9C56F536-65EE-46B1-8EF6-481D98533409/Corsair SSD/Users/bkelly/Dropbox/devel/node-netbios-session/node_modules/netbios-name

Its quite long, but if you look closely you can see that its complaining that it can’t copy a file from /usr/local/lib/node_modules to a node_modules directory in my development area. In particular, it was complaining about code that I recently had been referencing using npm link.

For those unaware, npm link is a handy command that lets you work with npm modules locally instead of installing them from the repository online. It does this by creating symlinks in /usr/local/lib/node_modules/ and the node_modules directory within a module you are working on. These symlinks point to another local module that may have edits that you have not yet published.

Of course, when you are done you may run npm unlink so that you can npm install the module from the repository again. This is where the problem can occur.

Unfortunately, it appears that Time Machine will sometimes get upset if you backup a symlink to a directory, convert the symlink back to a real directory, and then try to backup again.

Indeed, after I knew to search for symlink related problems, Google revealed that this problem is not new.

As mentioned in those other blogs, the solution to the problem is to delete the problematic directories, run another backup, and then replace the data. Fortunately, npm makes this fairly straightforward to do.

You can also add /usr/local/lib/node_modules to the Time Machine exclusions to try to proactively avoid problems.

tmutil addexclusion /usr/local/lib/node_modules

In fact, since the Time Machine exclusion is an attribute on the directory itself, its in theory possible to enhance npm or write a wrapper script to automatically set the exclusion whenever a new node_modules directory is created.

Of course, you may be saying to yourself that you do this all the time and have never had a problem. Well, just to make things fun, it appears that Time Machine only has problems with this situation periodically. In fact, I tried duplicating the error before writing this blog post and was unable to make it happen. Its unclear what other circumstances are required to trigger the condition.

So your workflow may work for some time and then seemingly break at random. Just be aware that it may be npm link related and try removing your node_modules before scrapping the backup disk.

Xerox + Apple === Node.js

In 2010 my wife and I decided to buy a fancy all-in-one, networked, laser printer and scanner. We were tired of the long, onerous tasks of cleaning ink cartridges and walking over to the scanner to plug in our laptops. A person can only take so much.

After extensive research we settled on a Xerox Phaser 6128MFP. It was a color printer with built-in ethernet. Scanning wrote files directly to a shared drive or sent them over email. It wasn’t cheap, but clearly this was the device for us. After a few minutes on Newegg it was ours.

For nearly a year, all was well. We scanned. We printed. A few times in color at first, to prove that it worked, and then mostly in black-and-white so as not to be wasteful.

Then in August, 2011 we decided to upgrade our laptops. Snow Leopard was clearly not fierce enough and we needed to move to Lion. I had waited a full month after release to upgrade in an attempt to avoid major problems. There were a few complaints online, but overall it seemed like a safe upgrade.

Of course, after installing Lion we ran into problems. Low and behold, Apple had chosen to discontinue support for the NetBIOS session protocol including SMB over NetBIOS. Its as if they thought a protocol from 1987 wasn’t good enough any more. This broke networked scanning for many people, including us.

No problem, I thought, I’ll just switch to the email delivery method instead. I soon discovered, however, that the Xerox firmware only supports sending email over unencrypted SMTP. Besides the obvious security concerns, this would not work because both my wife and I use gmail which requires SSL/TSL.

This left us with only a few solutions:

  1. Try configuration changes reported to have varying levels of success.
  2. Install and configure Samba on both our laptops.
  3. Setup a linux server on our home network to server as an SMTP MTA or file server.
  4. Walk over to the scanner and plug the USB cable into the laptop.

After careful consideration I have chosen to try something completely different:

I’m now working on a Node.js server to receive scanned files over NetBIOS/SMB and push them up to Dropbox.

Ultimately I’d like to host this server on an embedded device like a Raspberry Pi, Soekris, or Gumstix. To make it easier to deploy on these platforms I’m working towards a pure JavaScript implementation.

While not the most direct solution, it has a number of benefits:

  • Provides a permanent solution that is uncoupled from our chosen laptop operating systems.
  • Gives me the opportunity to learn more about node.js, JavaScript, and some new (to me) protocols. I’d rather learn about these things than configuration files.
  • Allows me to contribute some code and get more involved with the node.js community.
  • Could potentially provide a foundation for integrating any number of legacy devices and applications. There are undoubtedly small business and enterprise IT shops trying to figure out how to tie their aging, legacy equipment into fancy new cloud services. Perhaps this code will be helpful.

I figure as long as I am starting a project, I might as well start a blog as well. While all the code is hosted on GitHub, I’ll periodically post more updates on my progress here. The next post will be coming soon and will cover proxying NetBIOS sessions using node.js.