The road ahead: Protocols

Direct Connect was started with the protocol “Neo-Modus Direct Connect” (NMDC). This was named after the only client and hub available at the time. Over time, this protocol grew as more client and hub developers followed. The protocol was initially sniffed out by various people as the original system was closed-source. Over the years, the client and hub developers grew and discussions commenced that the protocol had become unmaintainable. The protocol was considered bad in various aspects and the request for a new protocol was underway.

The initial discussion was whether the “new” protocol should be binary or text based: a binary is less resource intensive as you put much more care into what is being sent, while a text protocol (like NMDC) is easier to read and implement.

The discussion eventually came down to a “my client has the most users, so here’s my protocol and I’ll implement it in my client” call from Jacek Sieka, the DC++ author. The new protocol was called ADC, sometimes referred to as “Advanced Direct Connect”, while this has never been its official name. ADC took the same fundamental structure of Direct Connect as NMDC did, but with the intent of increased usability and extensability. A few of ADC’s aspects can even be traced back to the protocol suggestion (made around the same time) called “Direct Connct The Next Generation” (DCTNG), as is noted from ADC’s website.

ADC eventually grew and there’s now “competition” between NMDC and ADC (although I believe NMDC is still ‘winning’ in user counts).

There are now various resources that people can use to implement and support their own component for the NMDC and ADC protocols, although not enough in my mind.

The tools

There are, as of writing, very few tools for supporting protocol developers.

The tool Wireshark can provide a tremendous support in filtering what information is actually being sent on the network. Effectively, if you didn’t have the specification(s), you could create your own implementation by simply looking at the network traffic. Wireshark uses plugins that are protocol-specific sniffing implementations. However, no plugin has been fully implemented for either NMDC or ADC. The ADC Wireshark plugin attempts to do just this, but it isn’t complete (at the moment you have to compile things on your own etc). Having a plugin (for either protocol) could provide an excellent opportunity for developers to learn the protocol’s raw nature. There’s probably other similar types of applications like Wireshark, but it is probably the most widely known and used tool for providing network information.

There are few ways of actually testing your NMDC and ADC implementation. As a client developer, you need to connect to a “normal” hub and see if that hub accept your data and see if other clients can see your data. As a hub developer, you need to take “normal” clients and connect it to the hub and see if you properly route traffic. This means that while we can point to the specifications as the reference, most often developers need some form of software to actually verify their application. The proper tool for doing this would be a full reference implementation of a client or hub. This implementation doesn’t have to be fast or be able to handle lots of data. It should provide the basic protocol implementation together with a list of things it should send out to test the application (like sending out faulty data to verify that the system handles it). Ideally in my mind, a public hub should be set up to act as a reference implementation — that way you must manage connections that didn’t originate from your own computer or LAN.

While a reference implementation is the way to go, the next step is to have something called “stress tester”. This is an application that pushes the software to its limits. For a hub, the stress tester could simulate hundreds or thousands of users, seeing whether the hub can cope with the information. For a client, the stress tester could simulate lots of search results, managing lots of searches and simulating lots of connection attempts. The stress tester could also include faulty data, but the key of the application is to test whether the underlying service is able to handle a huge amount of data.

While we can provide tools for those who decide to implement the protocols themselves, we should also strive after providing reference implementation code. That is, people shouldn’t have to re-write boring protocol implementations all the time: they should just be able to take an already created one and use that. The ADC code snippets is such an attempt and in the future my idea was to have further code such as hash or magnet implementations, in addition to the protocol specific implementation. The idea is to create a basic foundation for Direct Connect software, similar to how FlowLib is managed. Of course, having general code extends also to NMDC.

Discussion

The idea is to promote any type of venue where people can interact and discuss further protocol enhancements and issues. The DCBase forum (previously ADCPortal) intends to provide such a functionality. There’s also the FlexHub forum, which also have NMDC and ADC sections.

I am not sure about the use of a wiki, as I think much of the content can be written elsewhere in a better way, but I see no problem in having wikis that explain various implementations in more detail or the use of “workgroups”.

Regardless of the venue, it is best if we can create a service that gathers protocol issues and content. This blog has served as such an information pool in the past for various parts, but I’d not be sad if we could have a better place that would be easier to manage.

The previous DCDev archives provide a good picture of the early discussions about NMDC and ADC, and I’m sure there’s a couple of gems that should be discussed further or at least lifted. Not only is that old resource important, the resources we have today are also important: the developer hub can be an important source and the future might should be post any protocol discussion to a forum or alike so other can read the discussions.

Any type of document that describe the DC protocols (or any other incarnation of DC) should be made public.

Documentation

The ADC protocol is (well?) documented and the main document and partner (“extensions”) should hold its merits on their own. However, I believe it would be good if others provided suggestions how we can improve the specification to be easier to read and be less ambiguous. The intent of the “recommendations” document should also provide information that may not warrant inclusion in the main documents but can serve as good-to-know information as people are reviewing the protocol. There have also been suggestions for state diagrams over the protocol as it should provide better insight into the flow of information.

The NMDC protocol was never officially documented. The documentation that exist was scraped together by various people based on the behaviour of the protocol implementations. There today a few resources left, but I would like everyone to acknowledge the NMDC project as its intent is to provide the same level of information as the ADC project. While the specification for NMDC should provide information how implementations should behave, I believe it should also specify how implementations have done in the past (if they have deviated and how).

Down the line, perhaps an official RFC document is the target for both NMDC and ADC (one for each, obviously).

The road ahead

The NMDC and ADC protocol should increase in its documentation and the support that can be provided for their implementators. The tools should provide better support for developing something new as well as simply not having to do the implementation at all.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: