Mixed-hash DC hubs

They work fine if clients and hubs support both TTH and its successor adequately long.

While transitioning to a TTH successor, currently interoperable clients and hubs all supporting only TTH will diverge. In examining the consequences of such diversity, one can partition concerns into client-hub communication irrelevant to other clients; hub-mediated communication between two clients; and direct client-client communication. In each case, one can look at scenarios with complete, partial, and no supported hash function overlap. Complete overlap defines the all-TTH status quo and, clearly, works without complication for all forms of DC communication, so this post focuses on the remaining situations. In general,

Almost as straightforwardly, ADC but not NMDC client-hub communication irrelevant to other clients requires partial but not complete hash function overlap but only between each individual client/hub pair, and don’t create specific mixed-hash hub problems; otherwise, an ADC hub indicates STA error code 47. For ADC, This category consists of GPA, PAS, PID/CID negotiation (with length caveats as relate to other clients interpreting the resulting CID), and the establishment of a session hash function; NMDC does not depend on hashing at all for analogous functionality. Thus, for NMDC, no problems occur here. ADC’s greater usage of hashing requires correspondingly more care.

Specifically, GPA and PAS require that SUP had established some shared hash function between the client logging in and the hub, but otherwise have no bearing on mixed-hash-function DC hubs. Deriving the CID from the PID involves the session hash algorithm, which as with GPA/PAS merely requires partial hash function support overlap between each separate client and a hub. Length concerns do exist here, but become relevant only with hub-mediated communication between two clients.

Indeed, clients communicating via a hub comprise the bulk of DC client-hub communication. Of these, INF, SCH, and RES directly involve hashed content or CIDs. SCH ($Search) allows one to search by TTH and would also allow one to search by TTH’s successor. Such searches can only return results from clients which support the hash in question, so as before, partial overlap between clients works adequately. However, to avoid incentivizing clients which support both TTH and its successor to broadcast both searches and double auto-search bandwidth, a combined search method containing both hashes might prove useful. Similarly, RES specifies that clients must provide the session hash of their file, but also “are encouraged to supply additional fields if available”, which might include non-session hash functions they happen to support, such that as with the first client-hub communication category, partial hash function support overlap between any pair of clients suffices, but no overlap does not.

A more subtle and ADC-specific issue issue arises via RES’s U-type message header and INF’s ID field whereby ADC software commonly checks for exactly 39-byte CIDs. While clients need not support whatever specific hash algorithm produced a CID, the ADC specification requires that they support variable-length CIDs. Example of other hash function output lengths which, minimally, should be supported include:

Bits Bytes Bytes (base32) Supporting Hashes
192 24 39 Tiger
224 28 45 Skein, Keccak, other SHA-3 finalists, SHA-2
256 32 52 Skein, Keccak, other SHA-3 finalists, SHA-2
384 48 77 Skein, Keccak, other SHA-3 finalists, SHA-2
512 64 103 Skein, Keccak, other SHA-3 finalists, SHA-2

Finally, direct client-client communications introduces CSUP ($Supports), GET/GFI/SND ($Get/$Send) via the TTH/ share root or its successor, and filelists, all of which work if and only if partial hash function support overlap exists. CSUP otherwise fails with error code 54 and some subset of hash roots and hash trees regarding some filelist must be mutually understood, so as with the other cases, partial but not complete hash function support overlap between any given pair of clients is required.

Encouragingly, since together client-hub communication irrelevant to other clients; hub-mediated communication between two clients; and direct client-client communication cover all DC communication, partial hash function support overlap between any given pair of DC clients or servers suffices to ensure that all clients might fully functionally interact with each other. This results in a smooth, usable transition period for both NMDC and ADC so long as clients and hubs only drop TTH support once its successor becomes sufficiently ubiquitous. Further, relative to ADC, poy has observed that “all the hash function changes on NMDC is the file list (already a new, amendable format) and searches (an extension) so a protocol freeze shouldn’t matter there”, which creates an even easier transition than ADC in NMDC.

In service of such an outcome, I suggest two parallel sets of recommendations, one whenever convenient and the other closer to a decision on a TTH replacement. More short-term:

  • Ensure ADC software obeys “Clients must be prepared to handle CIDs of varying lengths.”
  • Create an ADC mechanism by which clients supporting both TTH and its successor can search via both without doubling (broadcast) search traffic. Otherwise, malincentives propagate.
  • Ensure BLOM scales to multiple hash functions.
  • Update phrasing in ADC specification to clarify that all known hashes for a file should be included in RES, not just session hash.

As the  choice of TTH’s successor approaches:

  • Disallow new hash function from being 192 bits to avoid ambiguity with Tiger or TTH hashes. I suggest 224 or 256-bit output; SHA-2 and all SHA-3 finalists (including Keccak and Skein) offer both sizes.
  • Pick either a single filelist with all supported hashes or multiple filelists, each of which only supports one hash. I favor the former; it especially helps during a transition period for even a client downloading via TTH’s successor to be able to autosearch and otherwise interact with clients which don’t yet support the new hash function, without needing to download an entire new filelist.
  • Barring a more dramatic break in Tiger than thus far seen, clients should retain TIGR support until the majority of ADC hubs and NMDC or ADC clients offer support for the successor hash function’s extension.

By doing so, clients both supporting only TTH and both TTH and new hash function should be capable of interacting without problems, transparently to end-users, while over time creating a critical mass of new hash function-supporting clients such that eventually client and hub software might outright drop Tiger and TTH support.

A Decade of TTH: Its Selection and Uncertain Future

NMDC and ADC rely on the Tiger Tree Hash to identify files. DC requires a cryptographic hash function to avoid the previous morass of pervasive similar, but not identical, files. A bare cryptographic hash primitive such as SHA-1 did not suffice because not only did the files need identification as a whole but in separate parts, allowing reliable resuming and multi-source downloading, and per-segment integrity verification (RevConnect unsuccessfully attempted to reliably use multi-source downloading precisely because it could not rely on cryptographic hashes).

Looking for inspiration from other P2P software, I found that BitTorrent used (and uses) piecewise SHA-1 with per-torrent segment sizes. Since the DC share model asks that same hash function work across entire shares, this does not work. eDonkey2000 and eMule, with per-user shares similar to those of DC, resolved this with fixed, 9MB piecewise MD4, but this segment size scaled poorly, ensured that fixing corruption demanded at least 9MB of retransmission, and used the weak and soon-broken MD4. Gnutella, though, had found an elegant, scalable solution in TTH.

This Tiger Tree hash, which I thus copied from Gnutella, scales to both large and small files while depending on what was at the time a secure-looking Tiger hash function. It smoothly, adaptively sizes a hash tree while retaining interoperability between all such sizes of files files on a hub. By 2003, I had released BCDC++ which used TTH. However, the initial version of hash trees implemented by Gnutella and DC used the same hash primitive for leaf and internal tree nodes. This left it open to collisions, fixed by using different leaf and internal hash primitives. Both Gnutella and DC quickly adopted this fix and DC has followed this second version of THEX to specify TTH for the last decade.

Though it has served DC well, TTH might soon need a replacement. The Tiger hash primitive underlying it by now lists as broken due to a combination of a practical 1-bit pseudocollision attack on all rounds, a similarly feasible full collision on all but 5 of its 24 rounds, and full, albeit theoretical, 24-round pre-images (“Advanced Meet-in-the-Middle Preimage Attacks”, 2010, Guo et al). If one can collide or find preimages of Tiger, one can also trivially collide or find preimages of TTH. We are therefore investigating alternative cryptographic hash primitives to which we might transition as Tiger looks increasingly insecure and collision-prone, focusing on SHA-2 and SHA-3.

How to crash DC++ 0.674

$ADCGET list //// 0 -1 ZL1|

A previous blog post mentions this, but apparently isn’t sufficiently explicit about what to send.

I aim to fix that.

Enjoy, all. This apparently works on DC++ clients older than 0.707 which still support $ADCGET.

DC++ 0.75 and older vulnerable to bzip2 filelist bomb

DC++ 0.75 and earlier can be remotely crashed via either bzip2 filelists (example filelist) or hublists (themselves compressed with bzip2). Such list downloads can be automatically triggered by automatic searches for alternate sources, so explicit user action is unnecessary [1]. Not every client seems to crash; the precise dependence on operating system or other factors remains unclear. However, crashes have been observed using both Windows XP and Windows 7.

As before, updating network-facing software remains important. Equally importantly, DC++ mod authors should attempt to update in a timely manner such as to avoid exposing their users to this bug.

[1] To catch a large number of clients in a hub in a relatively short period of time with no manual intervention, listen for searches (especially TTH searches) and always respond positively, such that clients try autosearching for alternate sources. Other tricks are possible as well, of course.

DC++ 0.75+1 Removes 35 Character Nick Limit

This commit by ullner, coming in the next DC++ release, causes some popular hubs perhaps unexpected problems.

Hub administrators should probably look into updating their hub software.

Enjoy, all.

DC++ Remote Crash/Exploit Disclosure

In the spirit of public disclosure to encourage users to move to recent versions and to encourage mod developers to fix their code, this post announces a well-known but not publicly announced somewhat recent remote DC++ exploit.

The DC++ NULL Pointer Remote Denial of Service Vulnerability involves sending an $ADCGET command such as “$ADCGET (%S) //+ 0 %-1 ZL1” to the other client along a client-client connection, which will promptly and reliably crash the latter client. This affects all recent versions until 0.707, so unless you’re running one of 0.707, 0.7091, 0.750, or a more recent development-snapshot, you’re probably vulnerable to this remote crash.

Furthermore, one doesn’t have to manually connect to another client for this crash to occur; a connection triggered by autosearch/add-queue is sufficient. Alternatively, one doesn’t even have to rely on that but can instead just send $(Rev)ConnectToMe commands to other clients to create client-client connections in the manner of some client-detection mods and systematically crash an entire hub of DC++-pre-0.707 users by connecting to them and sending them the example poison command.

This exploit is one example of why as with all network-facing software, one should keep DC++ updated.

Securing the unsecure ?!

Seasons greetings everyone from us at DCDev

Wanted to rant about private only scripts and the idiocy usage of em since a script really cant determine if its public or private.

Private only scripts are mainly used in Ptokax / Verlihub or any soft using lua scripting interface.

So what does the tag stand for in DC++


first digit is unregistered usually determined as public hub but all it really is is unregistered.

second digit is registered here is where most people thing it means private and hidden away in the deep corners of the direct connect network (it kinda amuses me) since it could be a public vip or just an account on hub that you registered yourself at.

Third and last is of course the operator digit this really doesn’t show if its private or public either.

and using hublist to check if same user is online at other hubs on NMDC is pretty dumb too since, It might be an shared connection.

What if the person has a brother or sister that uses DC++ and likes to be  on a public hub to talk about nothing and everything.

plus it really cracks me up that we are talking about securing private hubs that are communicating via cleartext protocol :D using NO ENCRYPTION thats really funny how secure is the hub in that regard let me tell you’ll.

Its really just a waste so in the holiday seasons i leave you with the words of wisdom your only as secure as your own box (hub).

P.S for all those people making these scripts for god sake do it right and put in a 60-90 second delay before kicking the poor sap since there can be a slight delay in updating myinfo on NMDC and for that matter ADC since i know the idiocy is gonna continue.

I help those who help themselves! – Zoidberg (Futurama)

DC++ pointing out the corrupted

One of the latest enhancements in DC++ is the hub referral on client-client connections, proposed by Jan Vidar Krey. The current bazaar trunk implements this mechanism and the next DC++ version that will be released soon will also have it. The purpose of this extension is to point out the corrupted hub that is sending the current client to a non DC client, with obvious malevolent purpose. This implies that the hub is either using exploitable software, or that it’s intentionally abusing the clients. Either way, the hubowners are solely responsible.

On connecting to the other party, DC++ will also send the hub URL that it used to connect to the hub sending out the CTM message. By packet inspection, an attacked party can figure out which is the corrupted hub (only a pointer is required, such that they have a point of reference ) . Another good part about this extension is that it works on both ADC and NMDC ( some workaround was found for NMDC: adding the url to the PK string since NMDC is not extensible nor flexible in this matter ) , with the least effort from the clients and it does not bother them in any way. A normal client should ignore the specific message ( I don’t find any particular usage for it ).

We strongly recommend all mods to inherit this extension and other clients out there to implement it so the CTM attacks impact on DC software will stop being so great.

DC++ CTM Proof

In a previous post I was wondering if the whole concept of centralization is obsolete or has major flaws. The problem that is bothering everybody in the last years is that clients can be used ( unwillingly ) as tools in distributed denial of service attacks. Jan Vidar Krey is proposing a hub refferal on c-c connections that can point to a source of CTM attacks via the messages that the client sends on first connection attempt. In this case an attacked entity can see the hub with problems/intentional flooding that is causing the attacks.

As a first step to prevent this kind of abuses in DC++, poy added a static IP protection for the major hublists that were attacked via the client. This kind of measure is just temporary since hublists can change IP anytime and it protects only them, not everybody else that can be attacked ( Also the fun part is that the hublist server is actually running a DC client and wants to download from other users, it can’t ! ) .  A second step was to dynamically resolve the hublist ip’s and block them for c-c connections.

The main idea that I considered is to practically check all the users on a specific hub to see if they actually are real. On CTM receive the client should not connect but send another CTM to see if that IP actually connects to them . This will make sure that the user is the actual owner of that specific IP address. Of course the biggest problem is if the user is passive, in which case it can’t send a CTM back. This could be against the protocol principles but it’s a solution to see if the other peer really exists. I don’t know if a RCM would do something good in this situation but it’s a start.

Another thing that should be done ( if not implemented already ) is that on c-c connections if the first attempt was unsuccessful then no further attempts should be done until the user at least reconnects or changes state ( passive/active ). Also the hubs should be trustworthy. In a previous post I suggested a way to make hubs trustful via a CA authority system, but most people were quite reticent about it. Perhaps this could be the only way to make hubs trustful. Warning messages will not help too much ( Strong DC implements such messages ) since most of the users either don’t read them or don’t care. We shouldn’t let users question this problem, but solve it for them. Continuous problems from the Direct Connect network might be a cause to mark DC software ( and DC++ ) as badware, which will definitely take down the network. It’s time to do something about it.

I’m hoping for more ideas how to make DC++ proof against CTM abuses and I’m waiting for opinions from you as well.

Are centralized networks doomed from the start ?

Recently I heard bad rumors around the DC network. Some malevolent person ( unknown ) has written several scripts for the most known DC hub software, that allow the hub owners to use their users in abusive forms of flooding, using the CTM feature of the protocol. These scripts have started to spread around and now “script kiddies” use it for flame wars and endless childish attacks.

Also, important sites that currently hold on the DC community like the major hublists OpenHublist or DCHublist and the ADC counterpart ADCHublist were attacked and were down for a long time. The major community for ADC , ADCPortal was also attacked ( ADCPortal also provide an alternative wiki to the one on the ADC Project ).

My first concern is that this problem can spread up to the centralized networks principle. In this case, the central node ( hub ) has the power to absolutely control the leaves that are connected, thus it can abusively send them in possible attacks at wish. This might be a serious problem for the centralized networks.

Secure policies have to be enabled in clients and hubs so that this kind of flaws do not affect the community. I don’t know yet if it’s possible in this current situation or the whole concept of centralization is flawed. I hope not, because the Direct Connect community has it’s advantages and there are a lot of people involved and which benefit from it every day.

My advice from the users: make sure you actually know what hubs you are using, and how your client can be abusively used for other purposes than the ones designed, and make sure you are using a proper firewall and perhaps package inspection to see if your computer is not part of a BotNet or similar flooding network.

And one last thing, Happy new year and all the best from the DC++ team.