Mixed-hash DC hubs

They work fine if clients and hubs support both TTH and its successor adequately long.

While transitioning to a TTH successor, currently interoperable clients and hubs all supporting only TTH will diverge. In examining the consequences of such diversity, one can partition concerns into client-hub communication irrelevant to other clients; hub-mediated communication between two clients; and direct client-client communication. In each case, one can look at scenarios with complete, partial, and no supported hash function overlap. Complete overlap defines the all-TTH status quo and, clearly, works without complication for all forms of DC communication, so this post focuses on the remaining situations. In general,

Almost as straightforwardly, ADC but not NMDC client-hub communication irrelevant to other clients requires partial but not complete hash function overlap but only between each individual client/hub pair, and don’t create specific mixed-hash hub problems; otherwise, an ADC hub indicates STA error code 47. For ADC, This category consists of GPA, PAS, PID/CID negotiation (with length caveats as relate to other clients interpreting the resulting CID), and the establishment of a session hash function; NMDC does not depend on hashing at all for analogous functionality. Thus, for NMDC, no problems occur here. ADC’s greater usage of hashing requires correspondingly more care.

Specifically, GPA and PAS require that SUP had established some shared hash function between the client logging in and the hub, but otherwise have no bearing on mixed-hash-function DC hubs. Deriving the CID from the PID involves the session hash algorithm, which as with GPA/PAS merely requires partial hash function support overlap between each separate client and a hub. Length concerns do exist here, but become relevant only with hub-mediated communication between two clients.

Indeed, clients communicating via a hub comprise the bulk of DC client-hub communication. Of these, INF, SCH, and RES directly involve hashed content or CIDs. SCH ($Search) allows one to search by TTH and would also allow one to search by TTH’s successor. Such searches can only return results from clients which support the hash in question, so as before, partial overlap between clients works adequately. However, to avoid incentivizing clients which support both TTH and its successor to broadcast both searches and double auto-search bandwidth, a combined search method containing both hashes might prove useful. Similarly, RES specifies that clients must provide the session hash of their file, but also “are encouraged to supply additional fields if available”, which might include non-session hash functions they happen to support, such that as with the first client-hub communication category, partial hash function support overlap between any pair of clients suffices, but no overlap does not.

A more subtle and ADC-specific issue issue arises via RES’s U-type message header and INF’s ID field whereby ADC software commonly checks for exactly 39-byte CIDs. While clients need not support whatever specific hash algorithm produced a CID, the ADC specification requires that they support variable-length CIDs. Example of other hash function output lengths which, minimally, should be supported include:

Bits Bytes Bytes (base32) Supporting Hashes
192 24 39 Tiger
224 28 45 Skein, Keccak, other SHA-3 finalists, SHA-2
256 32 52 Skein, Keccak, other SHA-3 finalists, SHA-2
384 48 77 Skein, Keccak, other SHA-3 finalists, SHA-2
512 64 103 Skein, Keccak, other SHA-3 finalists, SHA-2

Finally, direct client-client communications introduces CSUP ($Supports), GET/GFI/SND ($Get/$Send) via the TTH/ share root or its successor, and filelists, all of which work if and only if partial hash function support overlap exists. CSUP otherwise fails with error code 54 and some subset of hash roots and hash trees regarding some filelist must be mutually understood, so as with the other cases, partial but not complete hash function support overlap between any given pair of clients is required.

Encouragingly, since together client-hub communication irrelevant to other clients; hub-mediated communication between two clients; and direct client-client communication cover all DC communication, partial hash function support overlap between any given pair of DC clients or servers suffices to ensure that all clients might fully functionally interact with each other. This results in a smooth, usable transition period for both NMDC and ADC so long as clients and hubs only drop TTH support once its successor becomes sufficiently ubiquitous. Further, relative to ADC, poy has observed that “all the hash function changes on NMDC is the file list (already a new, amendable format) and searches (an extension) so a protocol freeze shouldn’t matter there”, which creates an even easier transition than ADC in NMDC.

In service of such an outcome, I suggest two parallel sets of recommendations, one whenever convenient and the other closer to a decision on a TTH replacement. More short-term:

  • Ensure ADC software obeys “Clients must be prepared to handle CIDs of varying lengths.”
  • Create an ADC mechanism by which clients supporting both TTH and its successor can search via both without doubling (broadcast) search traffic. Otherwise, malincentives propagate.
  • Ensure BLOM scales to multiple hash functions.
  • Update phrasing in ADC specification to clarify that all known hashes for a file should be included in RES, not just session hash.

As the  choice of TTH’s successor approaches:

  • Disallow new hash function from being 192 bits to avoid ambiguity with Tiger or TTH hashes. I suggest 224 or 256-bit output; SHA-2 and all SHA-3 finalists (including Keccak and Skein) offer both sizes.
  • Pick either a single filelist with all supported hashes or multiple filelists, each of which only supports one hash. I favor the former; it especially helps during a transition period for even a client downloading via TTH’s successor to be able to autosearch and otherwise interact with clients which don’t yet support the new hash function, without needing to download an entire new filelist.
  • Barring a more dramatic break in Tiger than thus far seen, clients should retain TIGR support until the majority of ADC hubs and NMDC or ADC clients offer support for the successor hash function’s extension.

By doing so, clients both supporting only TTH and both TTH and new hash function should be capable of interacting without problems, transparently to end-users, while over time creating a critical mass of new hash function-supporting clients such that eventually client and hub software might outright drop Tiger and TTH support.

A Decade of TTH: Its Selection and Uncertain Future

NMDC and ADC rely on the Tiger Tree Hash to identify files. DC requires a cryptographic hash function to avoid the previous morass of pervasive similar, but not identical, files. A bare cryptographic hash primitive such as SHA-1 did not suffice because not only did the files need identification as a whole but in separate parts, allowing reliable resuming and multi-source downloading, and per-segment integrity verification (RevConnect unsuccessfully attempted to reliably use multi-source downloading precisely because it could not rely on cryptographic hashes).

Looking for inspiration from other P2P software, I found that BitTorrent used (and uses) piecewise SHA-1 with per-torrent segment sizes. Since the DC share model asks that same hash function work across entire shares, this does not work. eDonkey2000 and eMule, with per-user shares similar to those of DC, resolved this with fixed, 9MB piecewise MD4, but this segment size scaled poorly, ensured that fixing corruption demanded at least 9MB of retransmission, and used the weak and soon-broken MD4. Gnutella, though, had found an elegant, scalable solution in TTH.

This Tiger Tree hash, which I thus copied from Gnutella, scales to both large and small files while depending on what was at the time a secure-looking Tiger hash function. It smoothly, adaptively sizes a hash tree while retaining interoperability between all such sizes of files files on a hub. By 2003, I had released BCDC++ which used TTH. However, the initial version of hash trees implemented by Gnutella and DC used the same hash primitive for leaf and internal tree nodes. This left it open to collisions, fixed by using different leaf and internal hash primitives. Both Gnutella and DC quickly adopted this fix and DC has followed this second version of THEX to specify TTH for the last decade.

Though it has served DC well, TTH might soon need a replacement. The Tiger hash primitive underlying it by now lists as broken due to a combination of a practical 1-bit pseudocollision attack on all rounds, a similarly feasible full collision on all but 5 of its 24 rounds, and full, albeit theoretical, 24-round pre-images (“Advanced Meet-in-the-Middle Preimage Attacks”, 2010, Guo et al). If one can collide or find preimages of Tiger, one can also trivially collide or find preimages of TTH. We are therefore investigating alternative cryptographic hash primitives to which we might transition as Tiger looks increasingly insecure and collision-prone, focusing on SHA-2 and SHA-3.

How to crash DC++ 0.674

$ADCGET list //// 0 -1 ZL1|

A previous blog post mentions this, but apparently isn’t sufficiently explicit about what to send.

I aim to fix that.

Enjoy, all. This apparently works on DC++ clients older than 0.707 which still support $ADCGET.

DC++ 0.75 and older vulnerable to bzip2 filelist bomb

DC++ 0.75 and earlier can be remotely crashed via either bzip2 filelists (example filelist) or hublists (themselves compressed with bzip2). Such list downloads can be automatically triggered by automatic searches for alternate sources, so explicit user action is unnecessary [1]. Not every client seems to crash; the precise dependence on operating system or other factors remains unclear. However, crashes have been observed using both Windows XP and Windows 7.

As before, updating network-facing software remains important. Equally importantly, DC++ mod authors should attempt to update in a timely manner such as to avoid exposing their users to this bug.

[1] To catch a large number of clients in a hub in a relatively short period of time with no manual intervention, listen for searches (especially TTH searches) and always respond positively, such that clients try autosearching for alternate sources. Other tricks are possible as well, of course.

DC++ 0.75+1 Removes 35 Character Nick Limit

This commit by ullner, coming in the next DC++ release, causes some popular hubs perhaps unexpected problems.

Hub administrators should probably look into updating their hub software.

Enjoy, all.

DC++ Remote Crash/Exploit Disclosure

In the spirit of public disclosure to encourage users to move to recent versions and to encourage mod developers to fix their code, this post announces a well-known but not publicly announced somewhat recent remote DC++ exploit.

The DC++ NULL Pointer Remote Denial of Service Vulnerability involves sending an $ADCGET command such as “$ADCGET (%S) //+ 0 %-1 ZL1” to the other client along a client-client connection, which will promptly and reliably crash the latter client. This affects all recent versions until 0.707, so unless you’re running one of 0.707, 0.7091, 0.750, or a more recent development-snapshot, you’re probably vulnerable to this remote crash.

Furthermore, one doesn’t have to manually connect to another client for this crash to occur; a connection triggered by autosearch/add-queue is sufficient. Alternatively, one doesn’t even have to rely on that but can instead just send $(Rev)ConnectToMe commands to other clients to create client-client connections in the manner of some client-detection mods and systematically crash an entire hub of DC++-pre-0.707 users by connecting to them and sending them the example poison command.

This exploit is one example of why as with all network-facing software, one should keep DC++ updated.

Securing the unsecure ?!

Seasons greetings everyone from us at DCDev

Wanted to rant about private only scripts and the idiocy usage of em since a script really cant determine if its public or private.

Private only scripts are mainly used in Ptokax / Verlihub or any soft using lua scripting interface.

So what does the tag stand for in DC++

1/1/1

first digit is unregistered usually determined as public hub but all it really is is unregistered.

second digit is registered here is where most people thing it means private and hidden away in the deep corners of the direct connect network (it kinda amuses me) since it could be a public vip or just an account on hub that you registered yourself at.

Third and last is of course the operator digit this really doesn’t show if its private or public either.

and using hublist to check if same user is online at other hubs on NMDC is pretty dumb too since, It might be an shared connection.

What if the person has a brother or sister that uses DC++ and likes to be  on a public hub to talk about nothing and everything.

plus it really cracks me up that we are talking about securing private hubs that are communicating via cleartext protocol :D using NO ENCRYPTION thats really funny how secure is the hub in that regard let me tell you’ll.

Its really just a waste so in the holiday seasons i leave you with the words of wisdom your only as secure as your own box (hub).

P.S for all those people making these scripts for god sake do it right and put in a 60-90 second delay before kicking the poor sap since there can be a slight delay in updating myinfo on NMDC and for that matter ADC since i know the idiocy is gonna continue.


I help those who help themselves! – Zoidberg (Futurama)