XML parsing of file lists

Many DC clients (and other software) have their own XML parser for parsing XML files and content. This means the parsers can be heavily specialized for performance (in the case of large file lists for instance) compared to just using a “standard parser” (i.e. one that has been used in multiple projects). However, building one’s own parser also means that the parser may be incorrect to a far greater extent, thereby increasing the risk that a malicious party (e.g. the one sending the file list) may try to remotely crash the receiver by sending incorrect files. Beyond the obvious concern for network security, clients may incorrectly allow files to be read or read incorrect data within those files.

I have compiled a list of potential errors that a file list may have, and generated file lists for each of those occurences. These file lists were then opened in DC++ (0.851) and verified to see what happened. This test should likely be done with all clients that don’t derive their own XML parsing with DC++’s (i.e., all DC++-mods will likely follow the below pattern).

I created multiple file lists (downloadable here), based on this file, generated with this C# snippet. Here are the results (Microsoft Office Excel file).

A summary of the results;

  • DC++ will parse invalid data (e.g. omission of data) and sometimes replace the faulty data with “something sensible”, although this is almost in all cases wrong.
  • In most cases where it is an invalid XML document, DC++ will ignore those sections or ignore the file altogether (this is good).
  • DC++ will not crash on invalid data.

Most of the issues found can be solved by performing a XML-sanitation check before reading the document, by validating against the XSD. DC++’s XML parser does not have any XSD validation, so it couldn’t be done at this point anyway, but should such a validation be implemented, it will cause a (small or big depends on the source file list) performance hit.

While I didn’t test it, parsing of the XML list for version.xml and any hublists will likely have the same issue(s) as mentioned above. At least we won’t crash DC++.

If someone has other software that they can test this with, please feel free to do so and let me know so I can update the Excel sheet. It’s also possible that the resulting files are named incorrectly (e.g. by not requiring a CID in the file name), so just run the snippet code.

(Note: The files in this post may have a file name such as “foo-zip.pdf”, and it is because the file is actually a zip file but this blog software couldn’t handle that, so just change the file-extension to the appropriate one.)

Organization meeting scheduled for 2016-01-10

An upcoming meeting for the Direct Connect Network Foundation is scheduled for 2016-01-10, at 19.00 CET. In the meeting, we will go over items from the past year and what we shall do in the coming year. The agenda will be according to the By-laws (https://www.dcbase.org/bylaws/) § 10.

You can see the previous meeting(s) here: https://www.dcbase.org/meetings/ so you can get a feel for the structure etc of the meeting. Feel free to suggest ways to improve the meeting process.

If you have additional items you wish the meeting to address (that people should think about beforehand), please post in this forum thread.

DCNF Resources

A new resources page at the dcbase.org site has now been created, where you can see all articles etc that relate to DC. Also, the page contains court cases where DC is involved in some way.

Addressing DC++’s service provider, SourceForge

There has been a lot of discussion regarding changes to SourceForge’s hosting practices [1][2][3]. There are two things that SourceForge have done; created an opt-in “revenue program” and begun taking over old or non-updating (or even non-existant) projects.

The opt-in program is DevShare and allow developers (project administrators) to receive revenue based on modified installers. FileZilla is one of the major projects that have done so. The modified installers embed additional programs, thereby acting as ad services. The developers can choose which type of ads/programs are suggested, although they cannot say exactly which may or may not show up. The developers do nothing extra to accomodate this feature. The difference, as noted by Ghacks.net is that SourceForge will change the appearance of the download page to highlight the ad-specific one whilst still having a link to the other one (albiet not as easy to see).

The DC++ administrators were sent an e-mail from SourceForge regarding the DevShare program whether DC++ should or should not also opt-in for the DevShare program. The DC++ administrators declined this offer as the additional revenue was not needed for any basic operation and it felt it might violate the integrety of the installers. This was just as the DevShare program had been announced. No further action for this has been taken and no additional requests from SourceForge have been made.

The second part of SourceForge’s changes are that of modifications to old projects or completely taking over the projects (or even creating them in the first place). This can be seen with e.g. GIMP. As long as DC++ does not become stale or otherwise non-active this will never affect DC++.

All of this have caused us (the developers of DC++) to review our stance with SourceForge. Some facts before I continue:

  • SourceForge have hosted DC++ (and other DC related software) since its inception (i.e. for several years) without any problems in this area.
  • SourceForge provides stable code repositories and website resources. Although the speed of SourceForge network may be questionable, it is able to withstand hard DDoS:ing.
  • DC++ hosts the source code repository, file downloads and website resources on SourceForge.
  • There are other DC related projects that are also hosted on SourceForge.
  • DC++ is considered a “valued projects” in that it has appeared on SourceForge’s project of the month as well as the DevShare offer. DC++ is also among the high-download projects at SourceForge.
  • DC++ will not be directly affected by DevShare as we have not accepted such an offer. (I must stress it is an opt-in offer.)
  • DC++ will not be directly affected by the abondoned projects changes as DC++ continue to be updated and will not qualify for such a change.
  • At least one browser plugin, uBlock, have started to block SourceForge as a whole, thereby potentially restricting users from accessing DC(++) resources.

So, in light of all of this, we have begun to look into other project repositories:

  • Launchpad – Already hosts other features for DC++, such as the bug tracker, but does not provide a sufficient code repository (Bazaar is near-dead), somewhat cumbersome download capabilities and no true website support.
  • Github – No real website support. This is more suited for just the code repository than a full-on project repository. We are more likely to host the source code on Github and proxy that through another service.
  • Bitbucket – Restricts number of contributors, no website support. poy suggests strongly that we do not move to Bitbucket.
  • Google Code – Recently closed registration of new projects. (Lacked anyway certain features.)

There are other project repositories available, although no one of us have experience with most of them.

It is important for us to move forward with this, so here is our plan forward:

  • Move (or at least parts of) source code repositories, websites and download facilities to our own hosting facilities. E.g., Rhodecode is being set up to address this for source code.
  • DC++ will continue to use SourceForge as a minimum as its backup service provider. It is important to note that we have had a relatively pleasant experience with SourceForge – as project administrators.
  • We will continue to monitor any further development in SourceForge management and changes.

We welcome suggestions, both from SourceForge and others, in how we can move forward.

Hardening DC++ Cryptography: TLS, HTTPS, and KEYP

BEAST, CRIME, BREACH, and Lucky 13 together left DC++ with no secure TLS support. Since then, the triple handshake attack, Heartbleed, POODLE for both SSL 3 and TLS, FREAK, and Logjam have multiplied hazards.

Fortunately, in the intervening year and a half, in response:

  • poy introduces direct, encrypted private messages in DC++ 0.830.
  • DC++ 0.840 sees substantial, wide-ranging improvements in KEYP and HTTPS support from Crise, anticipating Google sunsetting SHA1 by several months and detecting man-in-the-middle attempts across both KEYP and HTTPS.
  • OpenSSL 1.0.1g, included in DC++ 0.842, fixes Heartbleed.
  • DC++ 0.850 avoids CRIME and BREACH by disabling TLS compression; avoids RC4 vulnerabilities by removing support for RC4; prevents BEAST by supporting TLS 1.1 and 1.2; mitigates Lucky 13 through preferring AES-GCM ciphersuites; removes support for increasingly factorable 512-bit and 1024-bit DH and RSA ephemeral TLS keys; and with all but one ciphersuite, AES128-SHA, deprecated and included for DC++ pre-0.850 compatibility, uses either DHE or ECDHE ciphersuites to provide perfect forward secrecy, mitigating any future Heartbleed-like vulnerabilities.
  • DC++ 0.851 uses a new OpenSSL 1.0.2 API to constrain allowed elliptic curves to those for which OpenSSL provides constant-time assembly code to avoid timing side-channel attacks.

These KEYP, TLS, and HTTPS improvements have not only fixed known weaknesses, but prevent DC++ 0.850 and 0.851 from ever having been vulnerable to either FREAK or Logjam. As with perfect forward secrecy, these changes increase DC++’s ongoing security against yet-unknown cryptographic developments.

The upcoming version switches URLs in documentation, in menu items, and of the GeoIP downloads from HTTP to HTTPS. While these changes do not and cannot prevent attacks perfectly, it should now provide users with improved and still-improving cryptographic security for the benefit of all DC++ users.

Donations for DCNF (April 2015)

A big thank you to the following people who donated to the PayPal account for DCNF (and DC++). Your money will be spent on server and domain upkeep. We will be looking for a way for donators to receive something back.

Valentin B.
R P H.
Åke S.
Patrick H.
Alan D.

The organization now has 293,04 Euros raised from member fees and the donators above.

Direct Connect Network Foundation

In January 2015, a non-profit organization was set up, called Direct Connect Network Foundation (DCNF). The organization aims to provide information and resources for developers and users of Direct Connect. The website dcbase.org was chosen to be the main site for the organization.

DCNF is an actual registered organization in Sweden, with government number 802492-9716. See also the by-laws, and the annual meeting notes.

To become a member, simply donate to the PayPal account and make a note in the forum.

I or the others on the board will periodically make a note here about anyone who donates to the organization.


Get every new post delivered to your Inbox.

Join 31 other followers