Archive for January, 2008

DC++ changing the internationalization capabilities

January 23, 2008

The next version of DC++ will feature a change in its structure for internationalization (i18n).

This all is possible thanks to gettext, which is a tool for allowing applications i18n in a very standardized manner, and is available on a multitude of platforms.

Before, all visible strings in DC++ had to be written and stored in a specific manner, where we used XML files to use the texts. This could be annoying, as you needed to constantly check what a string was called when you wanted to use it.

In the next version all text are going to be written directly as the code is lined up. This does not change the fact that DC++ will still support i18n. However, all of the XML files are going to render useless, as DC++ will drop its support for it.

The new format is .pot or .po, and our Launchpad translation site will allow you to translate strings in a much faster way; Strings that also exist in other applications are crosslinked to the DC++ project as suggestions, so for some stuff you don’t even need to input anything. It’s just to click on a particular field.

There’s as of yet no tool to convert the XML files to .pot (or .po) files. So if anyone’s up to some work, we’ll publish your tool, allowing people to do the conversion.

Don’t forget that you can make topic suggestions for blog posts in our “Blog Topic Suggestion Box!”

Support forum, bug tracker, feature tracker and language facility for DC++

January 23, 2008

It’s been a while since we had a working support forum. Or a bug tracker. or a feature tracker. But we’re now announcing our projectspace at Launchpad!

You find that the bug and feature tracker is in the same place, so just note that it’s a feature request and not a bug per se.

The forum, or as they call it “answers”, is also available for people to use.

Beyond that, we’re also going to use Launchpad’s language features for internationalization.

You need to register with the site if you wish to post in any of the places, so go ahead and do that.

(Note: No, we’re not going to move the SVN to Launchpad. It’s just a mirror.)

Don’t forget that you can make topic suggestions for blog posts in our “Blog Topic Suggestion Box!”

Number of DC users

January 17, 2008

In a previous post, I (sort of) said that there were 650000 people who used DC++. PC Pitstop doesn’t really agree, but oh well… And I said in the interview with Robert Lemos that that figure was probably around 300000. So… Which is it? How many people are using DC++ and DC?

Well, we don’t know. It’s not possible for us to see and I certainly wouldn’t trust a report coming from a potential spyware company.

DC have declined in users, I don’t think anyone involved in DC can miss that. The question is how much. I believe my assertion that around 300000 users is still valid, but I can’t prove it.

However, perhaps instead of asking “how many people are using DC?”, we should be asking “what can we do to improve the current users’ experience?” and “what can we do to at least keep the current users?”. Most hubs are madly in love with the notion that the more users a hub has, the better it is. What people have come to understand the past years is that it’s more about content and not the amount of users the hub has.

Don’t forget that you can make topic suggestions for blog posts in our “Blog Topic Suggestion Box!”

PC Pitstop and its P2P-report

January 17, 2008

PC Pitstop recently released a report regarding P2P-applications that they had encountered while their spyware application had been running on their “customers’” computers. Go and download their relevant stats (.xls).

A couple of comments and things I noticed regarding it;
(1) The total amount of DC market share dropped from 0.94% to 0.43% over a period of a year.
(2) The total amount of PCs scanned were from 174000 to 103000 per month.
(3) The total amount of DC users per month is something you need to calculate, as it’s not noted…
(4) Sweden and Romania are two countries where DC is quite prevalent; The report had a very small number of customers in those countries.
(5) Windows XP and Vista are the operating systems that were recorded. Other systems are lumped together as “other”. Presumably are non-Windows systems not tested.
(6) DC++ was among the top 10 of applications that had dropped users.
(7) All applications that are able to connect to a given network are listed in the report. Eg, mIRC is listed as a P2P-application where “IRC” is the network, while IRC isn’t even a P2P-network.
(8) There were 8640 users reported that had DC++ installed.
(9) The reported DC++ versions were “DC++ 0.6″, “DC++”, “DC++ 0.4″ and “Acadia DC++ 1.5″, where “DC++ 0.6″ had over 8000 users.
(10) The reported NeoModus Direct Connect versions were “DC 2″, “DC1″ and even “DC hub”, which I didn’t really get… (Where “DC” is elongated…)
(11) StrongDC++ had “SDC++ 1″, “SDC++”, “SDC 2″ and “SDC++ 2″… (Where “S” in SDC is elongated.)

The only thing I can get in this report is the drop in market share. The rest is… Pointless and gibberish.

Don’t forget that you can make topic suggestions for blog posts in our “Blog Topic Suggestion Box!”

Press coverage regarding DC being used as a DDoS tool

January 17, 2008

There were some press coverage regarding my Denying distributed attacks post. Actually, I was interviewed by Robert Lemos. The following is the mail correspondance we had;

He wrote;

1) From your data, it looke like there is at least 650,000 people using the latest version of DC++ is that correct? How many users do you think you have? What do people typically use dc++ for?

2) Prolexic, an anti-DDoS firm, has used dc++ as an example of a new style of attack (link: http://www.prolexic.com/news/20070514-alert.php) that uses the directory servers of p2p software to direct the clients to DOS a particular IP address or network. Are these attacks common? Can you name an instance in particular where you observed such an attack and give me some concrete details on what happened?

3) How do attackers get control of the dc++ network or directory server to direct such attacks? Are there countermeasures that can be implemented in the software to determine when such an attack is happening?

I wrote;

1) No, I don’t think there’s 650,000 people. We currently have no real way of knowing how many people are using DC++, or even the latest version. We can calculate how many people are starting their clients, but that would require us to look at the logs on how many people connect to the DC++ website. SourceForge, which is hosting the website, does not provide us with those logs, so we can’t say for sure how many are using DC++.
At one point in time, we were able to see how many were using DC++, and it was somewhere around 300,000. But that figure may be misleading, as it’s incremented when someone open a particular window in DC++ as well as on the upstart. If people are then also using multiple instances of DC++, the number will also increase as there’s no lookup on indiviual IPs.
The vast majority of DC++ users (and generally Direct Connect users) use the network to trade files with each other. The major difference between Direct Connect and other P2P networks is the community; The ability to communicate with whoever you’re trading your files with (and of course other people). I personally mostly use Direct Connect for the community aspect. I believe one might compare Direct Connect with good old Napster, with a built in Instant Messenger.

2) These attacks are unfortunately getting more common. I think this phenomenom can be traced back to late 2005, early 2006. (Although, not in such a scale as Prolexic has experienced.)
The most public attack was against www.hublist.org, which was the largest hub list in DC (and the most influencial – DC++ had Hublist.org’s hub list as the single default list, until about a year back.) I say was because the site has been down and up since the attacks started.
Basically, the administrators of the hub list and a group of rogue Direct Connect users had a dispute that escalated beond common sense, and has ended up as a massive DDoS. The latter had created tools to ‘destroy’ hubs (eg, a tool to cause a flood in a hub – a different type of DoS) or cripple them. The administrators of Hublist.org responded by removing the hubs that the rogue group commanded, from the hub list Hublist.org provided. In return, the rogue group responded by attacking Hublist.org to show their dismay. Reportedly, the site had experienced a constant 10-90% bandwidth usage (on a 100 Mbit/s line), during about a year’s worth of time.
I mentioned that Hublist.org had been the only default hub list in DC++, since a year back. By that I mean that other hub lists were added at that time. The reason was because the rogue group (that had attacked previously) had launched a massive attack at Hublist.org, rending the site useless and unresponding. The site had been able to work, yet the heavy usage before. However, after the massive attack, the site was down for about six months before it was able to get back up.
Meanwhile, dcpp.net (an address that Todd Pederzani had bought) and the server it was on, was also getting hit, which is the reason the site was moved back to SourceForge, and other parts have been removed. DirectConnect.se, which is a Swedish oriented DC community site, was also taken down with dcpp.net as they were on the same server. YnHub.org, which serve the most popular hub for Windows, was also hit in the storm of attacks (though I don’t know if it was on the same computer as the other two sites).
You should contact the administrators of hublist.org concerning their DDoS. They have reportedly been in contact with Finnish police (their site is .fi based), Scotland Yard, FBI, UK authorities and Romanian authorities. (There may be more; I don’t remember more currently.)
Disclaimer: I have no idea if this is the same group as have been reported by Prolexic.

3) The attackers are able to take control over servers by two ways; [1] By exploiting the victim’s operating system, and gaining access to the hub and making the hub render as part of a bot net. And [2] by exploiting the victim’s hub software.
The former can of course be avoided by good firewall protection, but that’s quite difficult for anyone involved in DC to enforce.
The latter can be avoided by upgrading hub software that are known to have exploits. I don’t think there are any public hub softwares today that have these flaws; As I’ve written, the attackers take advantage of people’s reluctance to upgrade. (Out of lazyness or whatever.)
The attackers also have the possibility, as anyone else, to register a new hub on a hub list, which they are in complete control over. It’s difficult to impossible to restrict this. I think the majority of DDoS initiating hubs are in this category.

Lemos published his article on SecurityFocus. There are other sites he published on, too, but as far as I can tell, it’s all the identical article.

Don’t forget that you can make topic suggestions for blog posts in our “Blog Topic Suggestion Box!”

ADC hub referrer

January 17, 2008

In my Denying distributed attacks post, I described that users can be redirected to hubs. What I didn’t describe was whether there was a way for another hub to see if a user has been redirected or not. That is, something similar to the HTML referrer.

At the time, neither ADC nor NMDC had this type of functionality. However, before the official ADC 1.0 specification, a hub referral field was added to the INF.

What this means is that if a hub redirects a user, the client will automagically tell the hub so, when it successfully connects.

(NMDC still do not have this functionality.)

[If you're not a hub (but eg a HTTP server) that's being targeted, yes, you will need to do some communicating to find out from which address the user came from.]

Don’t forget that you can make topic suggestions for blog posts in our “Blog Topic Suggestion Box!”

Scamming passive users in ADC

January 17, 2008

Being in passive mode in ADC sure its disadvantages; less search results and less download sources. However, this also works the other way; you have to respond to less searches and have fewer potential uploads (because other people are in passive mode, too).

Why would anyone consider being passive as advantagious? Well, if you’re intentionally passive when you want to and intentionally active when you want to.

Consider the case of a download in active mode; You log in, broadcast to everyone your IP and port so they can connect to you (and you’ll be able to download) (“CTM = connect to me”).
Consider the case of a download in passive mode; You log in, ask whoever you want to download from what their “connect to me” is, and you connect to them.

Now what if I did a little scam? What if I logged in, not publicly but privately sent a user my IP and port, and did a “connect to me”? And afterwards I’ve sent the CTM, I’d just (privately again) send “I’m passive again, so I’m resetting my IP and port”. But your ongoing client-client connection would still be active.

What this will do is that only the active users in the hub will be able to download from you, and you won’t be relayed download requests by users who aren’t active. (Or if you are relayed, you can just ignore them “as you won’t be able to download from them”.)

This also works in searches. When you want to search, simply broadcast your info so people can relay the proper amount of search results to you. When you’re satisfied, just reset your information.

Note: This all could also be used as a way of saying “hey, I publicly said my port was this, but it’s shaped, so if you want to use one that’s not, use this port.”

(Note: I’m not sure if this all is possible in NMDC, because there’s no notion of “send only to a specific user”.)

Don’t forget that you can make topic suggestions for blog posts in our “Blog Topic Suggestion Box!”

Bloom filters

January 7, 2008

Currently, to ensure that a search query reaches all clients that might have files in their share matching it, DC hubs must broadcast that search query to all connected clients. This works, but scales poorly bandwidthwise with the number of distinct clients in a hub. DC++ 0.704 adds support for Bloom filters to ameliorate this bandwidth issue.

A Bloom filter, as used by a DC client or hub, concisely summarizes the contents of a share by constructing a bit vector which allows for probabalistic queries as to the presence of a file with a specified hash in that share. A motivating advantage of such a construction that whereas even filelists which contain only hashes will grow linearly with the number of files in the share. Further, such a fully accurate representation should not be compressible, because cryptographic hashes aim for each output bit to be independently 50% likely enabled and thus maximize apparent entropy. Instead, Bloom filters sacrifice losslessness in favour of an output that has a small size which need not grow with that of the share.

However, to remain useful in this filesharing context, such lossiness must remain limited. In particular, a Bloom filter produces false positives but not false negatives: it can state that an object exists when it does not, but it will never state conversely that an object does not exist if it does. A hub, given a Bloom filter representation of a share can thus safely refrain from sending a hash search query to a connected client the Bloom filter of the share of which has no record of a hash. This can save considerable bandwidth and allow a combination of more clients within the same bandwidth or a similar number of clients using less bandwidth.

A Bloom filter is determined by the hash keys in use and the table size. The smaller the ratio between the number of objects in the table and size of the table, the lower the false positive rate. Further, an optimal table does not utilize just one key but multiple keys simultaneously, such that the presence of an object in the table implies that the bits corresponding to each keys as applied to the input object are all enabled. Thus, when inserting an object for which hash functions used as keys map to bits 58, 1093, and 9865, then those bits are all enabled. Then to check whether that object might exist in the table, then one checks whether any one of bits 58, 1093, and 9865 are enabled. If all of them are, then it is possible that the object is in the table. However, if any of them are not, then the object is not in the table.

Bloom filters, then, allow for a scalable, efficiently constructed, efficiently queried, and efficiently communicated summary of the contents of a share which allows for hubs to spend less bandwidth wastefully broadcasting search queries.

Don’t forget that you can make topic suggestions for blog posts in our “Blog Topic Suggestion Box!”

ADCH++ First Release

January 6, 2008

After more than a year, here it comes, ADCH++ 2.1 , the first release. You can find it here.

Contains an installer and a zip for win32 users ( precompiled and ready to go ). Note that win32 testing has not been completed throughly ( may still have some problems ). If you have anything to report, you can visit the public development hub adc://devpublic.myhub.org:16591 or reply to this post.

The release also contains a source package ready to be compiled by linux or windows users. More information about compiling can be found in the readme.txt file supplied in the package, or in some previous blog posts :

Compiling ADCH++ under windows and
Compiling ADCH++ under POSIX

ADCH++ is compatible with ADC 1.0 ( but like DC++, supports older versions of protocol for easening the transition ).

This release does not include the Bloom filter (not fully completed ) , although it can be created by compiling the source . ( The bloom filter creates a hashmap of all users’ share, and searches that don’t have any possible result are ignored , hence reducing bandwidth and resource usage )

2.1 is marked unstable for obvious reasons, but hopefully in the future there will be a more stable release.

Beware of the PM flag

January 2, 2008

The action MSG in ADC for protocol chat have a PM flag to be used as a way to signal that the chat message is for “private” use. Most likely, clients will use this flag in combination with a D or E type message.

However, one should be aware of this. A client is not forced to use the PM flag, even if they intend for it to be a private message. Eg, if you send a message with the D-type and without the PM flag, the remote client should interpret the message as it was a “main chat message” (yet the message shall only appear in that client, per the type). The client don’t care whether that flag is present.

Also, the PM flag can be used in combination of the broadcast type, B. This leaves some room for spammers; send a broadcast message with the PM flag and all of the users in the hub will receive a “private message”.

The reason this all is confusing is most likely because NMDC only have a sense of “main chat” or “private” message, and not a sense of “message directed to someone [or a group of people]“, as ADC have.

Don’t forget that you can make topic suggestions for blog posts in our “Blog Topic Suggestion Box!”