Roadblock

Mea culpa for not updating this blog in quite a while, but it's not because i slowed down work on it, but rather, much worse, i've hit a roadblock which is pretty darn serious. In brief, when trying to write the algorithm that would enable a p2pOS client to act as a relay for its plugins (i.e. to "connect the crossed red lines" that i talked about in a previous post), i also had to try to define the low-level API that the plugins will be using to connect to one another (by means of their associated p2pOS clients that will be acting as the relays, one relay at each end of a P2P connection). And as one thing led to another, i eventually reached the conclusion that what i need to do first is to establish exactly how a peer joins the network and connects to another peer. And this is where things got really, really ugly.

Without entering into too many technical details, the important point here is that in order to have two peers connect to each other, they have to go through an initial "handshaking phase" during which the two peers learn some essential things about one another (e.g. their IP addresses, what kind of router(s)/firewall(s) they are behind, etc), and this handshaking phase has to be negotiated through a dedicated handshaking server. Well,  i can try to hide behind all sorts of technical arguments, but that fact of the matter is that ever since i started this project i never tried to calculate exactly how much traffic such a central handshaking server would require for a large P2P network (i'm talking about 100,000,000...1,000,000,000 users being online), only to find out now that the numbers are astronomical: namely, we're talking about thousands, or even tens of thousands, of terrabytes/month, which translates into a handshaking server operating cost somewhere in the hundreds of thousand, maybe millions, of dollars a month. This in turn means that hosting such a server is not something that just about any punk can do in his basement, which in turn means a large company would be required to finance the network operation. Or, in layman's terms: the network can never be truly open, no matter what license will be covering this project, no matter what verbal commitments a company would make, etc. And since the (maybe only) non-negotiable objective of this project is to create a truly open P2P platform, well... you guessed it: i'm stuck.

But, as i said at the beginning of this post, all this mess doesn't mean i gave up on the project; in fact, because it seems increasingly likely that some sort of distributed handshaking algorithm will be necessary, i made quite a few tweaks in the program in order to reduce the traffic between connected peers (e.g. i managed to reduce the P2P keep-alive traffic by about an order of magnitude by detecting the peers' routers' port timeouts and only send keep-alive messages at the required rate), i refined the router classes such that over 90% of the routers models can now act as relays, and i introduced an algorithm that detects if a peer is directly connected to the internet (i.e. public IP or UPnP) such that it can serve as handshaking server in the network. This is how the new "Settings" panel looks like now:

So what i'm doing right now is study what other smart-@$$ P2P projects have done (e.g. Gnutella, Freenet, etc), i'm trying to learn about various DHT approaches (there's a very nice tutorial talking about the basics here), etc, and i'll see if i'll be able to come up with a solution. Keep ya fingers crossed for me, it's in the world's best interest and sh*t :)

PS
Here's how a p2pOS-based P2P session is established through NAT routers with the help of a handshaking server: the blue messages are P2P messages, while the green messages are relayed via the handshaking server (once the handshaking phase is completed, all messages from one peer go directly to the other, i.e. they are P2P messages):