libagents-1.0.0 released

It's been about a year and a half since my last post on the 'libposif' library, and at that time my original intention was to bring together all of the various functions that I will need for the P2P OS algorithms inside a single library; however, after a while i realized that a better option would be to have two separate libraries: namely, the 'libposif' library should be dealing only with the cross-platform portability issues, and a separate library, 'libagents', should deal with the agents-based programming model.

So the 'libagents' library project was born: I wanted a pure C++ implementation of the actor model, and I wanted it released as a stand-alone open source project, with proper documentation and all. Now, after over a year of tweaks and hundreds of pages written and then deleted in the doc, today is the day when libagents-1.0.0 is finally available for download. Grab it, use it, and spice up your programs with some elegant parallel processing :)

PS
Don't be discouraged by the sheer size of the documentation, just take it one page at a time. I spent countless hours trying to make it easy to read, so don't try to cut corners, that ain't gonna cut it: the doc is written with the assumption that it will be read and understood incrementally, from start to finish.

Major breakthrough, project back on tracks

After over a year of crunching the IPv4 CGN traversal problem at the back of my mind, it finally clicked! Or, more appropriately called, it banged!
In fact, this click (or bang, or whatever else i should call it) is such a major breakthrough, with such immense potential implications, that i have to refrain from saying much about it on this blog before filing a provisional patent; but what i can say though is that i'm 99% confident i found a novel algorithm that can break through all the IPv4 CGN types that are out there in the wild - and this is not just in theory, i actually tested it for over two weeks on all the mobile connections that i could find, on multiple networks, in eight countries around the world (with a few more tests pending at the time of writing). It did take me three weeks to refine the algorithm down to its current details (and it was quite a bumpy ride), but the bottom line is that i now have a working solution for full P2P/IPv4 connectivity, down to the most minute details, so that i no longer have to wait for God knows how many more months (or years?) to see how the IPv6 dust will eventually settle; that is, i can start working on the production-quality implementation of P2P OS right now.

So the next step: full-throttle fund raising campaign for developing the release-version of P2P OS (i.e. with youtube promo, kickstarter project, call-a-friend, and whatever else it will take to get them wheels turning). The stars aligned, the time has come, let the fun begin!

PS
About the patent thingie, that's just for playing safe, be cool :) - this project will be open source after all ("networking for the masses", remember?).

Wrapping things up with a call for partners

After almost a year of hard work, i eventually realized that the P2P OS project is facing some very serious threats which i just don't feel capable of negotiating all by myself, so i decided to wrap things up with this "Call for partners" post. In a nutshell, here's a super-condensed list of the key things that might be of interest for a potential partner to this project:

• what i managed to do so far is to build a proff-of-concept skype alternative; in fact, the P2P OS networking algorithm has been deigned to allow for a much more robust P2P network than skype's, in the sense that while skype requires a large number of computers with direct internet connection (i.e. public IP or single-router UPnP) in order to support its network (about one such computer for ~100 registered users), P2P OS does not have any such restrictions (i.e. the P2P OS network is self-supporting even if all the computers in the network are running behind routers and/or firewalls).
• i believe this project can become a skype killer because of some key differentiators: it's open-source, it's not collecting any user data, and it consequently can be deployed (and replace skype) in corporate environments
• important: this project is intended to be released under some form of open source license primarily for allowing code inspection and for making it free for home users; however, the program can be released under a dual license, with a commercial license for business users
• i believe money can be made from this project if it is finalized, e.g. by monetizing it as a product (this can be particularly interesting for companies wishing to host their own private network), building services around it (there's a rather large set of options here, including per-user subscriptions for guaranteed uptime), third-party apps advertising on the P2P OS home page (sort of a marketplace, but only for hosting the apps' ads, i.e. without actually hosting the apps themselves), advertized sponsorships, etc, and of course, as a last resort, selling the project copyrights (potential buyers include open source-based and open source-firendly companies).

However, there are at least two major issues with this project and, as far as i can see, with any P2P-based application that someone would want to develop these days, and these two issues are also the sole reason for which i decided to put this project on hold:

• catch 1: the current version of P2P OS has been designed to circumvent the P2P connectivity issues specific to IPv4 in a NAT-based home network environment, but it does not implement provisions for supporting carrier-grade NAT (CGN), and, more importantly, porting the project to IPv6 might not be feasible at all. More specifically, whether any P2P application will be possible on the next-generation IPv6 networks depends on how IPv6 residential gateway devices (RG) will be implemented, and i can't find any way to guestimate, let alone negotiate, this threat because of the very large numbers of factors involved (e.g. whether the high-profile internet companies such as MS, google, or yahoo will lobby for P2P support or not, whether the large ISPs will consider P2P connectivity as being important to the end users or not, whether the present-day high-profile P2P applications such as skype or yahoo will keep their current P2P model or they will transition to a relays-only model - said applications already use relays whenever one user is on a 3G network, etc)
• catch 2: when i started this project i assumed an OS landscape that will allow easy porting and deployment via Qt on all three major desktop OSes Windows, OS X, Linux, plus on at least two major mobile OSes Android and Nokia's MeeGo. However, things have changed dramatically in the sense that (1) porting to the new Windows 8 mobile will be a tough nut to crack (to say the least) because of the limited WinRT API it offers by sandboxing the applications, (2) Nokia abandoned its efforts with MeeGo (so no Nokia port), and (3) Qt has been spun-off (read: all-but-abandoned) by Nokia such that the future of an Android port is uncertain at best; additionally, my paranoid view on the course of MS' and Apple's market strategies suggests that there is a real danger for Windows Phone/Tablet OS and iOS ports to be prohibited by these two OSes. To rise, Qt will probably still be a usable tool for porting P2P OS to the future versions of Windows Desktop, OS X, and Linux for the nest 5 years or so (i.e. porting to these OSes should be fairly easy), but the mobile and tablet ports might prove to be mission impossible.

Keeping the above in mind, should anybody be interested in learning more details about this project and its associated risks should it be continued, please drop me a line.

Update
As i gather more info on the current trends in CGN deployments (reading some articles e.g. from CISCO and Juniper on this topic and talking to industry insiders), adding full CGN support to P2P OS (or any other P2P application for that matter) increasingly seems technically impossible because of the algorithms that ISPs are enabling on their CGNs (namely, most ISPs seem to opt for what it's called EDM - endpoint dependent mapping, which maximizes the reuse of the CGN ports but breaks P2P), such that an IPv4-only P2P OS implementation will probably be impossible for the coming years when CGNs will be the IPv4 connectivity norm for the end user.

Signing off, very likely permanently

There is some good news, some bad news, and a killer (or maybe just artificially-induced coma - this remains to be seen) conclusion.

The good news:
There seems to be a way of shouldering one billion users on a P2P network with just some $1,000/mo central server traffic: it's called Kademlia (seminal article, wikipedia, search). Kademlia essentially creates an overlay network of micro-servers which, in turn, sustain the rest of the network users, while a central server is only used to log on to the network. But there's a catch: in order to distribute micro-server services over a P2P network there have to be at least ~1% of the participating peers that can act as micro-servers, i.e. they must:

1. be directly connected to the internet (i.e. they are not behind a NAT or firewall)
2. have relatively stable connections, i.e. they should stay connected tens of minutes after they completed a network operation (e.g. a chat session, a file transfer, etc)

Both of the above conditions are easily met in today's internet topology: (1) is achieved by any peer that has maximum one router and the router is UPnP-enabled, and (2) is achievable by having the P2P application remain running in the background for a certain amount of time after a specific p2p session ends, or by making the sessions themselves last a relatively long time. BTW, the most successful p2p applications currently available - Skype and BitTorrent - both do their best to enforce the above two conditions upon all the nodes on which it they are installed (by remaining online even after you click their 'close' button, and by turning on UPnP by default).

The bad news:
In brief, the bad news is that the good news don't do me much good. Because:

• a) i no longer trust that (1) will be maintained in the future. People increasingly install cascaded routers in their homes, and UPnP does not give any signs whatsoever that it is willing to address this issue (i.e. make a computer directly accessible on the internet when connected through cascaded routers). Furthermore, ISPs can deploy new methods of preventing p2p applications from running at any time (and i don't mean filters, but rather generic methods such as CGNs and firewalls, and i really don't think this is very far fetched)
• b) the new mobile communications paradigm will become an increasingly significant burden that a depleting pool of directly-connected peers will have to deal with (because mobile data plans are always offered through operator NATs - but, even if they weren't, a mobile peer can't be used a a server because of mobile traffic costs). And BTW, i have this feeling that one of the main reasons for which skype was sold was the realization that the pool of "supernodes" (skype's terminology for shamelessly using people's bandwidth and making $8 billions out of this scheme) is depleting and some big-pockets company will eventually need to step in and shoulder the network with dedicated servers of their own (sure, that's just a hunch, but it will be interesting to see if/when that "use UPnP" checkbox will go away from skype's connection settings, cause that'd pretty much say that skype fully transitioned to in-house supernodes)
  
  Update
  It happened: arstechnica.com/business/news/2012/05/skype-replaces-p2p-supernodes-with-linux-boxes-hosted-by-microsoft.ars (and never mind the M$ lady's reply, it's damage control nonsense)

My killer conclusion:
At this point in time i'm incapable of evaluating just how serious the above challenges (1) & (2) are, let alone other unforeseen issues that might creep in; and since implementing Kademlia would require anywhere from 6 moths to one year of hard work, i'm just not ready to plunge into such an effort alone and empty-handed.

So i stop. Very likely for good.

P2P OS in a nutshell

The P2P OS project aims at creating an open-source Peer-to-Peer Network Operating System that will enable users to communicate and share with each other without relying on any centralized infrastructure and/or service provider.

A top-level technical overview
From a technical perspective, the core of P2P OS will consist of a P2P Application Server that will implement an open-spec peer-to-peer communication API, thus enabling P2P OS-based applications to seamlessly connect with each other inside a transparently-managed P2P network.

• for example, a P2P Messenger text-only application will be able to simply request to the underlying P2P operating system to connect to [a copy of itself running on] a remote user's computer, and then send and receive messages without dealing with any of the P2P network connection details:
  
  
  
  
  
• additionally, a P2P application running on a user's computer will be able to start other "helper applications" on said computer, and thus it may both extend its own functionality via plug-ins, or it may even become a service provider (i.e. a server) for other, totally distinct, third-party applications
• for example, the above-mentioned text-based chat application can start, and fully control the operation of, a voice/video chat plug-in (which is itself a separate application), thus effectively integrating peer-to-peer voice/video chat functionality in the original (text-only) chat application:

The peer-to-peer application server module will be the first OS component to be developed, while additional kernel and userland components will be added at a later stage, with the end goal of supplying P2P OS with a comprehensive set of system management functions in areas such as the management of distributed resources, task control, creation and management of peer-to-peer overlay networks, system monitoring and diagnosis tools, etc.

P2P OS will be initially developed on the Windows platform with Windows-specific development tools, but should the project reach a stage where it becomes usable it will be ported to Nokia's Qt framework on Windows, Linux, MacOS, and MeeGo. Additionally, P2P OS will also be ported to all the mobile operating systems for which Qt support will be available, with a primary candidate being the Android OS when/if Qt support for Android will become adequate.

A few words on the business potential
Currently there are several peer-to-peer products on the market that share some common design elements with P2P OS (e.g. Yahoo messenger, Skype, Google talk, and various file sharing applications, all use some form of peer-to-peer communication), but all these products are a far cry from P2P OS in terms of both their technological foundation and their objectives: more specifically, all said products except for Skype are not pure peer-to-peer applications in that they rely on some form of centralized resources - i.e. they have the fundamental problem of requiring continuous and substantial financial backing, none of them is intended as a foundation platform for third-party applications, and none of them is open-source software. Given the above, a number of distinctive business models can be thought of which can add a business component to P2P OS, all without compromising its quintessential free public service role (i'll write a mode detailed post on this topic sometime in the coming weeks).

Support this project

If you consider this project interesting/meaningful/valuable in any way, please help spread the word to as many people you can, may they be software developers, potential sponsors and/or angel investors interested in supporting free software, or just about any other people you think might be willing to further spread the word to their own circle(s) of friends and colleagues.