|
I belive I never committed this to writing, but Slasti has a more narrow reason to exist than my general education about webapps. Back when I went to hack on tabled (S3 server) for Garzik, I very much thought that K/V (and maybe NOSQL) had underexplored merit, because I knew how Mixi begat Tokyo Cabinet and such. Surely if such great apps can be built on these APIs... If only we could create a system-level standard... And S3 might work just as well.
About that time I was at a lecture by an IBM cloud lady who mentioned how all these new APIs "promise to liberate researchers from POSIX", and that clearly was too much of a siren song, like microkernels. I carried on, but a few months later Darcy posted an article "Is POSIX Still Relevant?" At the time I still thought that most apps should use those S3 or DB-like APIs, and even commented to that effect on his posting.
It's not like "system-level K/V" was all honey and cream. A most annoying part of working with S3 was the need to pay Amazon for it. Performance was so-so too. Tabled was supposed to fix the first for good, and bring the second under local control. In fact, I rented out a VM at Rackspace because I was going to tinker with S3 front-end for Swift (It's Rackspace's S3 that's just incompatible enough, but very fast. Chuck Tier wrote a kind of S3 translator for it.)
But then input came from an unexpected side. I had to deal with a thing at work, which had two working back-ends: S3 and fs. The S3 side worked with tabled fine. But as users (that is, Aeolus developers) got their hand on it, it became clear that although they are officially supportive of S3, they would always install Hail, poke at it a bit, and then use fs for any real work. I quickly found that the value of having the universal environment was extremely high.
Again, tabled was supposed to address it by being there always. Garzik and I managed to get Hail and tabled into Fedora back in F12. So, there was no need to install a dozen of packages from tarballs or run python setup.py. Just edit a couple of configs, chkconfig tabled on and off you go. But... no sale. They just never did it.
Then it dawned on me that we have apps in big leagues, like Mixi, and apps in small leagues. In first, you have very insular, startup environment, and few people involved (or just one person). Their life turns upon that one website they're hacking on. They think nothing about wriging Cassandra from scratch and/or writing it all in Erlang (Mixi was written in Perl originally!) But in the second league, we have way more developers, who are not as invested, who have no access to good testbeds or production. When they hack on their old PCs, they have to make do. So yeah...
Upon that realizaton I shut down my personal Hail cell for good and started thinking about writing a post "Darcy was right". POSIX is super relevant unless you are committed to destroying Google or something. However, I'm not looking forward to building a custom data consistency layer using POSIX. It seems too much like the old Russian joke about removing tonsils through a butthole. For now, Slasti came out with a filesystem for its backing store. It is a test how far I can carry it before DB pounces. But it has a provision for pluggable back-ends, and I'm going to leave the question of Darcy's wisdom open for now.
UPDATE 2012/01: A few things happened. I tried to work with Gluster (before the company joined Red Hat), but was not successful at that. Could not even get it running reliably. HekaFS was very rough back then, too. Then, I had a look at Glance, the image service of OpenStack, and thus their equivalent of iwhd. Good grief, that was buggy as all get out. Jay and the crew were working like mad trying to get it running. And finally, Aeolus folks started looking at re-engineering all this image service business: pull all the schema or the links between assemblies and deployables into the database, and just use an image store as image store. If they succeed, they will get rid of iwhd. Hurray! Probably a lesson that OpenStack is yet to learn, but we'll see.
|
