I tinker with software radio as a hobby and I am stuck solving a very basic problem. But first, a background exposition.

Bdale, what have you done to me

Many years ago, I attended an introductory lecture on software radio at a Linux conference we used to have - maybe OLS, maybe LCA, maybe ALS/Usenix even. Bdale Garbee was presenting, who I mostly knew as a Debian guy. He outlined a vision of Software Defined Radio: take what used to be a hardware problem, re-frame it as a software problem, let hackers hack on it.

Back then, people literally had sound cards as receiver back-ends, so all Bdale and his cohorts could do was HF, narrow band signals. Still, the idea seemed very powerful to me and caught my imagination.

A few years ago, the RTL-SDR appeared. I wanted to play with it, but nothing worthy came to mind, until I started flying and thus looking into various aviation data link signals, in particular ADS-B and its relatives TIS and FIS.

Silly government, were feet and miles not enough for you

At the time FAA became serious about ADS-B, two data link standards were available: Extended Squitter aka 1090ES at 1090 MHz and Universal Access Transciever aka UAT at 978 MHz. The rest of the world was converging quickly onto 1090ES, while UAT had a much higher data rate, so permitted e.g. transmission of weather information. FAA sat like a Buridan's ass in front of two heaps of hay, and decided to adopt both 1090ES and UAT.

Now, if airplane A is equipped with 1090ES and airplane B is equipped with UAT, they can't communicate. No problem, said FAA, we'll install thousands of ground stations that re-transmit the signals between bands. Also, we'll transmit weather images and data on UAT. Result is, UAT has a lot of signals all the time, which I can receive.

Before I invent a wheel, I invent an airplane

Well, I could, if I had a receiver that could decode a 1 megabit/second signal. Unfortunately, RTL-SDR could only snap 2.8 million I/Q samples/second in theory. In practice, even less. So, I ordered an expensive receiver called AirSpy, which was told to capture 20 million samples/second.

But, I was too impatient to wait for my AirSpy, so I started thinking if I could somehow receive UAT with RTL-SDR, and I came up with a solution. I let it clock at twice of the exact speed of UAT, a little more than 1 mbit/s. Then, since UAT used PSK2 encoding, I would compare phase angles between samples. Now, you cannot know for sure where the bits fall over your samples. But you can look at decoded bits and see if it's garbage or a packet. Voila, making impossible possible, at Shannon's boundary.

When I posted my code to github, it turned out that a British gentleman by the handle of mutability was thinking about the same thing. He contributed a patch or two, but he also had his own codebase, at which I hacked a bit too. His code was performing better, and it found a wide adoption under the name dump978.

Meanwhile, the AirSpy problem

AirSpy ended collecting dust, until now. I started playing with it recently, and used the 1090ES signal for tests. It was supposed to be easy... Unlike the phase shift of UAT, 1090ES is much simpler signal: raising front is 1, falling front is 0, stable is invalid and is used in the preamble. How hard can it be, right? Even when I found that AirSpy only receives the real component, it seemed immaterial: 1090ES is not phase-encoded.

But boy, was I wrong. To begin with, I need to hunt a preamble, which synchronizes the clocks for the remainder of the packet. Here's what it looks like:

The fat green square line on the top is a sample that I stole from our German friends. The thin green line is a 3-sample average of abs(sample). And the purple is raw samples off the AirSpy, real-only.

My first idea was to compute a "discriminant" function, or a kind of an integrated difference between the ideal function (in fat green) and the actual signal. If the discriminant is smaller than a threshold, we have our preamble. The idea was a miserable failure. The problem is, the signal is noisy. So, even when the signal is normalized, the noise in more powerful signal inflates the discriminant enough that it becomes larger than the discriminant of background noise.

Mind, this is a long-solved problem. Software receiver for 1090ES with AirSpy exists. I'm just playing here. Still... How do real engineers do it?

well I had to write a sysv init script today and I wished it was systemd

— moonman, 21 December 2018

To quoth:

Why not walk down the wider path, using GNU/Linux as DOM0? Well, if you like the kernel Linux, by all means, do that! I prefer an well-engineered kernel, so I choose NetBSD. [...]

Unfortunately, NetBSD's installer now fails on many PCs from 2010 and later. [...]

Update 2018-03-11: I have given up on NetBSD/Xen and now use Gentoo GNU/Linux/Xen instead. The reason is that I ran into stability problems which survived many NetBSD updates.

You have to have a heart of stone not to laugh out loud.

P.S. Use KVM already, sheesh.

P.P.S. This fate also awaits people who don't like SystemD.

With one of the recent Firefox releases (current version is 64), autoplay videos began to play again, although they start muted now [1]. None of the previously-working methods work (e.g. about:config media.autoplay.enabled), the documented preference is not there in 64 (promised for 63: either never happened, or was removed). Extensions that purport to disable autoplay do not work.

The solution that does work is set media.autoplay.default to 1.

Finding the working option required a bit of effort. I'm sure this post will become obsolete in a few months, and add to the Internet noise that makes it harder to find a working solution when Mozilla changes something again. But hey. Everyting is shit, so whatever.

[1] Savour the bitterness of realization that an employee of Mozilla thought that autoplay was okay to permit as long as it was muted.

By whatever chance, I visited an old science laboratory where I played at times when I was a teenager. They still have a pile of old equipment, including the IBM PC XT clone that I tinkered with.

Back in the day, they also had a PDP-11, already old, which had a magnetic tape unit. They also had data sets on those tapes. The PC XT was a new hotness, and they wanted to use it for data visualization. It was a difficult task to find a place that could read the data off the tape and write to 5.25" floppies. Impossible, really.

I stepped in and went to connect the two over RS-232. I threw together a program in Turbo Pascal, which did the job of shuffling the characters between the MS-DOS and the mini, thus allowing to log in and initiate a transfer of the data. I don't remember if we used an ancient Kermit, or just printed the numbers in FORTRAN, then captured them on the PC.

The PDP-11 didn't survive for me to take a picture, but the PC XT did.

First things first: I am sorry for getting passive-aggressive on Twitter, although I was mad and the medium encourages this sort of thing. But this is the world we live in: the way to deal with computers is to google the symptoms, and hope that you don't have to watch a video. Something about this world disagrees with me so much, that I almost boycott Wikipedia and Stackoverflow. "Almost" means that I go very far, even Read The Fine Manuals, before I resort to them. As the path in tweet indicated, I built Ceph from source in order to debug the problem. But as the software stacks get thicker and thicker, source gets less and less useful, or at least it loses competition to googling for symptoms. My only hope at this point is for the merciful death take me away before these trends destroy the human civilization.

Imagine, purely hypothetically, that you were a kernel hacker working for Red Hat and for whatever reason you wanted to find a new challenge at a company with a strong committment to open source. What are the possibilities?

To begin with, as the statistics from the Linux Foundation's 2016 report demonstrate, you have to be stark raving mad to leave Red Hat. If you do, Intel and AMD look interesting (hello, Alan Cox). IBM is not bad, although since yesterday, you don't need to quit Red Hat to work for IBM anymore. Even Google, famous for being a black hole that swallows good hackers who are never heard from again, manages to put up a decent showing, Fuchsia or no. Facebook looks unimpressive (no disrespect to DaveJ intended).

Now, the no-shows. Both of them hail from Seattle, WA: Microsoft and Amazon. Microsoft made an interesting effort to adopt Linux into its public cloud, but their strategy was to make Red Hat do all the work. Well, as expected. Amazon, though, is a problem. I managed to get into an argument with David "dwmw2" Woodhouse on Facebook about it, where I brought up a somewhat dated article at The Register. The central claim is, the lack of Amazon's contribution is the result of the policy rolled all the way from the top.

(...) as far as El Reg can tell, the internet titan has submitted patches and other improvements to very few projects. When it does contribute, it does so typically via a third party, usually an employee's personal account that is not explicitly linked to Amazon.

I don't know if this culture can be changed quickly, even if Bezos suddenly changes his mind.

In a comment on the death of G+, elisteran brought up something that long annoyed me out of all proportion with its actual significance. What do you call a collection of servers communicating through NNTP? You don't call them "INN", you call them "Usenet". The system of hosts communicating through SMTP is not called "Exim", it is called "e-mail". But when someone wants to escape G+, they often consider "Mastodon". Isn't it odd?

Mastodon is merely an implementation of Fediverse. As it happens, only one of my Fediverse channels runs on Mastodon (the Japanese language one at Pawoo). Main one still uses Gnusocial, the anime one was on Gnusocial and migrated to Pleroma a few months ago. All of them are communicating using the OStatus protocol, although a movement is afoot to switch to ActivityPub. Hopefully it's more successful than the migration from RSS to Atom was.

Yet, I noticed that a lot of people fall to the idea that Mastodon is an exclusive brand. Rarely one has to know or care what MTA someone else uses. Microsoft was somewhat successful in establishing Outlook as such a powerful brand to the exclusion of the compatible e-mail software. The maintainer of Mastodon is doing his hardest to present it as a similar brand, and regrettably, he's very successful at that.

I guess what really drives me mad about this is how Eugen uses his mindshare advanage to drive protocol extensions. All of Fediverse implementations generaly communicate freely with one another, but as Pleroma and Mastodon develop, they gradually leave Gnusocial behind in features. In particular, Eugen found a loophole in the protocol, which allows to attach pictures without using up the space in the message for the URL. When Gnusocial displays a message with attachment, it only displays the text, not the picture. This acutally used to be a server setting, in case you want to safe your instance from NSFW imagery and save a little bandwidth. But these days pictures are so prevalent, that it's pretty much impossible to live without receiving them. In this, Eugen has completed the "extend" phase and is moving onto "extinguish".

I'm not sure if this a lost cause by now. At least I hope that members of my social circle migrate to Fediverse in general, and not to Mastodon from the outset. Of course, the implementation does matter when they make choices. As I mentioned, for anything but Linux discussions, pictures are essential, so one cannot reasonably use a Gnusocial instance for anime, for example. And, I can see some users liking Mastodon's UI. And, Mastodon's native app support is better (or not). So yes, by all means, if you want to install Mastodon, or join an instance that's running Mastodon, be my guest. Just realize that Mastodon is an implementation of Fediverse and not the Fediverse itself.

Reactions by G+ inhabitants were better than expected at times. Here's Jon Masters:

For the three people who care about G+: it's closing down. This is actually a good thing. If you work in kernel or other nerdy computery circles and this is your social media platform, I have news for you...there's a world outside where actual other people exist. Try it. You can then follow me on Twitter at @jonmasters when you get bored.

Rock on. Although LJ was designed as a shitty silo, it wasn't powerful enough to make itself useless. For example, outgoing links aren't limited. That said, LJ isn't bulletproof: the management is pushing the "new" editor that does not allow HTML. The point is though, there's a real world out there.

And, some people are afraid of it, and aren't ashamed to admit it. Here's Steven Rostedt in Jon's comments:

In other words, we are very aware of the world outside of here. This is where we avoided that world ;-)

So weak. Jon is titan among his entourage.

Kir enumerated escape plans thus (in my translation):

Where to run, unclear. Not wanting to Facebook, Telegram is kinda a marginal platform (although Google+ marginal too), too lazy to stand up a standalone. Nothing but LJ comes to mind.

One thing that comes across very strongly is how reluctant people are to run their own infrastructure. For one thing, the danger of a devastating DDoS is absolutely real. And then you have to deal with spam. Those who do not have the experience also tend to over-estimate the amount of effort you have to put into running "dnf update" once in a while.

Personally, I think that although of course it's annoying, the time wasted on the infra is not that great, or at least it wasn't for me. The spam can be kept under control with a minimal effort. Or, could be addressed in drastic ways. For example, my anime blog simply does not have comments at all. As far as DoS goes, yes, it's a lottery. But then the silo platform can easily die (like G+), or ban you. This actually happens a lot more than those hiding their heads in the sand like to admit. And you don't need to go as far as to admit to your support of President Trump in order to get banned. Anything can trigger it, and the same crazies that DoS you will also try to deplatform you.

One other idea I was very successful with, and many people have trouble accepting, is having several channels for social posting (obviously CKS was ahead of the time with separating pro and hobby). Lots and lots of G+ posters insist on dumping all the garbage into one bin, instead of separating the output. Perhaps now they'll find a client or device that allows them switch accounts easily.

Back in July, Economist wanted to judge a popularity of programming languages and used ... Google Trends. Python is rocketing up, BTW. Go is not even mentioned.

Unexpectedly in the fediverse:

[...] in my experience, postgres crashes less, and the results are less devastating if it does crash. I've had a mysql crash make the data unrecoverable. On the other hand I have a production postgres 8.1 installation (don't ask) that has been running without problems for over 10 years.

There is more community information and more third-party tools that require mysql, it has that advantage. the client tools for mysql are easier to use because the commands are in plain english ("show tables") unlike postgres that are commands like "\dt+". but if I'm doing my own thing though, I use postgres.

Reiser, move over. There's a new game in town.

The Huawei M3 gave me an unpleasant surprise a short time ago. I had it in my hands while doing something and my daughter (age 30) offered to hold it for me. When I received it back and turned it on, it was factory reset. What happened?

It turned out that it's possible to reset the blasted thing merely by holding it. If someone grabs it and pays no attention to what's on the screen, then it's easy to press and hold the edge power button inadvertently. That brings up a dialog that has 2 touch buttons for power off and reset. The same hand that's holding the power touches the screen and causes the reset (a knuckle where the finger meets the palm does that perfectly).

The following combination of factors makes this happen. 1. Power button is on the edge, and it sticks out. Some tablets like Kindle or Nexus have edges somewhat slanted, so buttons are somewhat protected. Holding the tablet across the face engages the power button. They could at least place the power button on the short edge of the device. 2. The size of the tablet is just large enough that a normal person can hold it with one hand, but has to stretch. Therefore, the base knuckles touch the surface. On a larger tablet, a human hand is not large enough to hold it like that, and on a phone sized device the palm cups, so it does not touch the center of the screen. 3. The protection against accidental reset is essentially absent.

Huawei, not even once.

Google, bring back the Nexus 7, please.

Usually I do not watch TV, but I traveled and saw a few of them in public food intake places and such. What caught my attention were ads for robotics companies, aimed at business customers. IIRC, the companies were called generic names like "Universal Robotics" and "Reach Robotics". Or so I recall, but on second thought, Reach Robotics is a thing, but it focises on gaming, not traditional robotics. But the ads depicted robots doing some unspecified stuff: moving objects from place to place. Not dueling bots. Anyway, what's up with this? Is there some sort of revolution going on? What was the enabler? Don't tell me, it's all the money released by end of the Moore's Law, seeking random fields of application.

P.S. I know about the "Pentagon's Evil Mechanical Dogs" by Boston Dynamics. These were different, manipulating objects in the environment.

A couple Fedora releases back, I noticed that my laptop stopped using IPv6 to access dual-hosted services. I gave RFC-6724 a read, but it was much too involved for my small mind. Fortunately, it contained a simplified explanation:

Another effect of the default policy table is to prefer communication using IPv6 addresses to communication using IPv4 addresses, if matching source addresses are available.

My IPv6 is NAT-ed, so the laptop sees an RFC-4193 address fc00::/7. This does not match the globally assigned address of the external service. Therefore, a matching source address is not available, and things develop from there.

For now, I forced RFC-3484 with gai.conf. Basically, reverted to Fedora 26 behavior.

Butenko was simply the most capable programmer that I've ever worked with. He was also very accomplished. I'm sure everyone has an idea what UNIX v7 was. Although BSD, sockets, and VFS were still in the future, it was a sophisticated OS for its time. Butenko wrote his own OS that was about a peer for the v7 in features (including vi). He also wrote a Fortran 77 compiler with IDE, an SQL database, and a myriad other things. Applications, too: games, communications, industrial control.

I still remember one of our first meetings in the late 1983. I wanted someone to explain me the instruction set of Mitra-15, a French 16-bit mini. Documentation was practically impossible to get back then, especially for undergrads. Someone referred me, and I received a lecture at a smoking area near an elevator, which founded my understanding of computer architecture.

The only time I ever got one up, was when I wrote a utility to monitor processes (years later, top(1) does the same thing). Apparently the concept never occurred to Butenko, who was perfectly capable to analyzing the system with a debugger and profiler. Seeing just my UI, he knocked out a clone in a couple of days. Of course, it was superior in every respect.

Butenko worked a lot. The combination of genius and workaholic was unstoppable. Or maybe they were sides of the same coin.

Unfortunately, Butenko was not in with the open source. He used to post to Usenet, lampooning and dismissing Linux. I suspect once you can code your own Linux any time you want, your perspective changes a bit. This was a part of the way we drifted apart later on. I was plugging on my little corner of Linux, while Butenko was somewhere out in the larger world, revolutionizing computer-intermediated communications.

He died suddenly, from a heart failure. Way too early, I think.

Wait, wasn't this supposed to be our future?

The abridged history, as I recall was as follows. Mike Burton started the work to port Swift to Go in early 2016, inside the Swift tree. As such, it was a community effort. There was even a discussion at OpenStack Technical Committee about allowing development in Go (the TC disallowed it, but later posted some conditions). At the end of the year, I managed to write an object with MIME and collapsed the stock Swift auditor (the one in Python). That was an impetus for PUT+POST, BTW. But in 2017, the RAX cabal - creiht, redbo, and later gholt - weren't very happy with trying to supplicate to the TC, as well as the massive overhead of developing in the established community, and went their own way. In addition to the TC shenagians, the upstream Swift at SwiftStack and Red Hat needed a migration path. A Hummingbird without a support for Erasure Coding was a non-starter, and the RAX group wasn't interested in accomodating that. By the end of 2017, they were completely on their own, and started going off at the deep end by adding database servers and such. They managed to throw off some good ideas about what the next-generation replication ought to look like. But by cutting themselves off Swift they committed to re-capturng the lightning in the bottle anew, and they just could not pull it off.

On reflection, I suspect their chances would be better if they were serious about interoperating with Swift. The performance gains that they demonstrated were quite impressive. But their paymasters at RAX weren't into this community development and open-source toys (not that RAX went through the change of ownership while Hummingbird was going on).

I think a port of Swift to Go is still on the agenda, but it's unknown at this point if it's going to happen.

Saw today at a respectable news site, which does not even nag about adblock:

451

We recognise you are attempting to access this website from a country belonging to the European Economic Area (EEA) including the EU which enforces the General Data Protection Regulation (GDPR) and therefore cannot grant you access at this time. For any issues, e-mail us at xxxxxxxx@xxxxxx.com or call us at xxx-xxx-4000.

What a way to brighten one's day. The phone without a country code is a cherry on top.

P.S. The only fly in this ointment is, I wasn't accessing it from the GDPR area. It was a geolocation failure.

See a mailing list message:

I would like to remove myself entirely from the decision process. // I am not going to appoint a successor.

Regarding the Huston missive, two quotes jumped at me the most. The first is just beautiful:

It may be slightly more disconcerting to realise that your electronic wallet is on a device that is using a massive compilation of open source software of largely unknown origin [...]

Yeah, baby. This moldy canard is still operational.

The second is from the narrative of the smartphone revolution:

Apple’s iPhone, released in 2007, was a revolutionary device. [...] Apple’s early lead was rapidly emulated by Windows and Nokia with their own offerings. Google’s position was more as an active disruptor, using an open licensing framework for the Android platform [...]

Again, it's not like he's actually lying. He merely implies heavily that Nokia came next. I don't think the Nokia blunder even deserve a footnote, but to Huston, Google was too open. Google, Carl!

Got Slasti 2.1 released today, the main feature being a support for Python 3. Some of the changes were somewhat... horrifying maybe? I tried to adhere to a general plan, where the whole of the application operates in unicode, and the UTF-8 data is encoded/decoded at the boundary. Unfortunately, in practice the boundary was rather leaky, so in several places I had to resort to isinstance(). I expected to always assign a type to all variables and fields, and then rigidly convert as needed. But WSGI had its own ideas.

Overall, the biggest source of issues was not the py3 model, but trying to make the code compatible. I'm not going to do that again if I can help it: either py2 or py3, but not both.

UPDATE: Looks like CKS agrees that compatible code is usually too hard. I'm glad the recommendation to avoid Python 3 entirely is no longer operational.