Log in

No account? Create an account
Pete Zaitcev's Journal [entries|friends|calendar]
Pete Zaitcev

[ userinfo | livejournal userinfo ]
[ calendar | livejournal calendar ]

Linode Object Storage [01 Nov 2019|10:20am]

Linode made their Object Storage product generally available. They dropped the Swift API that they offered in the beta, and only rolled with S3.

They are careful not to mention what implementation underpins it, but I suspect it's Ceph RGW. They already provide Ceph block storage as public service. Although, I don't know for sure.

[link] post comment

Samsung shutting down CPU development in Austin [29 Oct 2019|02:58pm]

An acquaintance of mine was laid off from Samsung. He was a rank-and-file ASIC designer and worked on FPU unit for Samsung's new CPU. Another acquaintance, a project manager in the silicon field, relayed that supposedly ARM developed a new CPUs that are so great, that all competitors gave up and folded their CPU development, resulting in the layoffs. The online sources have details.

In the same time they gave up on in-house cores, Samsung announced Exynos 990, a standalone version of the 980, based on the successors of the Cortex family, developed by ARM, of course.

As someone said on Glassdoor, "great place to work until you're laid off".

[link] post comment

?fbclid [26 Oct 2019|10:30pm]

As some of you might have noticed, Facebook started adding a tracking token to all URLs as a query string "?fbclid=XXXXXXXXX". I don't know how it works, exactly. Perhaps it rattles to FB when people re-inject these links into FB after they cycle through other social media. Either way, today I found a website that innocently fails to work when shared on FB: Whomp. If one tries to share a comic strip, such as "It's Training Men", FB appends its token, and makes the URL invalid.

[link] 6 comments|post comment

Github [24 Sep 2019|03:06pm]

A job at LinkedIn in my area includes the following instruction statement:

Don't apply if you can't really write code and don't have a github profile. This is a job for an expert level coder.

I remember the simpler times when LWN authors fretted about the SourceForge monopoly capturing all FLOSS projects.

P.S. For the record, I do have a profile at Github. It is required in order to contribute to RDO, because it is the only way to login in their Gerrit and submit patches for review. Ironically, Fedora offers a single sign-on with FAS and RDO is a Red Hat sponsored project, but nope — it's easier to force contributors into Github.

[link] 3 comments|post comment

Docker Block Storage... say what again? [30 Aug 2019|12:52pm]

Found an odd job posting at the website of Rancher:

What you will be doing

  • Design and implement a block storage solution for Docker containers
  • Working on development of various aspects of the storage stack: consistency, reliability, replication and performance
  • Using Go for product development

Okay. Since they talk about consistency and replication together, this thing probably provides actual service, in addition to the necessary orchestration. Kind of the ill-fated Sheepdog. They may under-estimate the amount of work necesary, sure. Look no further than Ceph RBD. Remember how much work it took for a genius like Sage? But a certain arrogance is essential in a start-up, and Rancher only employs 150 people.

Also, nobody is dumb enough to write orchestration in Go, right? So this probably is not just a layer on top of Ceph or whatever.

Well, it's still possible that it's merely an in-house equivalent of OpenStack Cinder, and they want it in Go because they are a Go house and if you have a hammer everything looks like a nail.

Either way, here's the main question: what does block storage have to do with Docker?

Docker, as far as I know, is a container runtime. And containers do not consume block storage. They plug into a Linux kernel that presents POSIX to them, only namespaced. Granted, certain applications consume block storage through Linux, that is why we have O_DIRECT. But to roll out a whole service like this just for those rare appliations... I don't think so.

Why would anyone want block storage for (Docker) containers? Isn't it absurd? What am I missing and what is Rancher up to?


The key to remember here is that while running containers aren't using block storage, Docker containers are distributed as disk images, and they get a their own root filesystem by default. Therefore, any time anyone adds a Docker container, they have to allocate a block device and dump the application image into it. So, yes, it is some kind of Docker Cinder they are trying to build.

See Red Hat docs about managing Docker block storage in Atomic Host (h/t penguin42).

[link] 1 comment|post comment

POST, PUT, and CRUD [15 Aug 2019|02:10pm]

Anyone who ever worked with object storage knows that PUT creates, GET reads, POST updates, and DELETE deletes. Naturally, right? POST is such a strange verb with oddball encodings that it's perfect to update, while GET and PUT are matching twins like read(2) and write(2). Imagine my surprise, then, when I found that the official definition of RESTful makes POST create objects and PUT update them. There is even a FAQ, which uses sophistry and appeals to the authority of RFCs in order to justify this.

So, in the world of RESTful solipcism, you would upload an object foo into a bucket buk by issuing "POST /buk?obj=foo" [1], while "PUT /buk/foo" applies to pre-existing resources. Although, they had to admit that RFC-2616 assumes that PUT creates.

All this goes to show, too much dogma is not good for you.

[1] It's worse, actually. They want you to do "POST /buk", and receive a resource ID, generated by the server, and use that ID to refer to the resource.

[link] post comment

Comment to 'О цифровой экономике и глобальных проблемах человечества' by omega_hyperon [10 Aug 2019|05:46am]
Я это всё каждый день слышу. Эти люди берут вполне определившуюся тенденцию к замедлению научно-технического прогресса, и говорят - мы лучше знаем, что человечеству нужно. Отберите деньги у недостойных, и дайте таким умным как я, и прогресс снова пойдёт. А заодно защитим природу! И всегда капитализм виноват.

О том, что цивилизация топчется на месте, спору нет. А вот пара вещей о которых этот гандон умалчивает.

Во-первых, если отнять деньги у Диснея и отдать их исследовательскому институту, то денег не будет. Вроде по-русски говорит, а такого простого урука из распада СССР не вынес. Американская наука разгромила советскую науку во времена НТР прежде всего потому, что капиталистическая экономика предоставила экономическую базу для этой науки, а социалистическая экономика была провальной.

Вообще, если сравнить бюджет Эппла и Диснея с бюджетом Housing and Urban Development и аналогичных учреждений, то там разница на 2 порядка. Если кто-то хочет дать науке больше денег, то нужно не грабить Дисней, а прекратить давать халявщикам бесплатное жильё. Замедление науки и капитализма идут рука об руку и вызваны государственной политикой, а не каким-то там биткойном.

Во-вторых, кто вообще верит этим шарлатанам? Нам забивали баки про детей в Африке десятилетиями, а за время ужасного голода в Эфиопии её население увеличилось с 38 миллионов до 75 миллионов. То же самое произошло с белыми медведями. Площадь лесов на планете растёт. Допустим в Бразилии срубили какие-то леса под паздбища... Но кто в это поверит?

Этот кризис экспертизы - не шутка. Боязнь вакцин создана не капитализмом и биткойном, а загниванием и распадом системы научных исследований в целом. Он не назвал институт, бюджет которого он сравнил с Диснеем, а вот интересно, сколько там бездельников среди сотрудников.

Коллапс науки отражается не только в том как публика утратила веру в учёних. Объективные показатели тоже просели. Подтверждаемость публикаций очень плохая, и идёт вниз. Тоже биткойн виноват?

В обшцем большая часть этого нытя мне видится крайне вредной. Если он не в состоянии диагностировать причины кризиса, предлогаемые решения ничего нам не дадут, и биодиверии не прибавится.

View the entire thread this comment is a part of

[link] 4 comments|post comment

Swift is 2 to 4 times faster than any competitor [24 Jul 2019|09:38pm]

Or so they say, at least for certain workloads.

In January of 2015 I led a project to evaluate and select a next-generation storage platform that would serve as the central storage (sometimes referred to as an active archive or tier 2) for all workflows. We identified the following features as being key to the success of the platform:

  • Utilization of erasure coding for data/failure protection (no RAID!)
  • Open hardware and the ability to mix and match hardware (a.k.a. support heterogeneous environments)
  • Open source core (preferred, but not required)
  • Self-healing in response to failures (no manual processes required, like replacing a drive)
  • Expandable online to exabyte-scale (no downtime for expansions or upgrades)
  • High availability / fault tolerance (no single point of failure)
  • Enterprise-grade support (24/7/365)
  • Visibility (dashboards to depict load, errors, etc.)
  • RESTful API access (S3/Swift)
  • SMB/NFS access to the same data (preferred, but not required)

In hindsight, I wish we would have included two additional requirements:

  • Transparently tier and migrate data to and from public cloud storage
  • Span multiple geographic regions while maintaining a single global namespace

We spent the next ~1.5 years evaluating the following systems:

  • SwiftStack
  • Ceph (InkTank/RedHat/Canonical)
  • Scality
  • Cloudian
  • Caringo
  • Dell/EMC ECS
  • Cleversafe / IBM COS
  • HGST/WD ActiveScale
  • NetApp StorageGRID
  • Nexenta
  • Qumulo
  • Quantum Lattus
  • Quobyte
  • Hedvig
  • QFS (Quantcast File System)
  • AWS S3
  • Sohonet FileStore

SwiftStack was the only solution that literally checked every box on our list of desired features, but that’s not the only reason we selected it over the competition.

The top three reasons behind our selection of SwiftStack were as follows:

  • Speed – SwiftStack was—by far—the highest-performing object storage platform — capable of line speed and 2-4x faster than competitors. The ability to move assets between our “tier 1 NAS” and “tier 2 object” with extremely high throughput was paramount to the success of the architecture.
  • [...]

Note: While SwiftStack 1space was not a part of the SwiftStack platform at the time of our evaluation and purchase, it would have been an additional deciding factor in favor of SwiftStack if it had been.

Interesting. It should be noted that performance of Swift is a great match for some workloads, but not for others. In particluar, Swift is weak on small-file workloads, such as Gnocchi, which writes a ton of 16-byte objects again and again. The overhead is a killer there, and not just on the wire: Swift has to update its accounting databases each and every time a write is done, so that "swift stat" shows things like quotas. Swift is also not particularly good at HPC-style workloads, which benefit from a great bisectional bandwidth, because we transfer all user data through so-called "proxy" servers. Unlike e.g. Ceph, Swift keeps the cluster topology hidden from the client, while a Ceph client actually tracks the ring changes, placement groups and their leaders, etc.. But as we can see, once the object sizes start climbing and the number of clients increases, Swift rapidly approaches the wire speed.

I cannot help noticing that the architecture in question has a front-facing cache of pool (tier 1), which is what the ultimate clients see instead of Swift. Most of the time, Swift is selected for its ability to serve tens of thousands of clients simultaneously, but not in this case. Apparently, the end-user invented ProxyFS independently.

There's no mention of Red Hat selling Swift in the post. Either it was not part of the evaluation at all, or the author forgot about it for the passing of time. He did list a bunch of rather weird and obscure storage solutions though.

[link] post comment

PostgreSQL and upgrades [07 Jun 2019|09:04am]

As mentioned previously, I run a personal Fediverse instance with Pleroma, which uses Postgres. On Fedora, of course. So, a week ago, I went to do the usual "dnf distro-sync --releasever=30". And then, Postgres fails to start, because the database uses the previous format, 10, and the packages in F30 require format 11. Apparently, I was supposed to dump the database with pg_dumpall, upgrade, then restore. But now that I have binaries that refuse to read the old format, dumping is impossible. Wow.

A little web searching found an upgrader that works across formats (dnf install postgresql-upgrade; postgresql-setup --upgrade). But that one also copies the database, like a dump-restore procedure would. What if the database is too large for this? Am I the only one who finds these practices unacceptable?

Postgres was supposed to be a solid big brother to a boisterous but unreliable upstart MySQL, kind of like Postfix and Exim. But this is just such an absurd fault, it makes me think that I'm missing something essential.

UPDATE: Kaz commented that a form of -compat is conventional:

When I've upgraded in the past, Ubuntu has always just installed the new version of postgres alongside the old one, to allow me to manually export and reimport at my leisure, then remove the old version afterward. Because both are installed, you could pipe the output of one dumpall to the psql command on the other database and the size doesn't matter. The apps continue to point at their old version until I redirect them.

Yeah, as much as I can tell, Fedora does not package anything like that.

[link] 1 comment|post comment

Pi-hole [03 Jun 2019|09:37am]

With the recent move by Google to disable the ad-blockers in Chrome (except for Enterprise level customers[1]), the interest is sure to increase for methods of protection against the ad-delivered malware, other than browser plug-ins. I'm sure Barracuda will make some coin if it's still around. And on the free software side, someone is making an all-in-one package for Raspberry Pi, called "Pi-hole". It works by screwing with DNS, which is actually an impressive demonstration of what an attack on DNS can do.

An obvious problem with Pi-hole is what happens to laptops when they are outside of the home site protection. I suppose one could devise a clone of Pi-hole that plugs into the dnsmasq. Every Fedora system runs one, because NM needs it in order to support the correct lookup on VPNs {Update: see below}. The most valuable part of Pi-hole is the blocklist, the rest is just scripting.

[1] "Google’s Enterprise ad-blocking exception doesn’t seem to include G Suite’s low and mid-tier subscribers. G Suite Basic is $6 per user per month and G Suite Business is $12 per user month."

UPDATE: Ouch. A link by Roy Schestovitz made me remember how it actually worked, and I was wrong above: NM does not run dnsmasq by default. It only has a capability to do so, if you want DNS lookup on VPNs work correctly. So, every user of VPN enables "dns=dnsmasq" in NM. But it is not the default.

UPDATE: A reader mentions that he was rooted by ads served by Space.com. Only 1 degree of separation (beyond Windows in my family).

[link] 4 comments|post comment

Google Fi [20 May 2019|10:39am]

Seen an amusing blog post today on the topic of the hideous debacle that is Google Fi (on top of being a virtual network). Here's the best part though:

About a year ago I tried to get my parents to switch from AT&T to Google Fi. I even made a spreadsheet for my dad (who likes those sorts of things) about how much money he could save. He wasn’t interested. His one point was that at anytime he can go in and get help from an AT&T rep. I kept asking “Who cares? Why would you ever need that?”. Now I know. He was paying almost $60 a month premium for the opportunity to able to talk to a real person, face-to-face! I would gladly pay that now.

Respect your elders!

[link] post comment

YAML [04 May 2019|01:48pm]

Seen in a blog entry by Martin Tournoij (via):

I’ve been happily programming Python for over a decade, so I’m used to significant whitespace, but sometimes I’m still struggling with YAML. In Python the drawbacks and loss of clarity are contained by not having functions that are several pages long, but data or configuration files have no such natural limits to their length.


YAML may seem ‘simple’ and ‘obvious’ when glancing at a basic example, but turns out it’s not. The YAML spec is 23,449 words; for comparison, TOML is 3,339 words, JSON is 1,969 words, and XML is 20,603 words.

There's more where the above came from. In particular, the portability issues are rather surprising.

Unfortunately for me, OpenStack TripleO is based on YAML.

[link] post comment

Fraud in the material world [02 May 2019|12:52pm]

Wow, they better not be building Boeings from this crap:

NASA Launch Services Program (LSP) investigators have determined the technical root cause for the Taurus XL launch failures of NASA’s Orbiting Carbon Observatory (OCO) and Glory missions in 2009 and 2011, respectively: faulty materials provided by aluminum manufacturer, Sapa Profiles (SPI). LSP’s technical investigation led to the involvement of NASA’s Office of the Inspector General and the U.S. Department of Justice (DOJ). DOJ’s efforts, recently made public, resulted in the resolution of criminal charges and alleged civil claims against SPI, and its agreement to pay $46 million to the U.S. government and other commercial customers. This relates to a 19-year scheme that included falsifying thousands of certifications for aluminum extrusions to hundreds of customers.

BTW, those costly failures probably hastened the sale of Orbital to ATK in 2015. There were repercussions for the personnell running the Taurus program as well.

[link] 1 comment|post comment

Swift(stack) bragging today [28 Mar 2019|10:54pm]

On their official blog:

Over the last several months, SwiftStack has been busy helping two large autonomous vehicle customers. These data pipelines are distributed across edge (vehicle sensors) to core (data center) to cloud/multi-cloud locations, and are challenged with ingest, labeling, training, inferencing, and retaining data at scale. [...] one deployment is handling more than a petabyte of data per week, with four thousand GPU cores from NVIDIA DGX-1 servers fed with 100 GB/s of throughput from SwiftStack cluster.

I suspect the task-queue expirer could be helpful at this. Although, if you're uploading 1 PB per week, it takes about a year to fill out a cluster as big as Turkcell's.

Apparently the actual storage is provided by Cisco UCS S3260. Some of our customers use Cisco UCS to run Swift too. I always thought about Cisco as a networking company, but it's different nowadays.

[link] post comment

Suddenly RISC-V [28 Feb 2019|01:51pm]

I knew about that thing because Rich Jones was a fan. Man, that guy is always ahead of the curve.

Coincidentially, a couple of days ago Amazon announced support for RISC-V in FreeRTOS (I have no idea how free that thig is. It's MIT license, but with Amazon, it might be patented up the gills.).

[link] 2 comments|post comment

Mu accounts [28 Feb 2019|11:06am]

Okay, here's the breakdown:

@pro: Programming, computers, networking, maybe some technical fields. It's basically migrated from SeaLion and is the main account of interest for the readers of this journal.

@stuff: Pictures of butterflies, gardening, and general banality.

@gat: Boomsticks.

@avia: Flying.

@union: Politics.

@anime: Anime, manga, and weaboo. Note that Ani-nouto is still officially at Smug.

Thinking about adding @cars and @space, if needed.

You can subscribe from any Fediverse instance, just hit the "Remote follow" button.

[link] post comment

Multi-petabyte Swift cluster [28 Feb 2019|10:44am]

In a Swift numbers post in 2017, I mentioned that the largest known cluster had about 20 PB. It is 2019 now and I just got a word that TurkCell is operating a cluster with 36 PB, and they are looking at growing it up to 50 PB by the end of the year. The information about its make-up is proprietary, unfortunately. The cluster was started in Icehouse release, so I'm sure there was a lot of churn and legacy, like 250 GB drives and RHEL 6.

[link] post comment

Elixir of your every fear [24 Feb 2019|07:33pm]

TFW you consider an O'Reily animal-cover book and the blurb says:

Authors Simon St. Laurent and J. David Eisenberg show you how Elixir combines the robust functional programming of Erlang with an approach that looks more like Ruby, and includes powerful macro features for metaprogramming.

[link] post comment

Mu! [22 Feb 2019|03:14pm]

In the past several days, I innaugurated a private Fediverse instance, "Mu", running Pleroma for now. Although Mastodon is the dominant implementation, Pleroma is far easier to install, and uses less memory on small, private instances. By doing this, I'm bucking the trend of people hating to run their own infrastructure. Well, I do run my own e-mail service, so, what the heck, might as well join the Fediverse.

So far, it was pretty fun, but Pleroma has problem spots. For example, Pleroma has a concept of "local accounts" and "remote accounts": local ones are normal, into which users log in at the instance, and remote ones mirror accounts on other instances. This way, if users Alice@Mu and Bob@Mu follow user zaitcev@SLC, Mu creates a "remote" account UnIqUeStRiNg@Mu, which tracks zaitcev@SLC, so Alice and Bob subscribe to it locally. This permits to send zaitcev's updates over the network only once. Makes sense, right? Well... I have a "stuck" remote account now at Mu, let's call it Xprime@Mu and posit that it follows X@SPC. Updates posted by X@SPC are reflected in Xprime@Mu, but if Alice@Mu tries to follow X@SPC, she does not see updates that Xprime@Mu receives (the updates are not reflected in Alice's friends/main timeline) [1]. I asked at #pleroma about it, but all they could suggest was to try and resubscribe. I think I need to unsubscribe and purge Xprime@Mu somehow. Then, when Alice resubscribes, Pleroma will re-create a remote, say Xbis@Mu, and things hopefully ought to work. Well, maybe. I need to examine the source to be sure.

Unfortnately, aside from being somewhat complex by its nature, Pleroma is written in Elixir, which is to Erlang what Kotlin is to Java, I gather. Lain explains it thus:

As I had written a social network in Ruby for my work at around that time, I wanted to apply my [negative] experience to a new project. [...] This was also to get some experience with Elixir and the Erlang ecosystem, which seemed like a great fit for a fediverse server — and I think it is.

and to re-iterate:

When I started writing Pleroma I was already writing a social network in Ruby for my day job. Because of that, I knew a lot about the pain points of doing it with Ruby, mostly the bad performance for anything involving concurrency. I had written a Bittorrent DHT client in Elixir, so I knew that it would work well for this kind of software. I was also happy to work with functional programming again, which I like very much.

Anyway, it's all water under the bridge, and if I want to understand why Xprime@Mu is stuck, I need to learn Elixir. Early signs are not that good. Right away, it uses its own control entity that replaces make(1), packaging, and a few other things, called "mix". Sasuga desu, as they say in my weeb neighbourhood. Every goddamn language does that nowadays.

[1] It's trickier, actually. For an inexplicable reason, Alice sees some updates by X: for example, re-posts.

[link] post comment

Feynman on discussions among great men [11 Feb 2019|01:52pm]

One of the first experiences I had in this project at Princeton was meeting great men. I had never met very many great men before. But there was an evaluation committee that had to try to help us along, and help us ultimately decide which way we were going to separate the uranium. This committee had men like Compton and Tolman and Smyth and Urey and Rabi and Oppenheimer on it. I would sit in because I understood the theory of how our process of separating isotopes worked, and so they'd ask me questions and talk about it. In these discussions one man would make a point. Then Compton, for example, would explain a different point of view. He would say it should be this way, and was perfectly right. Another guy would say, well, maybe, but there's this other possibility we have to consider against it.

So everybody is disagreeing, all around the table. I am surprised and disturbed that Compton doesn't repeat and emphasize his point. Finally, at the end, Tolman, who's the chairman, would say, ``Well, having heard all these arguments, I guess it's true that Compton's argument is the best of all, and now we have to go ahead.''

It was such a shock to me to see that a committee of men could present a whole lot of ideas, each one thinking of a new facet, while remembering what the other fella said, so that, at the end, the discussion is made as to which idea was the best — summing it all up — without having to say it three times. These were very great men indeed.

Life on l-k before CoC.

[link] 1 comment|post comment

[ viewing | most recent entries ]
[ go | earlier ]