Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It's been close to 1.5 years. I've been using both since the beginning and can see pretty well how quickly both systems pick up updates. But let's just look at release delays (in days since the official RHEL is shipped):

  ver   Alm  Rocky
  8.4   8    34
  8.5   3    6
  8.6   2    6
  9.0   9    58
Not seeing any patterns here?

One of them is being done by a team that's been shipping another Linux distribution for a decade and has the whole process streamlined and automated. The other started by loud release announcements, creating Slack groups and marketing materials, only then going for solving the technical stuff. I think I've made my choice pretty much right then and there.

Seeing how Rocky guys behaved towards the community (like their refusal to go to a popular Linux podcast unless the host was willing to forego any comparisons with other Linux distributions), and these release delays proved that.



It's amusing that you're presenting Alma as the veterans, seeing as how several founding members of Rocky were responsible for starting CentOS in the first place.

But you're right, they're both RHEL clones, so it's only worth differentiating based on externalities. Rocky is backed by industry veterans and part of the 9.0 delay was so they could dogfood Peridot. Alma is backed by a web company who spent the majority of the past couple years Valley-washing their Russian origins. A while back I watched their CEO beg their executive team to cut ties with Russian media sites. Rockey had a community governance model first, they had a distro-dedicated SecureBoot solution first (Alma 'borrowed' CloudLinux's), and so forth. If your metric is 'get package releases to my AWS fleet first' then Alma is winning. For all the rest of the provisioning and longevity issues, Rocky is the winner. It's all a matter of priorities.


I don't quite understand "provisioning and longevity issues, Rocky is winning."

As a user of CentOS looking for a replacement, Alma and Rocky should be 100% identical. The _only_ difference is delay after RHEL launches.


Alma is not as RHEL-compatible as rocky, they use subkeys for signing whereas rocky just uses their signing keys. This is enough that I can’t use alma for building my rpm’s using mock.


I'd love to see a technical writeup of the differences. If Rocky genuinely were better, a 5-week delay vs Alma would probably be ok, at least for the original release, as long as the dot releases are faster. No one wants security vulnerabilities delayed by 5 weeks.


This is not correct. The issue is trying to use Alma 8’s mock chroot on a CentOS 7 host. The (older) versions of yum and rpm there don’t support subkeys.

There is a Red Hat BZ issue here: https://bugzilla.redhat.com/show_bug.cgi?id=2017069


Do RHEL/Rocky/CentOS Stream 8's mock chroots work on CentOS 7 hosts? If so, then this does seem like a legitimate binary incompatibility.


That is not what binary compatibility means when talking about RHEL. It refers to the actual package ABIs and the compatibility levels we assign them:

https://access.redhat.com/articles/rhel-abi-compatibility

https://access.redhat.com/articles/rhel8-abi-compatibility

https://access.redhat.com/articles/rhel9-abi-compatibility

This would be closer to a bug-for-bug compatibility issue, as a result of an implementation change on the infra side of things. It's not due to the OS which is bug-for-bug compatible with its origin, RHEL 8.

While it would be nice if EL7 stacks are considered by the rebuild distributions, it's not a requirement and they are free to use features supported by the platforms they're building.


Well, I can do a mock -r almalinux-8-x86_64 bootstrap build on RHEL7. (I can't remember whether there are currently any problems maintaining for RHEL8 on RHEL7, but it has been impossible to maintain packages for RHEL N+1 on RHEL N in the past. I just moved to a RHEL8 VM rather than prat about with 7 more.)


> several founding members of Rocky were responsible for starting CentOS in the first place.

The "several" part is false. Greg is the only one with any claim at all to the early days of CentOS. He created the Caos Foundation, which is where CentOS was started. He believes that makes him the CentOS founder, despite others disputing that claim. None of the other early CentOS folks are working on Rocky.


I wonder what is the issue if the company "borrowed" its own stuff to use in a different context. Alma always looked like the more serious operation and, if nothing else, the company supplying kernelcare might be considered useful, even if it's proprietary, unfortunately.


> several founding members of Rocky were responsible for starting CentOS in the first place

Hum, that's a subjective view of the history of "CentOS" told by Greg Kurtzer...

In practical terms, Greg just helped setup the legal structure around the distribution that Rocky McGough created.

In the 20 years history of CentOS, Greg Kurtzer stayed 2 years at the beginning, doing mainly PR/legal/community stuff, then left when Lance Davis inherited the ownership of the project after Rocky committed suicide.

I'm not sure anyone is calling Greg a "CentOS founding member" except himself, everywhere.


The CentOS project itself called Greg a founding member in a blog post interviewing him in 2019, well before the Red Hat/CentOS events in December 2020.

https://blog.centos.org/2019/03/greg-kurtzer-centos-founder/

For what it's worth, The Register refers to Greg as a CentOS founder in their first Rocky article:

https://www.theregister.com/2020/12/10/rocky_linux

If you have some references to back your assertions, I would very much like to see them. You're not the first person I've seen assert that Greg shouldn't be called a founder of CentOS, but I've yet to see that backed up.

Also, you seem to minimize the non-technical, especially legal, work around setting up and maintaining a distribution, but I don't understand why. In terms of legal work, trademarks need to be registered, non-profits need to be set up to fund critical items like build infrastructure, and more. Each of these seems like exceptionally valuable, if not critical, to the success of a distribution.


That CentOS blog post is an interview with Greg. The source for Greg being the founder is...himself. That post was supposed to be the first in a series of interviews with people who were around in the early days of CentOS, which is mentioned in the second to last paragraph. Those interviews happened, but they were never published because the interviewees disagreed with the premise set forth in the first post about Greg being the founder. It was decided that publishing additional interviews with conflicting claims would not be a good look for the project. All other articles referring to Greg as the founder use this blog post or Greg's own claims elsewhere as the source. Greg is the root source for all claims of himself being the founder. None of the other early CentOS folks have backed up his claims, despite him asking them to. There are even people in the Rocky project that have asked him to stop claiming this. Greg's response is typically to post archive links that show 1) he started the cAos Foundation and 2) CentOS started in the cAos Foundation. These are facts that no one is disputing. What is disputed is whether his level of involvement in CentOS itself merits declaring himself the founder many years after the fact.


Lance did get control of the project, but not due to Rocky's death.

He got it because after he suggested the name (CentOS) and I approved it and it was then accepted by the others, he promised to hand over the domain name which he was squatting on to the Caos Foundation (a 501(c)3).

Jump forward, Lance and the Centos web team had Red Hat trademark violations on the centos.org website and they alienated Red Hat's legal team enough for them to come after me personally. I believed in Red Hat and we needed to correct our actions, so I reached out to Lance and the web team to rectify the situation. My requests were ignored as they didn't care that they were doing a disservice to Red Hat. I forced the situation as much as I could, which was purely political as the Caos Foundation didn't have access or control over the domain. After the PNALV fiasco, matters got worse, and Lance was able to control the project because he retained controlled the domain.

This was how Lance inherited the ownership of the project, well over a year after Rocky passed away. It was a matter of board manipulation (he convinced the Caos Foundation board that it was okay for him to own the domain). Note, he continued being a bad actor even afterwords when he went AWOL and was pocketing donations personally when the developers had to write him a public open letter to hand over the domain.

What I said is not my subjective view of history, it is corroborated fact which was never challenged or questioned until I founded Rocky Linux.

The right question to ask now... Why are some people spreading rumors trying to rewrite history and slander me (and thus Rocky).


> The right question to ask now... Why are some people spreading rumors trying to rewrite history and slander me (and thus Rocky).

Nobody here or anywhere is trying to slander Rocky, or yourself. And everything I said (or others said) cannot be called a "rumor". Everything said is factually correct, there is just a disagreement on the interpretation of these facts.

I will be honest with you, every time I read something where you claim to be a "founder of CentOS", it irritates me, for a number of reasons.

1/ CentOS was not created in a short time. Ideas, source code and history were spread and shared over multiple years and from multiple projects, I tend to consider that "founders" are "recognized as", not "claimed to be". Yet you are very loud about shouting everywhere your "founder" status to promote your Linux distribution. All in all, this status of yours is the sole selling point of Rocky Linux.

2/ There is, IMHO, a difference between "being there during the foundation" and "being a founder". I value the work you did with Caos, but to me being there in 2002 does not make you a "founder".

3/ There is a meaning in the status of founder, it implies some legacy on the future success of the project. Bill gates is a legitimate founder of MSFT because even though he's not there anymore, he did put the company on a track for what it is today.

I do not agree that you have a legacy claim on CentOS' future success. You were let go of the org after 2 years (I know you see it as the board having been "manipulated" by Lance).

TLDR: I do not contest your contribution to CentOS in the early days. But I think calling yourself a "founder" is misleading, and is a borderline dishonest attempt at reaping the success of an org that earned 99% of its respect long after you were let go.


> Nobody here or anywhere is trying to slander Rocky, or yourself. And everything I said (or others said) cannot be called a "rumor".

Did you even read what you wrote? The entire thing is slanderous and based on rumors and you have provided ZERO facts or citations. Others have even pointed you to some links, and if you were really curious, the facts are easily found, go look before you post inaccurate information and perpetuate rumors.

Another thing, I don't appreciate being attacked anonymously when all I'm trying to do is help the community by filling a much needed pain point.

> Everything said is factually correct, there is just a disagreement on the interpretation of these facts.

Actually, what you said is not factually correct and you've provided zero proof. Justify your allegation that me being a founder of CentOS is subjective or you just look like a hater and a troll.

> 1/ CentOS was not created in a short time. Ideas, source code and history were spread and shared over multiple years and from multiple projects, I tend to consider that "founders" are "recognized as", not "claimed to be". Yet you are very loud about shouting everywhere your "founder" status to promote your Linux distribution.

First off, CentOS was absolutely founded in a short time. It's all clearly documented on the Caos email list and easily corroborated by archive.org, but that point is neither here nor there. I led the project from inception to the point where it was literally a household name. If you don't think my contribution was valuable, then you are entitled to that perspective.

> All in all, this status of yours is the sole selling point of Rocky Linux.

Nope, wrong, I never said that and I don't like anyone putting words in my mouth.

The background I have from early days of CentOS is entirely valid and what we did was amazing, but there were of course things we could have done better. Based on this experience, myself and the team leads/board were able to make decisions for the betterment of the project and recapitulate what CentOS did right, and reevaluate what could have been done better.

The Rocky team has done remarkably amazing to bring Rocky Linux out to the community and our growth demonstrates we solved a major pain point for the community. To marginalize all of that because my role with CentOS "irritates" you is just childish.

> 2/ There is, IMHO, a difference between "being there during the foundation" and "being a founder". I value the work you did with Caos, but to me being there in 2002 does not make you a "founder".

That point is accurate, there were a lot of people who were "there during the foundation", and that doesn't make them "founders".

I created the Caos Foundation, I incorporated it, managed it, converted it to a 501(c)3 which then became the origin of CentOS.

Red Hat EOLed the freely available RHL and pivoted it to RHEL thus adversely affecting many people in the Linux and open source community. We (Rocky, Lance, Russ, and myself -- the primary CentOS Founders) decided to rebuild the sources of RHEL with Rocky taking the first stab at it with CentOS3 (which was the current version of RHEL). That was the moment of Founding CentOS. Its initial name was "Caos EL"[1] and it wasn't until a bit later did Lance suggest the name CentOS to me and I, as the project lead, approved it first, then it went to group vote. Rocky was the person who announced the name change from Caos-EL to CentOS on the Caos email list[2] along with letting the world know he was about 99% done with it (this is right before he passed away). Then I continued to lead the CentOS project for another 18'ish months.

> 3/ There is a meaning in the status of founder, it implies some legacy on the future success of the project. Bill gates is a legitimate founder of MSFT because even though he's not there anymore, he did put the company on a track for what it is today.

If you don't consider the contribution I provided the project as valuable, then sorry, but that does not delegitimize my role as a Founder.

> I do not agree that you have a legacy claim on CentOS' future success. You were let go of the org after 2 years (I know you see it as the board having been "manipulated" by Lance).

I've never, not once, tried to substantiate a claim on CentOS' long term success. You are putting words in my mouth. What the team did after me is 100% on them. Their dedication, and work, is an amazing tribute and should not be marginalized by anyone. I was there, I saw what they did and how they held the project, especially after the "CentOS Debacle" and the open letter to Lance. The team did amazing, and it has withstood an amazing set of hurdles. I have nothing but admiration for that team and I'm still friends with many of them.

Again, you are spreading missinformation and rumors. I was not "let go" from the org... CentOS, via Lance, left the Caos Foundation.

Was the board manipulated by Lance? Yes, absolutely. He convinced them that the Caos Foundation 501(c)3 did not need to own the CentOS.org domain assets which in the end was the main leverage he had to remove it out of the Caos Foundation. Couple that with the legal issues he got me into, and the Caos board agreed to let CentOS goto Lance.

> TLDR: I do not contest your contribution to CentOS in the early days. But I think calling yourself a "founder" is misleading, and is a borderline dishonest attempt at reaping the success of an org that earned 99% of its respect long after you were let go.

So you believe that it is misleading and dishonest for someone to call themselves a founder to a project which they helped to create and then go onto lead for years?

-- [1] https://web.archive.org/web/20040909231416/http://www.caosit... [2] https://web.archive.org/web/20040630213827/http://www.caosit...


> doing mainly PR/legal/community stuff

Why are you not considering PR/legal/community stuff as being worthy?


It would frighten me having Kurtzer doing legalities, given his bizarre legal theory on copyright specifically (no longer recorded in the issues of a deleted github repo for me to reference).


Can you elaborate on my bizarre legal theory on copyrights?

Thanks.


Sorry, I don't know why I missed off the sentence. The statement was that putting something under the Berkeley licence used by Singularity made it public domain, i.e. they could strip my employer's copyright, specifically. No answer to whether that was speaking for LBL legal.


Sorry, I don't remember the specific instance of what happened there, but specifically Singularity was never technically "public domain". It was always copyrighted and licensed (firstly with the DOE/LBL license based on the 3-clause BSD license, then just BSD3).

Was the issue on the project copyright, or was it the copyright in the source files?

While I have some experience in open source and copyright, I'm certainly no expert nor do I claim to be. Luckily I have people I can rely on for advice and guidance (e.g. back then, it was probably LBL/UCOP's Tech Transfer), but sometimes things still get mucked up.

Sorry if there was a miscommunication or an error on my part back then.


That's pretty opaque. :(


> Greg Kurtzer stayed 2 years at the beginning, doing mainly PR/legal/community stuff

Heck, that's the very stuff nobody wants to take care of, kudos to him he wanted to spend his time on these.


Faster like Springdale? This is an endurance race. CentOS is the survivor of several clones.

"The best reason we have is our speed. If we assume all RHEL clones are equal in terms of software, people tend to then weigh speed and community size/support very heavily. We get new packages out very very quickly because nearly all the rebuilds can be automated. We had PUIAS 6 out over a month before CentOS 6 came out. The same is true of minor revisions - of CentOS, SL, and PUIAS, we had a 5.8 release out first." - "IAmA Developer for the PUIAS Linux distribution - AMA"


Rocky made it clear they have a new build system which lead to their delay. I think it's called peridot. I would imagine that it's not easy to start fresh in two places at once. Alma has an advantage with a process from CloudLinux already in place and now they've rebranded that build system as Alma and so on. Credit to Alma for keeping up and doing their thing; hopefully they'll be fully decoupled from CL in the future.

CentOS had historically fell behind in release times, but we all of a sudden want to paint others in a bad light for being behind. Did everyone forget about CentOS 7 having average of 30 days delay behind each point release? 7.4 being the most at 43 days. What about CentOS 6.0, with 242 days?

As for their behavior, it takes two to tango. I usually like to thank carlwshill and "conan_kudo" (who should pick a better name since he thinks he's from the anime his picture is from) for being true stars in the open source community and really bringing out not only the best in others, but the best in themselves day in and day out. I'm not sure how anyone can put up with them, regardless of which community you're in (CentOS/Fedora EPEL/others), but who am I to judge, I'm just a user.


I don't have exceptional experience with either Alma or Rocky, but I've been administering enterprise Red Hat and CentOS for years.

1.5 years in enterprise time is hardly any time at all. Heck, it takes that time to approve a budget in some enterprises. Yeah, time-to-release is an important metric, but software compatibility and industry support is really the magic sauce.

In my industry, we'll be using current CentOS 7 installs until EOL, and watch all derivatives with interest over the long term - given they provide anything over current RH ecosystem (RHEL, CentOS Stream, Fedora).




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: