quarterly vs. latest

[ASX]

I would think that it would be useful to write a forum announce, and a website page about the upcoming repos,
explaining our choices and also what actions to take for a successsful transition to use our pkg repos.

Volunters welcomed!

[ericbsd]

ASX I had start something in http://www.ghostbsd.org/node/151 but did not put much time in it, as you will see.

[ASX]

ASX I had start something in http://www.ghostbsd.org/node/151 but did not put much time in it, as you will see.

OK, it is a start!

adding a few tip here:

- why our own packages (binary need to be synced between ports and packages, including our own ports)
- why synth (and thanks to J.R. Marino
- repos layout (and reasons for the 4 repos for each arch)
- changes required to use the new repos (from 11.0 only)
- how to fallback to a "previous" build after something went wrong with an update.

[kraileth]

Yes, I can think of one are that would need help, it is in writing the upcoming documentation about using our new pkgs repos, the reasons behind our choices and the need to document a path for a successful transition from using the freebsd repos to use our own repos.

kraileth, would you pick up that task ?

Sure thing! I'll set some time aside for that. How much time would you (roughly) estimate does it take before the repos can be made available for the general public?

adding a few tip here:

- why our own packages (binary need to be synced between ports and packages, including our own ports)
- why synth (and thanks to J.R. Marino
- repos layout (and reasons for the 4 repos for each arch)
- changes required to use the new repos (from 11.0 only)
- how to fallback to a "previous" build after something went wrong with an update.

I'll write something up and post a draft here, then you can comment on it. ASX, maybe I'll mail you some questions if I need details or am not sure about something.

[ASX]

Yes, I can think of one are that would need help, it is in writing the upcoming documentation about using our new pkgs repos, the reasons behind our choices and the need to document a path for a successful transition from using the freebsd repos to use our own repos.

kraileth, would you pick up that task ?

Sure thing! I'll set some time aside for that. How much time would you (roughly) estimate does it take before the repos can be made available for the general public?

adding a few tip here:

- why our own packages (binary need to be synced between ports and packages, including our own ports)
- why synth (and thanks to J.R. Marino
- repos layout (and reasons for the 4 repos for each arch)
- changes required to use the new repos (from 11.0 only)
- how to fallback to a "previous" build after something went wrong with an update.

I'll write something up and post a draft here, then you can comment on it. ASX, maybe I'll mail you some questions if I need details or am not sure about something.

Thanks!

Please keep posting here, the info will be useful to Eric too or anyone who would join us later.
Of course, I will be available for any question.

[kraileth]

I thought about this for a moment and came to the conclusion that we could either write one big wall of text that only hardcore fans would read or break it up into a series of smaller posts. Obviously I like the latter option better. Here's my first take on it:

Upcoming GhostBSD 11.0: Revelations #1

Dear users and friends of GhostBSD,

as everybody keeping an eye on the project will have noticed, the release of version 11.0 has been delayed for a considerable amount of time. Think there's not much going on right now? On the contrary! The team is preparing the most sophisticated release yet for you! We're really busy working on tasks that require a lot time but are mostly "invisible" to non-developers.

It's not that we're being secretive about what we do. We've made statements about what you can expect from 11.0 in various forum posts. However we figured it would be a good idea to sum it all up and make a small series of official announcements to keep you informed.

Today I'll give you a first bit of information about the biggie that has been holding everything back: For the upcoming version 11.0 we're switching away from simply using FreeBSD's packages to providing our own package repository - or rather: repositories!

Technical details will follow in further posts. First let's talk about the motivation for taking this step. So why have we decided to roll our own packages instead of continuing to use FreeBSD's?

1) GhostBSD is a desktop variant (or "distribution" if you will) of FreeBSD. Vanilla FreeBSD is a versatile general purpose OS, however with a clear focus on server usage. The ports infrastructure of FreeBSD allows for great flexibility in building your system. However ports maintainers define default options for each port and FreeBSD's packages are built with these options. In several cases these are geared more towards server usage and the choice is not ideal for a project like GhostBSD which is all about the desktop. Building packages ourselves puts us in control and gives us the freedom to make changes to the port options and optimize things for desktop usage!

2) To make GhostBSD a user-friendly FreeBSD desktop we make use of several applications that are not part of regular FreeBSD (e.g. the update station). In the past those were built once for every release and put on the installation media. Once installed, they could not easily receive updates since they are not available on the FreeBSD package repos. We could have tried to add them to the FreeBSD ports tree. This has happened e.g. for the "network manager" application because it was deemed that it would be useful for users of vanilla FreeBSD, too. Other ports are probably tied too closely to GhostBSD to have a chance of being accepted. Having our own repositories solves this long standing problem: Even GhostBSD specific applications can be updated using the regular update mechanism beginning with version 11.0!

3) The first thing that comes to mind is taking FreeBSD's packages, add our own applications and put those togehter in one new repository. This is impossible for several reasons, the most important one being that FreeBSD does not allow mirroring their package repositories (for bandwidth reasons. With this restriction alone that way is blocked and does not need to be discussed further.

4) In theory it would be possible to provide an additional repository just for our applications and keep using the original FreeBSD packages. This is a bad idea, however, mainly for one reason: Ports that packages are built from need to be in sync. If they are not, there's always the possibility of breaking things! Since we cannot influence how and when FreeBSD build their packages, keeping things in sync would be a task that we'd have to constantly see to. While this is not technically impossible it's absolutely not feasible to even attempt.

So for a couple of reasons we have decided to cut the cord and become independent from FreeBSD when it comes to packages. Mind you, this does not mean we will fork the ports tree or anything like that! This is completely out of question. So we're not moving away from upstream ports or FreeBSD in general. However we're taking an important step with our own repositories to gain much more control and to be able to provide you with better packages that focus on desktop usage. Some of the cool things this will allow us to do will be revealed together with technical details in the next posts of this mini series. Stay tuned for more information - and of course a great new release 11.0 once things are ready for prime time!

Any thoughts on the "format" of several posts or on the contents? ASX: This is meant to primarily cover the first point of the list you posted. Am I missing anything important in this regard?

I took the freedom of shuffeling things around a bit and would like to write the next post about the "repos layout (and reasons for the 4 repos for each arch)" topic with "why Synth" third. Would that be ok?

[ASX]

Any thoughts on the "format" of several posts or on the contents?

yes, please see below.

ASX: This is meant to primarily cover the first point of the list you posted.

you have already exceed the amount of allowed words.
I'm starting to think you don't want to use a mobile phone because SMS are 140 chars constrained.

Am I missing anything important in this regard?

Yeah, we are not a books editor.

I took the freedom of shuffeling things around a bit and would like to write the next post about the "repos layout (and reasons for the 4 repos for each arch)" topic with "why Synth" third. Would that be ok?

yes and no.

a) we need a web page mostly announcing the new repos, with "some" (only some) hint about the reasons behind that choice.
b) that should be done preferably in the style of a blog entry, relatively short, limited to the facts, possibly avoiding autocelebration like "we spent a lot of time" ...
c) there a place to eventually discuss about details, if requested, and that is the forum, a discussion may follow a forum announce, but the announce itself should be "synthetic".

kraileth, I know you can do much better.

[kraileth]

ASX: This is meant to primarily cover the first point of the list you posted.

you have already exceed the amount of allowed words.
I'm starting to think you don't want to use a mobile phone because SMS are 140 chars constrained.

And why I'm not on Twitter.

Am I missing anything important in this regard?

Yeah, we are not a books editor.

Dang, wrong address again...

a) we need a web page mostly announcing the new repos, with "some" (only some) hint about the reasons behind that choice.
b) that should be done preferably in the style of a blog entry, relatively short, limited to the facts, possibly avoiding autocelebration like "we spent a lot of time" ...
c) there a place to eventually discuss about details, if requested, and that is the forum, a discussion may follow a forum announce, but the announce itself should be "synthetic".

Ok, here's another version covering points 1 to 3 in a brief, unspirited way:

Upcoming GhostBSD 11.0: New pkg repositories

The next major version of GhostBSD will feature our own package repositories instead of continuing to use those from FreeBSD.

There are several reasons for this:

1) FreeBSD is geared more towards servers while GhostBSD targets the desktop. Maintaining our own repositories gives us more control and allows us to set buildtime options differently.

2) It allows us to update GhostBSD specific applications (e.g. the update station) via pkg, too.

3) Creating an additional repository would lead to technical problems and quite possibly break applications.

A separate bild server was rented and will be building updated packages weekly. We found using Synth to build ports fits us better than Poudriere (thanks to J.R. Marino for that nice program!).

GhostBSD will provide packages for i386 and amd64. For both architectures there will be 3 public repositories:

"latest", which matches FreeBSD's ports + GhostBSD ports + changed options and perhaps patches. The devs will run this and advanced users who are willing to test are welcome, too.
"current", which will be the default repository for GhostBSD.
"previous", which keeps the previous packages available after "current" was updated.

By using "latest" we hope to catch problems before broken packages go into "current". If something we don't use breaks, there's "previous" as a fallback repo.

Is that more like it?

[ASX]

Is that more like it?

Indeed, it is, looks great to me!

You could also expand a bit ... no I'm not joking. Well only some bits

Reasons why two separate repos could break things, is because packages from different repos my rely on different version of the same dependency package.

Worth also to explain that we switched to "latest", after noticing that "quarterly" has demonstrated to not be as stable and reliable as expected. (specifically breaking X for some users).

Additionally, we can mention that we will provide the matching source tree along with the repos, so that people that want/like to customize their own settings will be allowed to do it the proper way.

Also wait for ericbsd comment.

[kraileth]

Also wait for ericbsd comment.

I definitely won't publish it before Eric gave feedback, no worries.

Here's a slightly ( ) extended version, adding the things you mentioned:

Upcoming GhostBSD 11.0: New pkg repositories

The next major version of GhostBSD will feature our own package repositories instead of continuing to use those from FreeBSD.

There are several reasons for this:

1) FreeBSD is geared more towards servers while GhostBSD targets the desktop. Maintaining our own repositories gives us more control and allows us to set buildtime options differently.

2) It allows us to update GhostBSD specific applications (e.g. the update station) via pkg, too.

3) Creating an additional repository would lead to technical problems and quite possibly break applications (packages from two different repositories could depend on different versions of the same dependency package).

A separate bild server was rented and will be building updated packages weekly. We found using Synth to build ports fits us better than Poudriere (thanks to J.R. Marino for that nice program!).

Initially the plan was to go with the "quaterly" ports branch from FreeBSD. Unfortunately the last quaterly update broke X11 for a lot of users and thus proved not to meet our requirements at all.

GhostBSD will provide packages for i386 and amd64. For both architectures there will be 3 public repositories:

"latest", which matches FreeBSD's ports + GhostBSD ports + changed options and perhaps patches. The devs will run this and advanced users who are willing to test are welcome, too.
"current", which will be the default repository for GhostBSD.
"previous", which keeps the previous packages available after "current" was updated.

By using "latest" we hope to catch problems before broken packages go into "current". If something we don't use breaks, there's "previous" as a fallback repo.

For users who like packages but need to build some ports themselves to change options, we will provide the patched ports tree that we build packages from. It will be made able it via git.