X server issues and ports

[kraileth]

What about something like this https://www.soyoustart.com/us/offers/e5-sat-2-32.xml?

Looks like a really nice system, missing just the SSDs! Well, to be honest, our usual prices are much higher than that, but our whole business isn't exactly targeted at being cheap. However my boss has already decided that a FreeBSD project can be sponsored (so that we get the free virtual server); perhaps I can arrange something where we pay a way lower than usual price for some real hardware. The big advantage here would be that we aren't tied to some pre-defined option but could actually get pretty much what we need. If we can get a server for a reasonable monthly price we can also save the setup cost as I can simply assemble the server and install fbsd after work in my free time.

So in general we're talking about something like this?

• Quadcore or Hexcore with a decent to somewhat high clock rate
• A lot of RAM, the more the better: 32 GB+
• Sufficient storage space on HDD: 2x2 TB
• Fast (SSD) disk space for swap, ccache, /usr/ports and OS: 2x300 GB
• Sophisticated disk setup: 2 TB ZFS mirror for the actual packages, 25 (?) GB ZFS mirror on SSD for the OS, 15 GB SWAP on each SSD, rest of SSD space allocated as two UFS2 partitions (or rather one ZFS stripe??)
• 1 IP addr (a buildserver shouldn't need more, right?)
• Root server (we're better off to do things ourselves here)
• No backup needed (everything except for some configuration files should be simple to replace if the system needs to be rebuilt for whatever reason)

[ericbsd]

What about something like this https://www.soyoustart.com/us/offers/e5-sat-2-32.xml?

Looks like a really nice system, missing just the SSDs! Well, to be honest, our usual prices are much higher than that, but our whole business isn't exactly targeted at being cheap. However my boss has already decided that a FreeBSD project can be sponsored (so that we get the free virtual server); perhaps I can arrange something where we pay a way lower than usual price for some real hardware. The big advantage here would be that we aren't tied to some pre-defined option but could actually get pretty much what we need. If we can get a server for a reasonable monthly price we can also save the setup cost as I can simply assemble the server and install fbsd after work in my free time.

So in general we're talking about something like this?

• Quadcore or Hexcore with a decent to somewhat high clock rate
• A lot of RAM, the more the better: 32 GB+
• Sufficient storage space on HDD: 2x2 TB
• Fast (SSD) disk space for swap, ccache, /usr/ports and OS: 2x300 GB
• Sophisticated disk setup: 2 TB ZFS mirror for the actual packages, 25 (?) GB ZFS mirror on SSD for the OS, 15 GB SWAP on each SSD, rest of SSD space allocated as two UFS2 partitions (or rather one ZFS stripe??)
• 1 IP addr (a buildserver shouldn't need more, right?)
• Root server (we're better off to do things ourselves here)
• No backup needed (everything except for some configuration files should be simple to replace if the system needs to be rebuilt for whatever reason)

I can't argue against that, if you can get that I think that would be our best option.

[ASX]

karileith, that config looks really good.

we will need a bir more swap, actually @marino suggest 4x the ram size, but that IMO is referred at dragonflybsd (where swap is faster, and SSD are also faster on dragonfly).

Only the OS need mirroring, all the rest can be rebuild and the only things to backup will be:
ports list
ports options
and a few config settings

anyway, we can live also with something less ... and I like a lot the idea of a machine where you can have physicalk access ... chance to expand ... etc.

[ASX]

A little update about storage requirements:

The last night (and day) I have build 1000 packages, both for amd64 and i386. (I have used the list provided from NevilleGoddard in the pkg requests thread, so that's a real thing.

The total storage used is around 14 GB, which will translate to 14 * 27000 / 1000 ~= 378 GB (both arch, including ports and distfiles).

That considerably less than the previous estimate, even doubling it for test and release repo fall below 1 TB.

That's to say that our buillder could be configured with 2x300 GB SSD + 1x 2 TB disk.

~~~

The builds didn't took advantage of ccache and the average throughput was around 120 pkgs/hours. :/
chromium alone is taking more than 4 hours to build and still running. libreoffice was built in 2.5 hours.

[ericbsd]

Was it on the server or your desktop?

[ASX]

Was it on the server or your desktop?

on the server, it is all under /build (as zfs dataset)
.

[ericbsd]

kraileth What's any information on tje servers for building packages?

Also I would like to know how it will that I can't starts to plan for that.

[kraileth]

kraileth What's any information on tje servers for building packages?

Also I would like to know how it will that I can't starts to plan for that.

Not really. I intended to talk with my boss about it this week however both of my children are currently ill which is why I had to ask to work from home instead. And the build server is a topic that I'd rather discuss in person than over phone as I'm sure that this would increase he chances to come to a positive agreement (since we do not target the mass market but focus on professional support instead, our usual prices are not exactly cheap; we do not offer ANY physical servers in the price range of what you linked to and thus I really have to explain things in person).

Do you know what their cancelation period is? If it's rather short and the server is needed now, you should perhaps rent it. We could always migrate over if I can make an agreement and stick with the other server if I can't.

[ericbsd]

It is month to month base contract. Basically pay as you go.

For the rest I will wait on ASX output.

[ASX]

It is month to month base contract. Basically pay as you go.

For the rest I will wait on ASX output.

Not sure what you are referring to, however I will try to recap the situation:
- tried a build "everything", but the build stopped at approx 2700 pkgs, due to a bug in synth that was fixed in latest version; so far those 2700 amd64 pkgs, + 1000 i386 pkgs + ports + distfiles are using 18 GB.
- I will not update synth on the server before we have a verified backup.
- as you can see from my latest posts, guessing the total required amount of storage is just that: guessing ...
- I cannot push the current server to the maximum load because the same server is running our forum, website and tracker, so far I used 3 builders x 2 Jobx and the system remain fully responsive, on average there remain 10 GB of free RAM (out of 32), and with that setup the build speed is low ... ~ 200 pkgs per hour, to low to build everything.
Either we step back to a partial repo (10000 pkgs) or we need a faster server.

- about a possible new server I asked if we can have both a traditional disk and an SSD, because that is in my opinion our best option, but is still unaswered.

- From my experience, a good cpu is good, but we need a disk subsystem that can support the high I/O load as required from synth; I expect that ccache may work better on UFS non journaled than on ZFS like now, but really have not tested it.

Build server as designed in my mind:
- ports, and distfiles may stay on traditional hard disk (synth will unpack those files in RAM+SWAP)
- OS must be on SSD (32 GB, mirrored)
- swap must be on SSD (32 + 32 GB)
- ccache must be on SSD (UFS 32 + 32 GB)
- repo would be better on SSD, but we can fallback on traditional hard disk (Synth will install all required pkgs in RAM + SWAP, to build the local base, that in some rare case is as large as 8 GB per builder).
- all the other things will be on traditional hard disk.

Note that the build process, for each package, will install the local base, it mean it will unpack lot of dependencies and dependencies libs reading from hd and writing to RAM; that's why it would be optimal using a SSD to store the pkgs. (test-repository).

Instead, when we are fine with test-repository, we can move it to the traditional disk as release-repository, to be served from apache or something.

Details remain to be refined, but I have in my mind a mixed setup ... UFS + ZFS ... SSD + spiining disks ...