new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB
Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB
Performance looks reasonably good, but there is an unexpected high number of build failures, and as a consequence 6000 ports are skipped (not visible due to a ncurses/terminfo bug), but you can desume it from the difference between: total - built - left.
I'm going to stop this build to look at what is wrong.
I'm going to stop this build to look at what is wrong.
Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB
OK, the failre was due to a ccache flag (run_second_cpp = false), that accordingly to the man page should have only affected the diagnostic output, clearly it is not true.
I took the chance to show the ccache effect, now built for the first 1000 pkgs ...
I took the chance to show the ccache effect, now built for the first 1000 pkgs ...
Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB
Something is not right yet ...
Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB
better, yet not enough ...
Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB
This machine has been a delusion.
A bit of info, for your own knowledge:
The operating system was installed on a single SATA disk, using ZFS. That leave 2 nvme disk and 1 SATA disk free.I have setup a new pool, made up from the two NVME disks, in RAID 0/stripe.
The NVME disk transfer rate is approx. 1000 MB/s (as reported from diskinfo), the aggregate transfer rate of the striped pool is approx 1500 MB/s. Impressive!
But that doesn't translate to a builder performance improvement, (ok, there is an improvement, but can be measured like +3% or+ 4%). Why so ?
a) ZFS, (and the 64 GB RAM installed), do an astonishing job in caching lot of things, as a result, the disk is really used only to write the built packages, the logs, and eventually the ccache. Additionally the write operations are of course asyncronous, in that synth "request" the write, the kernel/ZFS take in charge the job, and return an ACK to synth, before the files are effectively stored to the disk. The kernel will do the effective write sometimes later, by itself, in the meantime synth will continue with other packages.
The same "write offload" happens with SATA disks too, that's why there is no real difference by using NVME vs SATA. Probably swapping onto an NVME disk would be much faster, but here we don't swap at all.
b) The high speed of the nvme disks come to a price, in term of CPU usage, true you can achieve impressive write speed, but the CPU usage (sys in this case), will be relatively high. By doing that, those CPU cycles are detracted from the build processes. This situation could be eventually improved by using "intelligent/smart/cached/hw controllers, but that would be difficult to manage remotely. (and very likely would come at a cost too).
long story short: for builders choose the better CPU you can afford.
Now, the good news: we (me and ericbsd) already did it.
SoYouStart renewed their offer, and we decided to grab two machines now that are available:
We have got two of these: https://www.soyoustart.com/ca/en/offers/173sys20.xml
These two machines (both) come at the same price of the E3-1670, but each of these perform better compared to the E3-1670, and we have two of them.
My personal thanks to the E3-1670, that I stressed a lot, but we will let it expire.
PS: ericbsd performed also a test using "poudriere" on this machine, the result was 440 pkg/hour.
I'm now doing a last round, just for comparison and will post the result here when completed.
A bit of info, for your own knowledge:
The operating system was installed on a single SATA disk, using ZFS. That leave 2 nvme disk and 1 SATA disk free.I have setup a new pool, made up from the two NVME disks, in RAID 0/stripe.
The NVME disk transfer rate is approx. 1000 MB/s (as reported from diskinfo), the aggregate transfer rate of the striped pool is approx 1500 MB/s. Impressive!
But that doesn't translate to a builder performance improvement, (ok, there is an improvement, but can be measured like +3% or+ 4%). Why so ?
a) ZFS, (and the 64 GB RAM installed), do an astonishing job in caching lot of things, as a result, the disk is really used only to write the built packages, the logs, and eventually the ccache. Additionally the write operations are of course asyncronous, in that synth "request" the write, the kernel/ZFS take in charge the job, and return an ACK to synth, before the files are effectively stored to the disk. The kernel will do the effective write sometimes later, by itself, in the meantime synth will continue with other packages.
The same "write offload" happens with SATA disks too, that's why there is no real difference by using NVME vs SATA. Probably swapping onto an NVME disk would be much faster, but here we don't swap at all.
b) The high speed of the nvme disks come to a price, in term of CPU usage, true you can achieve impressive write speed, but the CPU usage (sys in this case), will be relatively high. By doing that, those CPU cycles are detracted from the build processes. This situation could be eventually improved by using "intelligent/smart/cached/hw controllers, but that would be difficult to manage remotely. (and very likely would come at a cost too).
long story short: for builders choose the better CPU you can afford.
Now, the good news: we (me and ericbsd) already did it.
SoYouStart renewed their offer, and we decided to grab two machines now that are available:
We have got two of these: https://www.soyoustart.com/ca/en/offers/173sys20.xml
These two machines (both) come at the same price of the E3-1670, but each of these perform better compared to the E3-1670, and we have two of them.
My personal thanks to the E3-1670, that I stressed a lot,
but we will let it expire.PS: ericbsd performed also a test using "poudriere" on this machine, the result was 440 pkg/hour.
I'm now doing a last round, just for comparison and will post the result here when completed.
Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB
half repo .. and peaking at high speed
Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB
This build is nearly complete, the pkg rate is very similar to poudriere result, moreove I switched from a NVME based setup to a SATA setup. Once again a confirmation that the bottleneck is the CPU. 
Stopping and goint to build the updated tree.
Stopping and goint to build the updated tree.
Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB
This server is going to expire in one week or so.
@ericbsd, let me know what you think, but IMO this should be dismissed.
If you want to perform furtther tests it is at your complete disposal.
~~~
I need a "script " to sync github.com/ghostbsd/freebsd-ports with upstream freebsd-ports; it is about time to start a next build.
@ericbsd, let me know what you think, but IMO this should be dismissed.
If you want to perform furtther tests it is at your complete disposal.
~~~
I need a "script " to sync github.com/ghostbsd/freebsd-ports with upstream freebsd-ports; it is about time to start a next build.
Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB
Nah we should dismiss that server.ASX wrote:This server is going to expire in one week or so.
@ericbsd, let me know what you think, but IMO this should be dismissed.
If you want to perform furtther tests it is at your complete disposal.
I could maybe do simple shell script to sync the GitHub repository.I need a "script " to sync github.com/ghostbsd/freebsd-ports with upstream freebsd-ports; it is about time to start a next build.