ASX wrote:Honestly I'm not a lover of "wrapper tools" in general, I guess I would prefer to use the plain jails commands instead of ezjail, iocage and similar, however I'm not an expert on the matter.
Understand that. They offer convenience by hiding complexity but you're of course trading control for that. That's nice on the command line if you only want to quickly achieve simple things. For automation purposes I'd also prefer to remove unneeded "additional layers". The problem is that I know too little about jail management (I'm looking forward to the jails book that M.W. Lucas wanted to write sometime in the future!) and therefore using a wrapper is currently my only choice due to the lack of time to really dig into it. Yeah, I really should because a somewhat deep understanding never hurts - but there are simply so many other topics that need some love as well that I'm willing to make a compromise here...
It was my understanding that the iocage project is dead, instead its fork iocell is alivem or at least so I read on fbsd forum.
The old iocage (now refered to as "iocage legacy") is dead, yes. It was meant to be rewritten in go and because some people liked the shell approach, iocell was forked from it. The go port didn't happen, instead Kris Moore asked the author to go for Python instead because they want to use it over at iX for FreeNAS/TrueNAS to replace the old warden utility (and their product already comes with Python). So the new Python-based iocage is very much alive and they have some nice ideas. In the latest BSDNow show (before stepping down as co-host for time reasons) announced that they aim to go the "docker path" when it comes to simplicity. And I must admit that it would really help people who are not knowledgeable in networking and stuff to get their jails up and running if iocage manages the NAT part for you. That would be a nice, user-friendly feature. And once you understand the magic, you can always do without the tool.
The other thing I don't like about this "do it me all" tools, is that they are fooled very easily: what if a 00_Synth.conf files already existed ion the target machine ?
What if on the same machine, because (synth) the FreeBSD was previously disabled ?
That's not a problem per se. You have to plan for that. Salt offers a lot of functionality regarding files. Let me explain a bit of what this is all about (I wanted to do that anyway and after blocking some idiot I'm currently waiting for the load on a server to drop in the third value far enough to make the monitoring system happy again):
SaltStack works with .sls files written in YAML. They contain so-called
states which define the desired state of some kind of resource on a system. One main benefit of this declarative approach over, say, some shell script which could do much of the same, is that you only tell Salt
what to do but not
how. This is extremely helpful in a heterogeneous environment when you can just say "ensure that htop is installed" and the tool knows to use pkg on FreeBSD, apt-get on Debian, emerge on Gentoo, ... This is not so important in our FreeBSD-only environment. However using some kind of configuration management allows you to centrally manage servers and do so even at a large scale. We'll soon have three machines. Certainly not too much to manage by hand. But let's assume we get two more or so - this will get tiresome quite fast. Even if you only use Salt to deploy a baseline configuration and choose to do everything else by hand, that's potentially a huge relief!
But what I've come to appreciate most is that this helps to keep servers in a consistent state. Something's not right? Run Salt. Now the system should behave correctly again (if the problem was due to something that a state was previously written for). And even if a system completely dies (say a VM with just one virtual drive). Not a problem: Clone a machine template, start up the fresh system, give it the correct hostname - and run Salt. Done. The resulting machine exactly matches the previous one and can assume the same role.
One requirement for that: DON'T do manual editing of the config files if you want to keep those changes! You can edit things manually ad-hoc or to debug things. But once you settle on a new configuration that you want to keep, you're expected to tell Salt (i.e. update the corresponding states). They will be kept in a Git repo (Salt can directly read them from there) which allows for one central point to store configuration that we can all pull from and push to. And it also automatically adds the benefit of version control for configuration management. Something's not right now? Take a look at the changelog and revert to a known good state. It's as easy as that! No more "###, that used to work yesterday before I changed something that I just can't remember the old value of!".
States consist of several things, the first one being an ID. The only requirement here is that it is unique which means that Salt is happy with using something like "aa", "ab", "ac", ... However it totally makes sense to use that ID to describe what the state is supposed to be. That way everybody who speaks English quickly gets an idea even with 0 Salt knowledge.
The next line (how I use it, it can be done differently) is telling Salt what the state is actually about. Does the desired state mean a file may need to be altered? A package installed? A command run? The important thing here is
idempotence: A routine is idempotent when it it guaranteed to produce the same result no matter how often it is run. This is a hard requirement for Salt being able to be run again and again without messing up things that are already in the desired state. If you are making use of pre-defined internal states, this is nothing to worry about as it is being taken care of for you. Example: "Salt, please make sure that user asx is present on the system and can login via SSH". Salt knows how to do that. But it also recognizes if that's already the case and won't do anything in that case. A script on the other hand needs checks written for everything or, if it's a dump quick script, tries to add an existing user again, etc. When using Salt to run commands, you have to take care of the idempotence yourself, though. Fortunately Salt offers some friendly functions which make this easier.
In case of files, you'd use a state provided by "file". Which state to use depends on what you want to do. If I want to configure a system to allow users to use passwordless sudo, I can use
file.uncomment to have Salt remove the comment sign before the "# %wheel ALL=(ALL) NOPASSWD: ALL" line that I know is in sudoers by default (I just have to write other states to depend on that ensure the user is present and in the wheel group and that sudo is installed). If I want to add anything to /etc/rc.conf, I'd probably use
file.append. Salt is smart enough to only act if the desired line(s) is/are not already in the file. Often you'll probably use
file.managed which allows for several things; you could define the file contents (which only makes sense for short files) or simply point Salt to an actual file that you want to be placed on the system. Salt has an internal file server for that purpose and the files can also reside in a git repo, thus be versioned as well. You also have a lot of choices: "Salt, please put that file there - but only if a file by that name is not already there!", "Salt, please compare that hash first to see if the file is already in the desired state", "Salt, please make sure that the file is owned by root and has permissions 0644", etc.
So about your examples: The state for Synth's configuration
could be something like "Salt, this is the base configuration for Synth. Please put it there if it doesn't exist. If there already is a file of that name, leave it alone!". This is possible if Salt is to be used for one-time deployment only (a perfectly legit use and probably the right solution for your second example). However the better way (for the first case) is (IMO) to actually never modify the file on the server directly (or only do so temporarily for tuning purposes and such) but instead make a change on your local workstation's git clone of the configuration repo, commit, push and then run Salt. This is the cleanest way of doing administration. Every change is documented (by git). If something goes wrong, you can always go back to a known good state. If somebody who usually does something is not available (at work, on vacation, left the project , ...), somebody else can at least take a look at what has been done in this regard. This can also be very valuable.
Beside that, yesterday I started to look into saltstack tutorials .... we will see.
SaltStack's
Getting started is actually a pretty worthwhile read. Note however that Salt can potentially do
a ### of a lot more than we'd probably ever need. I basically want to use it for configuration management. It can also do orchestration, agentless runs over SSH, offers the "event reactor" (which I did not even bother to take a closer look at, yet) and can even be used for sophisticated monitoring and things like that.
Well, the monitoring aspect
could in fact be interesting for us some time in the future. Knowing when your services are down is always a good thing after all!
And since Salt knows what state the various system resources should be in, it can also be used to monitor those. In fact using the reactor I think some kind of "self healing" could even be implemented ("Hey, that machine's web jail is not running but my state says it should! Better try to start it up again!").
This whole topic is
huge. However while I want to concentrate on a few rather simple things, I also wanted to let you know how much potential this thing actually has. And the basics are fortunately rather simple. I hope that I'll be able to convince you of the benefits of using a system like Salt. I won't give up easily, but if I really can't, I'll propose Puppet/Chef/Ansible!
While I don't like especially Puppet much, it's still just so much better of doing stuff manually like in the 90's... So you've been warned. You better like Salt.
). I agree that this might as well be public and everybody is invited to participate in the discussion. Feel free to ask questions, too!
No, seriously, I used to write fantasy stuff but I haven't found the time to do so for about 2 years now. However I like to explain things and to argue about them now and then. 