Linux system security (or not)

There is a splendid set of Bash scripts at https://github.com/ovh/debian-cis for both assessing and, if you wish, enhancing, the security of your Linux system.

The scripts are specific to Debian-based systems (so, it works for things like Ubuntu and Devuan as well), but it's based on generic Linux security recommendations for PCI-DSS compliance.

It's been developed by the hosting provider OVH since 2016 (for Debian 7 "Wheezy") and as their github page says, they use it to harden their own PCI-DSS infrastructure (which, in case you didn't follow the Wikipedia link above, means systems which deal with credit card payments). It's currently apparently "stable" for Debian 11 "Bullseye" and "in development" for Debian 12 "Bookworm" (which is the current stable release of Debian).

The scripts offer 5 levels of security assessment or hardening (the difference being that "assessment" just means checking lots of stuff on your machine and telling you which bits meet their standards and which bits don't, whereas "hardening" means applying changes to your machine so that it does meet the standards).

These levels are described in the documentation as:

1. very basic policy, failure to pass tests at this level indicates severe misconfiguration of the machine that can have a huge security impact
2. basic policy, some good practice rules that, once applied, shouldn't break anything on most systems
3. best practices policy, passing all tests might need some configuration modifications (such as specific partitioning, etc.)
4. high security policy, passing all tests might be time-consuming and require high adaptation of your workflow
5. placebo, policy rules that might be very difficult to apply and maintain, with questionable security benefits

Incidentally, the "hardening" option should be treated cautiously, because if you just let it loose on a machine (especially one which you're logged into over a network), you may well find that you can't log back in again after it's finished.

Anyway, it (currently) has a suite of 243 checks which it performs on your machine (the "security level" simply determines how many of these are checked or applied: see below) and at the end it gives you a Conformity Percentage telling you what percentage of checks which were performed at that security level were passed.

Now, in my most humble opinion, this is ridiculous.

Given that some of the checks are things like "are you allowing passwordless root logins by SSH" and some of them are "do your log files have the correct permissions" (and one is "does your GRUB configuration file include a password") I think there is a big difference between the significance of these checks, and assigning them all equal weight in arriving at the overall percentage is just astonishing.

Unfortunately, it gets worse.

Some checks are "do you have X installed on this machine" (where X may be something desirable for security, or it may be something undesirable) and other checks (for the situation where X is desirable) are "is X well-configured".

These also carry totally the same weights as each other when it comes to the final percentage, which many people are going to interpret as "how secure is my system?"

This means that if there are ten checks for application X, where one check is "is it installed" and the other nine are "does it have this things in its configuration", you can simply score nine out of ten by creating the configuration file with the appropriate settings, and not installing the application at all.

This then results in a noticeably increased Conformity Percentage with no actual change to the security of your machine in the slightest.

Admittedly, anyone who uses this script suite with the intention of increasing the security of a machine would make sure that X is installed as well as well-configured, but someone who simply wants to get a good score for a machine which is not in fact as "secure" as the result makes out, can abuse this aspect in multiple ways.

I find myself seriously questioning whether I would like to trust a credit card payments system with OVH, having seen the way this thing can be fooled.

The number of checks performed at the different security levels are:

• 27 (one of which is "is this a Debian machine", so if you think Devuan is preferable, and just as secure, you immediately lose 3.71%)
• 126
• 215
• 242
• 243 (the extra one is whether "CONFIG_VFAT_FS is enabled", which seems like a strange thing to me to make the difference between Security Level 4 out of 5 and Level 5 out of 5)

Now clearly, one person's definition of "secure" is not going to be the same as another's, and any assessment of security needs to question things like:

• is the machine connected to (and accessible from) a wide-area network (possibly the Internet)?
• are the users who have login access to the machine trusted?
• are the applications which are installed and running on the machine trusted?
• is the machine physically secured against people logging in on a console, or rebooting from their own USB stick?
• what is the value of the data stored on the machine (and what are the potential, possibly legal, consequences if it is compromised)?

The answers to these sorts of questions make (in my still very humble opinion) a big difference to whether you think network services such as SSH are more or less important than local users having access to stuff in /var/log, or whether an audit daemon or apparmor need to be installed, etc.

So, I think this is certainly a useful tool, but it has some significant shortcomings:

• checking whether a security application is well-configured should be entirely predicated on whether it is installed and running
• not every check for "security" is equally important
• not every check for "security" is equally important for different types of systems which are running in different environments
• in some cases, it doesn't even understand the concept of configuration files (which may contain insecure settings) being included from other configuration files (which look secure), or of configuration settings which look secure but are in fact commented-out and therefore don't apply

I think less than a day's work on this suite of scripts could make the results a lot more useful, meaningful and trustworthy.

In the meantime, if you ask someone to harden a system because it only scored 44% on its base installation, and they come back and tell you that it now scores 84% at Security Level 4 ("high security policy, passing all tests might be time-consuming and require high adaptation of your workflow"), you can be pretty sure that it's better than it was, but not necessarily as much better as it appears.

Go up
Return to main index.