Create a development certificate authority

[matze]

When working with the infrastructure repository, especially when accessing a box via HTTPS, one currently needs to manually create exceptions for the self-signed certificates in modules/private-stub. This is not only annoying and - over extend of time - a bit time-consuming, it also requires test-code being specifically aligned. Especially the latter is worth avoiding, in order to ensure tested and productive code (and thus behavior) is as similar as possible.

Fortunately, improvement is quite simple here: We can establish our own, local CA for development and testing purpose. Subsequently, we can replace the current certificates in the private-stub module with new ones signed by that instance. Developers and testers would then just need to register the CA in their own system, so that these certs would be accepted just as if it where an official one.

There's just one caveat to consider: The CA's private bits themselves should be maintained by a trusted body. Otherwise we could be "compromised from the inside".

[trev]

There's just one caveat to consider: The CA's private bits themselves should be maintained by a trusted body. Otherwise we could be "compromised from the inside".

That's exactly the issue I see with this approach - we need to keep the CA's private key secure, otherwise we will create quite a security issue for ourselves here. Personally, I don't have an issue with adding a permanent certificate exception for test servers, that's two seconds I need to spend once for each IP and is perfectly secure. Is there really an issue here worth the headache that this MITM CA certificate will create?

[kzar]

Well for what it's worth I found development to be a pain for the reasons matze said. If we set up our own CA we could mention it in the new developer starter's guide and they'd be set up straight away. (I appreciate it would be extra work maintaining the keys though.)

[trev]

Replying to kzar:

Well for what it's worth I found development to be a pain for the reasons matze said.

That's not an answer to my question. Did you add a persistent exception for the testing certificate? And if you did, why was it still a pain?

[fhd]

I don't really see the problem here - what's wrong with accepting the unsafe certificate once (!) in the browser and using wget --no-check-certificate/curl -k?

[kzar]

@fhd Chrome no longer gives you the option to accept the unsafe certificate once :(

[matze]

@trev
I'd like to know why you consider this a "MITM CA certificate". And why it may cause "headache". Is it really that hard to keep a private key secure? (Note that the “trused body” I’ve mentioned was actually a synonym for “Wladimir and Felix” - according to our current structure.)

@fhd
Nothing is wrong with the exceptional approach, if you know what you are doing and only have to use it occasionally. Unfortunately, most users, testers and developers do not know (though the latter often believe they do). In addition, especially when developing in infrastructure and testing components for Eyeo, it's also not something that occurs occasionally - but a bunch of times each day. However, I won't start a theoretical discussion about the definition of exceptions here (nor a sociological one about people who get used to circumvent rules and start making mistakes).

@all
In addition to the Chrome situation, one must also consider additional issues when developing with self-signed certificates:

Any additional step to circumvent certificate checks within software and infrastructure tests is an unnecessary derivation from the production environment. Especially in infrastructure, this can be prone to errors and to not discovering issues before deploy or - even worse - exploit. Though the rather is pretty rare and a lot of steps have to fail before it comes that far - it happens.

Please also consider the scenario where one’s used to accepting the self-signed certificate for a test role within our infrastructure repo and the Vagrant boxes. How often does anyone check whether the browser actually prompts for a self-signed key - and not some other issue with the chiffres? Most likely, any of the possible issues will not be discovered until it reaches production.

In addition, there’s plenty of possible scenarios where the requirement for an exception is not that obvious. Consider ​https://reports.adblockplus.org/, for example, where the ABP extension queries the URL, not the user. Without being fully aware of the situation, a developer can spend hours to figure out why that one won’t work in development - until white-listed explicitly. (Actually, I’m not even sure that this works with the user exceptions.)

For myself I consider development with self-signed certificates being unprofessional, due to the aforementioned reasons and a bunch of others (which do not apply to ABP/Eyeo so far).

Cheers!
Matze

PS: I've worked with a bunch of global players where employees (including testers) weren't even allowed to make (e.g. browser-) exceptions, and developers weren't allowed to produce code circumventing certificate validation. Well, back then, our own company was pretty embarrassed being the only party providing stubs - and never made the same mistake again.

[fhd]

OK, now I got it - if you can't accept an exception permanently in Chrome anymore, that's a pain indeed.

The security problem seems quite manageable, so I'm all for it.

[trev]

@fhd: I guess that adding the private key for this CA certificate to our shared passwords would be ok. That way we'll have an encrypted backup copy. And everybody can decide for themselves whether they want to add the CA or rather work with exceptions instead.