Improper Certificate Validation in hMailServer

Summary

Progressive Robot hMailServer 6.0.0-B1 through 6.3.7 on Windows hands OpenSSL the names of its own TLS files - the listeners' certificates, chains, private keys and Diffie-Hellman parameters, a port's client-certificate CA file (from 6.2.19), the object store's CA file (from 6.3.4) and the ACME client's keys and certificates - as Latin-1 bytes, which on code page 1252 are the code page's own for U+00A0 to U+00FF, and OpenSSL's file opening on Windows reads a name as UTF-8 first and opens the file at that reading when it is there, or creates it there when its folder is. When every byte of a configured name outside ASCII also reads as UTF-8 - for example U+00C3 U+00A9, the bytes C3 A9, which are U+00E9 in UTF-8 - the server used the file at that other name whenever one was there, which allows a local attacker who can create that file or its folder but cannot change the configured one, as any user can by default beside a file an administrator keeps under ProgramData, to have a port that logs on by client certificate (from 6.2.25) accept certificates from a certificate authority of the attacker's own and so sign in over the network as any mailbox without its password; to have the server trust such an authority for its object store and, intercepting that connection, read and modify the messages it stores; to receive the private keys the ACME client writes, the key of a certificate issued for the server's names among them; or to keep TLS ports from starting. The configuration is not the default: by default these files have names in ASCII, client certificates are off on every port, and the object store and ACME are off. The Linux builds are not affected.