WIP: build on illumos.

[tobhe]

OpenIKED on illumos

This PR tracks the progress on getting OpenIKED to run on illumos. With this initial patch it is possible
to compile and run iked. The pfkey kernel interface does not work yet so iked will not be able to store
negotiated SAs and policies in the kernel database.

Changes

• Remove u_intXX fixed types, use stdint.h types instead
• rename sun to saun to avoid name conflict
• Add illumos specific libraries
• Fix missing includes
• Disable SO_REUSEPORT and dirent.d_type on illumos
• fix missing -o flag in yacc
• replace pfkey backend with empty dummy for now
• Provide WAIT_ANY replacement

Setup

A few manual steps are required to get iked running:

# make install

Add service group + user for privsep

# groupadd iked
# useradd -d /var/empty -s /bin/false -c "IKEv2 Daemon" -g iked iked

Generate keys

# openssl ecparam -genkey -name prime256v1 -noout -out "/etc/iked/private/local.key"
# openssl ec -in "$DIR/private/local.key" -pubout -out "/etc/iked/local.pub"

With this, the user land parts work with a few minor error messages being logged on startup:

root@openindiana:~# iked -d
ca_reload: failed to load ca file .: Is a directory
ca_sslerror: ca_reload: error:0909006C:PEM routines:get_name:no start line
ca_sslerror: ca_reload: error:0B06F009:x509 certificate routines:X509_load_cert_file:PEM lib
ca_reload: failed to load ca file ..: Is a directory
ca_sslerror: ca_reload: error:0909006C:PEM routines:get_name:no start line
ca_sslerror: ca_reload: error:0B06F009:x509 certificate routines:X509_load_cert_file:PEM lib
ikev2_init_ike_sa: initiating "policy1"
spi=0x74bf1d367d2af4f3: send IKE_SA_INIT req 0 peer 192.168.122.105:500 local 192.168.122.220:500, 518 bytes
spi=0x74bf1d367d2af4f3: recv IKE_SA_INIT res 0 peer 192.168.122.105:500 local 192.168.122.220:500, 214 bytes, policy 'policy1'
spi=0x74bf1d367d2af4f3: send IKE_AUTH req 1 peer 192.168.122.105:500 local 192.168.122.220:500, 296 bytes
spi=0x74bf1d367d2af4f3: recv IKE_AUTH res 1 peer 192.168.122.105:500 local 192.168.122.220:500, 193 bytes, policy 'policy1'
spi=0x74bf1d367d2af4f3: ikev2_childsa_enable: loaded SPIs: 0x4aa1eda6, 0x00000000 (enc aes-128-gcm)
spi=0x74bf1d367d2af4f3: ikev2_childsa_enable: loaded flows: ESP-192.168.122.220/32=192.168.122.105/32(0)
spi=0x74bf1d367d2af4f3: established peer 192.168.122.105:500[FQDN/obsd] local 192.168.122.220:500[FQDN/illumos] policy 'policy1' as initiator (enc aes-128-gcm group curve25519 prf hmac-sha2-256)

The tunnels are not actually usable because iked currently has no way of storing them in the kernel.

[tobhe]

Uploaded a new, much smaller, rebased diff. I committed a lot of the smaller clean-up changes such as removing u_intXX and including <endian.h> directly in OpenBSD and cherry-picked them to portable. What is still missing for illumos is a pfkey backend, a solution to make our socket bypass IPsec (probably via sockopt) and possibly a SO_REUSEPORT equivalent though we might be able to just ignore that.