TCP_INFO is a tcp option that a user could set in order to get access to some of the kernel tcp metrics counters. According to Linux documentation the TCP_INFO socket option is defined as follows:
TCP_INFO (since Linux 2.4) Used to collect information about this socket. The kernel returns a struct tcp_info as defined in the file /usr/include/linux/tcp.h. This option should not be used in code intended to be portable.
The tcp_info struct exposes some of kernel tcp metrics, which are helpful for users to monitor performance or connectivity problems in their network. These metrics are described in RFC-4898 , some systems like OpenBSD and FreeBSD implement their own extensions and Linux even today keeps adding fields to this struct.
TCP_INFO tcp option has been around for a while (around 2001-2011?), giving the amount of time that have past, applications are already been using this tcp option. So not having this option available means we need to patch applications that require it, to be able to run them in illumos. That's far from ideal but as most applications just assume only Linux exists, is easy to bump into this. That's reason enough to implement this option in illumos, and also in the process, our Linux emulation layer will benefit from it as we could make the option available to our LX branded zones. meaning our emulation will be more up to date.
There are two sides for this implementation that we need to consider. The OS implementation sure, it can be not ABI compatible with Linux, but lx brand must be. That means that there some extra work that should be done in the lx brand so programs could start using this option.
Other systems like OpenBSD and FreeBSD use the same structure for tcp_info as Linux but their implementations only setups some of the values, leaving the rest as 0s, and they also add their own extensions to the tcp_info struct.
I'll describe here every field of the tcp_info data structure as a way to compare both systems (illumos an Linux) side by side, to be sure that my undestanding about these fields is somewhat accurate. I'll keep the same Linux struct members names but the illumos tcp_info struct but will not be ABI compatible, as there are differences between Linux and illumos tcp counters and data structures uses uint8_t for tcp states, and illumos uses uint32_t, also constants for TCP states are mapped differently, and so on...
This field in Linux is defined as uint8_t, in illumos is called tcp_state, it represents the current state in the TCP state machine.
States in Linux could be the following:
Now in illumos we have the following tcp states:
Which are the mostly the same but internally their values are mapped differently
This field represents the current state of the congestion avoidance field. In Linux the states are defined in here
__u8 icsk_ca_state:5, icsk_ca_initialized:1, icsk_ca_setsockopt:1, icsk_ca_dst_locked:1;
In illumos this information is defined in tcp_ccv field of tcp_t, which has a member named flags which holds the current state
Number of unrecovered timeouts, this is covered in RFC6298
And differences keep piling on ...
For the moment I have the following patch that add said socket option to OS and LX branded zones, but the tcp states on the LX side are not correct and in the OS, side I'm not certain what each field means at this moment. In the meantime here is a WIP patch : 14744-Add-TCP_INFO-socket-option
This other patch has less impact as it only adds this socket option to LX branded zones OS-4525 patch
References