Guard the loop unroll and noinline definition of parse_hdr_opt behind…

… a flag

Summary:
There are two things that are only supported in kernel-5.6+:
1) passing of variable defined in stack (in this case opt_state to parse_hdr_opt())
2) bounded loop without the #pragma unroll (https://lore.kernel.org/netdev/[email protected]/)

The existing check meant to guard w/ the kernel version `#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 3, 0)`
at compile time isn't always effective since the bpf object can be compiled in host with different kernel version.

So, this diff adds a flag (TCP_HDR_OPT_SKIP_UNROLL_LOOP) for users to be able to toggle both #1 and #2.
'__always_inline__' addresses #1 since it makes the parse_hdr_opt() impl inline.
Therefore, the restriction on passing of the variable in stack memory isn't applicable.

Reviewed By: avasylev

Differential Revision: D32035828

fbshipit-source-id: 4b52652b761133359795414028777e331d54870f

katran/lib/bpf/pckt_parsing.h

@@ -135,7 +135,11 @@ struct hdr_opt_state {
 };
 
 #ifdef TCP_SERVER_ID_ROUTING
+#ifdef TCP_HDR_OPT_SKIP_UNROLL_LOOP
 __attribute__ ((noinline))
+#else
+__attribute__ ((__always_inline__))
+#endif
 int parse_hdr_opt(const struct xdp_md *xdp, struct hdr_opt_state *state)
 {
   const void *data = (void *)(long)xdp->data;
@@ -219,7 +223,8 @@ __attribute__((__always_inline__)) static inline int tcp_hdr_opt_lookup(
 
   opt_state.hdr_bytes_remaining = tcp_hdr_opt_len;
   opt_state.byte_offset = sizeof(struct tcphdr) + tcp_offset;
-#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 3, 0)
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 3, 0) || \
+    !defined TCP_HDR_OPT_SKIP_UNROLL_LOOP
   // For linux kernel version < 5.3, there isn't support in the bpf verifier
   // for validating bounded loops, so we need to unroll the loop
 #pragma clang loop unroll(full)