打造Linux下的高性能网络.pdf

打造Linux下的高性能网络 北京酷锐达信息技术有限公司 技术总监 史应生 BY DEFAULT, LINUX NETWORKING NOT TUNED FOR MAX PERFORMANCE, MORE FOR RELIABILITY Trade-off :Low Latency, throughput, determinism Performance Goals  Throughput – Optimize for best average – Default design criteria for most operating systems – how much can you do at a time?“  Low Latency – Optimize for best minimum – Minimize execution times for certain paths – whats the fastest we can push a packet out?“  Determinism – Optimize for best (lowest) maximum – Fewest/lowest outliers – whats the maximum time it will take? State of the Art NIC characteristics  56 Gigabits per second (Unix network stack was designed for 10Mbits)  4-7 Gigabytes per second (Unix: 1 MB/s)  8 million packets per second (Unix: ~1000 packets per second).  Less than a microsecond per packet for processing. Latency Factors BIOS Settings for Low Latency CSTATE default – C7 on this config CSTATE disabled – Note speed NPtcp latency vs cstates – c7 vs c0 Firmware tuning impact – a drastic picture Different Technology To Max Performance  IPOIB = Kernel Sockets layer using IP emulation on Infiniband.  SDP = Kernel Sockets layer using Infiniband native connection.  IB = Native Infiniband connection. User space → User Space  Rsockets = Socket Emulation layer in user space  Performance comparison shows that kernel processing is detrimental to performance. Bypass is essential. Different Technology To Max Performance Why bypass the kernel?  Kernel is too slow and inefficient at high packet rates. Problems already begin at 10G.  Contemporary devices can map user space memory and perform transfer to user space.  Kernel must copy data between kernel buffers and userspace.  Kernel is co