背景
经常忘记使用ib_write_bw打流的一些参数，特此整理记录在这里方便快速查阅。尤其是run_infinitely这个参数容易写错。
最简洁
ib_write_bw -d mlx5_0 # server
ib_write_bw -d mlx5_0 1.1.1.1 # client
常用参数
非常常用
-d mlx5_0, --ib-dev= 指定ib设备，比如：-d mlx5_0表示用mlx5_0设备-q 8指定qp数量，比如8个qp, --qp=<num of qp’s> Num of qp’s(default 1)。服务器和client都需要指定-R使用rdma_cm建链, --rdma_cm 。服务器和client都需要指定--run_infinitely一直持续的运行，每间隔-D的参数秒打印-s 1024指定size大小， --size=
一般常用
-D 10, --duration 指定打流时间 比如-D 10指定10秒-a默认只会用65535的msgsize，这里会使用从2到2^23（8M）大小的size, --all-c RC指定类型，默认RC。, --connection=<RC/XRC/UC/DC>-i 1指定IB的port。, --ib-port= Use port of IB device (default 1)-m 4096指定mtu, --mtu= MTU size : 256 - 4096 (default port mtu)-p 18516Listen on/connect to port (default 18515) 指定建链监听端口, --port=-u 14指定qp超时时间，默认, --qp-timeout= QP timeout, timeout value is 4 usec * 2 ^(timeout), default 14 ,约等于 65ms--report_gbits使用Gbps的方式报告--rate_limit=<rate>设置最大限速Set the maximum rate of sent packages. default unit is [Gbps]. use --rate_units to change that.
其他 全量help信息
root@localhost:~# ib_write_bw --help
Usage:ib_write_bw start a server and wait for connectionib_write_bw <host> connect to server at <host>Options:-a, --all Run sizes from 2 till 2^23-b, --bidirectional Measure bidirectional bandwidth (default unidirectional)-c, --connection=<RC/XRC/UC/DC> Connection type RC/XRC/UC/DC (default RC)--log_dci_streams=<log_num_dci_stream_channels> (default 0) Run DC initiator as DCS instead of DCI with <log_num dci_stream_channels>--log_active_dci_streams=<log_num_active_dci_stream_channels> (default log_num_dci_stream_channels)--aes_xts Runs traffic with AES_XTS feature (encryption)--encrypt_on_tx Runs traffic with encryption on tx (default decryption on tx)--sig_before Puts signature on data before encrypting it (default after)--aes_block_size=<512,520,4048,4096,4160> (default 512)--data_enc_keys_number=<number of data encryption keys> (default 1)--kek_path path to the key encryption key file--credentials_path path to the credentials file--data_enc_key_app_path path to the data encryption key app-d, --ib-dev=<dev> Use IB device <dev> (default first device found)-D, --duration Run test for a customized period of seconds.-f, --margin measure results within margins. (default=2sec)-F, --CPU-freq Do not show a warning even if cpufreq_ondemand module is loaded, and cpu-freq is not on max.-h, --help Show this help screen.-i, --ib-port=<port> Use port <port> of IB device (default 1)-I, --inline_size=<size> Max size of message to be sent in inline-l, --post_list=<list size>Post list of send WQEs of <list size> size (instead of single post)--recv_post_list=<list size> Post list of receive WQEs of <list size> size (instead of single post)-L, --hop_limit=<hop_limit> Set hop limit value (ttl for IPv4 RawEth QP). Values 0-255 (default 64)-m, --mtu=<mtu> MTU size : 256 - 4096 (default port mtu)-n, --iters=<iters> Number of exchanges (at least 5, default 5000)-N, --noPeak Cancel peak-bw calculation (default with peak up to iters=20000)-O, --dualport Run test in dual-port mode.-p, --port=<port> Listen on/connect to port <port> (default 18515)-q, --qp=<num of qp's> Num of qp's(default 1)-Q, --cq-mod Generate Cqe only after <--cq-mod> completion-R, --rdma_cm Connect QPs with rdma_cm and run test on those QPs-s, --size=<size> Size of message to exchange (default 65536)-S, --sl=<sl> SL (default 0)-t, --tx-depth=<dep> Size of tx queue (default 128)-T, --tos=<tos value> Set <tos_value> to RDMA-CM QPs. available only with -R flag. values 0-256 (default off)-u, --qp-timeout=<timeout> QP timeout, timeout value is 4 usec * 2 ^(timeout), default 14-V, --version Display version number-w, --limit_bw=<value> Set verifier limit for bandwidth-W, --report-counters=<list of counter names> Report performance counter change (example: "counters/port_xmit_data,hw_counters/out_of_buffer")-x, --gid-index=<index> Test uses GID with GID index-y, --limit_msgrate=<value> Set verifier limit for Msg Rate-z, --comm_rdma_cm Communicate with rdma_cm module to exchange data - use regular QPs--out_json Save the report in a json file--out_json_file=<file> Name of the report json file. (Default: perftest_out.json in the working directory) --cpu_util Show CPU Utilization in report, valid only in Duration mode --dlid Set a Destination LID instead of getting it from the other side.--dont_xchg_versions Do not exchange versions and MTU with other side --force-link=<value> Force the link(s) to a specific type: IB or Ethernet.--ipv6 Use IPv6 GID. Default is IPv4--ipv6-addr=<IPv6> Use IPv6 address for parameters negotiation. Default is IPv4--bind_source_ip Source IP of the interface used for connection establishment. By default taken from routing table.--mmap=file Use an mmap'd file as the buffer for testing P2P transfers.--mmap-offset=<offset> Use an mmap'd file as the buffer for testing P2P transfers.--mr_per_qp Create memory region for each qp.--odp Use On Demand Paging instead of Memory Registration.--output=<units> Set verbosity output level: bandwidth , message_rate, latency --payload_file_path=<payload_txt_file_path> Set the payload by passing a txt file containing a pattern in the next form(little endian): '0xaaaaaaaa, 0xbbbbbbbb, ...' .Latency measurement is Average calculation --use_old_post_send Use old post send flow (ibv_post_send).--perform_warm_up Perform some iterations before start measuring in order to warming-up memory cache, valid in Atomic, Read and Write BW tests--pkey_index=<pkey index> PKey index to use for QP--report-both Report RX & TX results separately on Bidirectional BW tests--report_gbits Report Max/Average BW of test in Gbit/sec (instead of MiB/sec)Note: MiB=2^20 byte, while Gb=10^9 bits. Use these formulas for conversion:Factor=10^9/(2^20*8)=119.2; MiB=Gb_result * factor; Gb=MiB_result / factor--report-per-port Report BW data on both ports when running Dualport and Duration mode--reversed Reverse traffic direction - Server send to client--run_infinitely Run test forever, print results every <duration> seconds--retry_count=<value> Set retry count value in rdma_cm mode--tclass=<value> Set the Traffic Class in GRH (if GRH is in use)--flow_label=<value> Set the flow_label in GRH (if GRH is in use)--use_hugepages Use Hugepages instead of contig, memalign allocations.--use-null-mr Allocate a null memory region for the client with ibv_alloc_null_mr.--wait_destroy=<seconds> Wait <seconds> before destroying allocated resources (QP/CQ/PD/MR..)--disable_pcie_relaxed Disable PCIe relaxed orderingRate Limiter:--burst_size=<size> Set the amount of messages to send in a burst when using rate limiter--typical_pkt_size=<bytes> Set the size of packet to send in a burst. Only supports PP rate limiter--rate_limit=<rate> Set the maximum rate of sent packages. default unit is [Gbps]. use --rate_units to change that.--rate_units=<units> [Mgp] Set the units for rate limit to MiBps (M), Gbps (g) or pps (p). default is Gbps (g).Note (1): pps not supported with HW limit.Note (2): When using PP rate_units is forced to Kbps.--rate_limit_type=<type> [HW/SW/PP] Limit the QP's by HW, PP or by SW. Disabled by default. When rate_limit is not specified HW limit is Default.Note: in Latency under load test SW rate limit is forced--write_with_imm use write-with-immediate verb instead of write
