This page contains information on data servers network benchmarking and memory benchmarking; all data is still work-in-progress.

CLIENT SPECS & BENCHMARKS

SPECS

• 160 client nodes
• 1x Samsung PM9A3 Gen4 NVMe SSD 1.92TB
• 1x ConnectX-6 (200 Gb/s)
• 1 Gb/s management
• Single socket AMD EPYC 7453/7413 processor
  • 96 client nodes -> 7413
  • 64 client nodes -> 7453
• 128 GB DDR4 3200 MT/s

BENCHMARKS

• Aggregated node performance
  • Streaming reads -> 850 GB/s
  • Streaming writes -> 340 GB/s
  • Random reads -> 130.6M IOPS
  • Random writes -> 84M IOPS
• Average node performance
  • Streaming reads -> 5.3 GB/s
  • Streaming writes -> 2.15 GB/s
  • Random reads -> 816k IOPS
  • Random writes -> 525k IOPS
• Average network performance
  • 23.1 GB/s (185 gb/s)
• Average memory performance
  • TBA

DATA SERVER SPECS & BENCHMARKS

SPECS

• 12 data server nodes
• Assorted variety of NVMe SSD’s
• 2x ConnectX-6 (200 Gb/s)
• 1 Gb/s management
• Single socket AMD EPYC 7502 processor
• 128 GB DDR4 2933 MT/s

BENCHMARKS**

• Aggregated node performance
  • Streaming reads -> 490 GB/s
  • Streaming writes -> 440 GB/s
  • Random reads -> 70.5M IOPS
  • Random writes -> 64.1M IOPS
• Average node performance
  • Streaming reads -> 54.1 GB/s
  • Streaming writes -> 48.7 GB/s
  • Random reads -> 7.83M IOPS
  • Random writes -> 7.12M IOPS
• Average network performance
• TBA
• Average memory performance
• TBA

** Missing 3 drives across two data servers; benchmarks only across 9 of 12 total data servers

VERY LARGE SERVER SPECS & BENCHMARKS

SPECS

• 4x Mongo server nodes
• 16x 2TB Kioxia CD8P-R Gen5 x4 U.2
• 4x ConnectX-7 (400gb/s each)
• 1 Gb/s Management
• Dual socket 64-core 3.3Ghz EPYC 9575F CPU - 256 threads with HT
• 24 x 32 (768) GB 5600 MT/s DDR5

BENCHMARKS

• Aggregated node performance
  • Streaming reads -> 558.2 GB/s
  • Streaming writes -> 225.3 GB/s
  • Random reads -> 84.8M IOPS
  • Random writes -> 41.5M IOPS
• Average node performance
  • Streaming reads -> 139.5 GB/s
  • Streaming writes -> 56.3 GB/s
  • Random reads -> 21.2M IOPS
  • Random writes -> 10.4M IOPS
• Average network performance
  • 133.8 GB/s (1070 gb/s)
• Average memory performance
  • TBA

GRACE-GRACE SERVER SPECS & Benchmarks

SPECS

• 1x Grace-Grace node
• 2x Samsung PM9A3 Gen4 x4 M.2 3.84 TB
• 1x ConnectX-7 (400gb/s)
• 1 Gb/s Management
• Dual Socket 72-core 3.4Ghz NVIDIA ARM Neovers-V2
• 2x 240 GB 8532 MT/s LPDDR5X

BENCHMARKS

• Node performance
  • Streaming reads -> 10.9 GB/s
  • Streaming writes -> 4.3 GB/s
  • Random reads -> 1.64M IOPS
  • Random writes -> 1.05M IOPS
• Average network performance
  • 49.5 GB/s (396 gb/s)
• Average memory performance
  • TBA

BENCHMARK DETAILS

• FIO to benchmark storage
• iperf2/perftest to benchmark network
• STREAM to benchmark memory? (running into issues with STREAM)