Recently Gigabyte sent their MD70-HB0 motherboard over for review. The Gigabyte MD70-HB0 is a dual-socket LGA2011-3 motherboard that accepts up to two Intel Xeon E5-2600 V3 processors. We originally spotted the board in our Gigabyte Haswell-EP motherboard release piece. Since then we have had a significant amount of time to spend with the platform. This is certainly a higher-end motherboard option so there are lots to go over in the review. Aside from being based on the newest Haswell-EP platform including the Intel C612 PCH, the platform also includes best-of-breed solutions like an integrated dual-port 10Gbase-T NIC and an LSI SAS3008 onboard. The motherboard is already available in many parts of the world and is expected to hit major US retailers shortly. mz7km240hmhq-00005
We used a fairly typical DP setup for this type of server and what we have used previously in our Intel Xeon E5-2600 V3 coverage.
- CPU: 2x Intel Xeon E5-2690 V3
- Motherboard: Gigabyte MD70-HB0
- Memory: 8x SK. Hynix 16GB DDR4 2133MHz ECC RDIMMs
- SSD: 2x Samsung 840 Pro 256GB 4x Crucial M500 240GB, 4x Samsung SM843 480GB + 6x SanDisk Lightning SLC 200GB SAS and 2x SanDisk Optimus MLC 400GB SAS
- Operating Systems: Ubuntu 14.04 LTS, Windows Server 2012 R2, VMware ESXi 5.5, CentOS 6.5 Intel Xeon Processor
The test platform was rather expansive. There was one point while running through our compatibility testing where we had a second HBA with 8 drives, an Intel Fortville NIC, Mellanox ConnectX-3 FDR NIC, an Intel DC P3700 NVMe SSD and a Fusion-io 160GB SLC PCIe SSD installed in the system. Talk about a slow boot!
Gigabyte MD70-HB0 Overview
First off, the Gigabyte MD70-HB0 is a large EATX/ SSI EEB motherboard measuring 13″ x 12″ in size. Several years ago motherboards this large were less common, making fitment a potential challenge. These days, such large motherboards are accepted by almost all chassis except the short-depth variants making it fit in the majority of cases.
As one can see, the eight DDR4 DIMM slots per processor are aligned so airflow will go through CPU0 and warm CPU1. With today’s fans this is less of an issue but something to keep in mind when cabling 1U servers around this processor configuration.
In terms of expansion there are three PCIe 3.0 x16 slots and three x8 slots. These can run in either 3x PCIe 3.0 x16 and 1x PCIe 2.0 x4 in a x8 slot. Alternatively using a PCIe switch 1x PCIe 3.0 x16 (x16 bus) slot
2x PCIe 3.0 x16 (x8 electical), 2 x PCIe x8 (x8 electrical) and 1x PCIe 2.0 x8 (x4 electrical) slot.
In addition, there is a copper heatsink that keeps the Intel X540 10Gbase-T controller cool. 10Gbase-T controllers run significantly warmer than SFP+ alternatives so the addition of the higher-end copper heatsink is excellent. 10Gbase-T does offer one major benefit compared to SFP+ based interfaces, it can integrate easily into existing gigabit or 10-gigabit networks using inexpensive and often existing cabling.
Beyond the add-on 10Gbase-T Gigabyte adds a LSI-Avago SAS3008 controller. The sample we received came default in IR (RAID) mode which can be useful for running disks in RAID 0, 1 or 10. Furthermore, the 8-port 12gbps SAS controller can connect to eight SAS 12gbps drives or SAS expanders. LSI-Avago’s newer 12gbps expanders can even use the 8x 12gbps lanes to provide a 96gbps back haul using 6.0gbps drives. That is a major improvement over previous generations which were limited to the lowest common speed in the system.
There are a total of ten 6.0gbps SATA3 ports along the bottom of the motherboard which are from the Intel C612 PCH. Combined with the LSI-Avago SAS controller, the motherboard has 18 total storage ports which is enough for most 1U, 2U and 3U chassis.
Gigabyte also takes advantage of the new C612 PCH’s USB 3.0 capabilities and adds an internal Type-A header for USB keys. A common use is to install VMware ESXi as a hypervisor on a USB key. The attached drives are then used for virtual machine storage. USB 3.0 drives are faster so for the rare reboots, this is a great feature.
In terms of the rear I/O panel there are are two legacy ports, a 9-pin serial port and a 15 pin VGA port. Next to these are the two 10Gbase-T ports providing high-speed networking. On the right side there are two more USB 3.0 ports and the IPMI network jack atop.
Overall the platform has excellent storage features with 18 high-speed storage ports and 10GbE built in.
To capture these images we utilize our FLIR Ex series professional thermal imaging camera and turn on FLIR’s MSX enhancements so we can see components outlined clearly. We put the system under 100% CPU load for a period of 24 hours to let “heat soak” set in prior to taking the images.
As we would expect, there are a few extra hot spots on the motherboard. Integrated LSI HBAs and Intel X540 10Gbase-T NICs are some of the warmest components on the motherboard. Still, the warmest spot in our test bed was under 44C so well within operational tolerances. Gigabyte’s inclusion of a copper heatsink on the Intel X540 does seem to keep the component cool. One can also see that the second CPU area is warmer than the first, this is expected behavior with higher-power processors and in-line configurations.
Gigabyte – Avocent IPMI Interface
Using this motherboard and testing it through VMware ESXi 5.5, Windows Server 2012 R2, Hyper-V Server 2012 R2, Ubuntu 13.10 and CentOS 6.5 required many operating system installations. We also needed to reboot the system many times during our card compatibility testing where we cycle through various cards such as Mellanox ConnectX-3 VPI 56gbps Infiniband/ 40GbE and LSI SAS 3008 controllers. The Avocent IPMI solution is simply great. After a few weeks of testing it was very stable and more responsive than a standard AMI MegaRAC solution common in the industry.
For those wondering, we did a full two part Gigabyte IPMI walk through (see Part 1 and Part 2). Key features include the ability to remotely mount multiple images over the network and iKVM features.
After spending some time with the Gigabyte-Avocent MergePoint solution it is certainly one of the better web interfaces out there remaining responsive during our testing. Lenovo uses a similar Avocent based system for their server OOB management so it is a relatively mature solution.
Overall this is an excellent motherboard by Gigabyte. Hopefully they continue to build out their offerings. One thing I would like to see is a common rear I/O format that is compatible with existing enclosures. Right now for 2U and larger enclosures, this is not an issue. For 1U enclosures that do not have a standard IO shield opening, this does limit options slightly. On the other hand, most of these motherboards will likely be sold with enclosures and many will be 2U or larger so this is likely a non-issue.
In terms of value, an Intel X540-t2 adapter currently costs more than $500 and a LSI SAS 9300-8i costs about $350. The Gigabyte MD70-HB0 will have a MSRP of under $700 so while it is a higher-end motherboard, the price is probably 35%-40% lower than buying a comparable motherboard, plus two add-in cards. Overall, the high-level of top quality integrated components (Intel X540 and Avago-LSI SAS3008) and management features (Avocent) combine to make an awesome package.