10Gb and SFP+ Transceivers

A recent network change in my home led me down the rabbit hole of how transceivers convert SFP+ to RJ45/Ethernet and SFP+ modules.

My goal was to determine if an SFP+ transceiver’s power conversion is less optimal 24/7 than a baseT NIC (like an X540-T2). I did not get a clear answer from Google, Reddit, Discords, ServeTheHome, and more. Most of the time, asking questions about any 10Gbps and copper gets you back an answer like “Fiber.” Often unhelpful and defeating the purpose of exploration and curiosity.

My conclusion at the end of all of this is:

  • Avoid SFP+ transceivers as much as you can. From the Fiber people to pragmatic low-voltage engineers, converters aren’t great. Basically, avoid conversion. If you can roll DAC or Ethernet on either side, try to do that.
  • For 10Gbps over existing Ethernet runs, a BaseT NIC is better for these reasons:
    • Many have a larger heatsink or a larger heatsnk can be installed
    • Many have a small fan on them to help with cooling
    • They’ll be inside of your chassis where you can add additional fans more easily to help with cooling
    • The above points matter because, anecdotally, 10Gbps runs very hot and consumes power and the heat tends to lead to hardware failures. If you read through enough network switch documentation, it’s not clear if running a SFP+ transceiver for BaseT conversion 24/7 is sustainable. Some switches will say only some of the SFP+ cages can be used this way. Some allow all of them to be used this way. The consensus is the power and heat create problems. All things that can be solved, though.
  • The X540-T2 and similar cards are rated for 10Gbps at 100m Cat6A runs and can do that over Cat5e despite claiming not. That is farther than most of the cheap SFP+ transceivers on the market (30m rating). So if you’re running a long distance or using existing long-run Ethernet, having one side of that equation (the host) running an x540-T(1 or 2) or BaseT network card could perform better and be easier to cool.

The top 10GBase-T SFP+ transceiver I would recommend is: 10GTek 80m (Broadcom) and my runner up would be the Wiitek 30m (Marvell)

So here’s some background that led me here, and some more information about the transceiver modules I tested.

The home layout is:

  • The Mini is in an office and has 2 sets of runs to a central network room. One run is Cat5e from when the house was built. A second run was done recently with Cat6A.
  • The server room has only one run with 4 drops of Cat6A (done at the same time the office was done).
  • The network room has around 30+ inbound Ethernet cables from various parts of the house. Half are original to the home construction, and the other half from a set of runs I did when we moved in (the Cat6A runs, as I refer to them as). The Cat6A runs are, when alongside existing Cat5e runs, are longer as we allocated an additional 10’ or so inside of the network room. All of the Cat5e runs terminate in a network in-wall box (very annoying as I have a rack and have to use couplers on many of the runs).

When I first set up a new Aruba 1930 switch, the connection between the server room and network room negotiated 10Gbps over Ethernet using SFP+ transceivers on the switch. I was using, initially, the 10GTek 30m SFP+ to Ethernet transceiver. Unfortunately, the run to my office’s Mac Mini over Cat6A was only negotiating at 5Gbps and was struggling to receive at a rate above 2Gbps (it could send at a full 5Gbps). I tested that same 10GTek 30m transceiver using the older Cat5e run to my office’s Mac Mini and negotiated and achieved a full 10Gbps connection. So something was/is wonky about my Cat6A… I think it is the length.

I did discover modules that worked (I’ll share in follow-up replies) but here are some of my learnings…

I found that some transceivers run wildly hotter than others. My ipolex runs noticeably less hot than my 10GTek transceiver of the same rating (30m). Data flow impacts the heat. But generally found ipolex ran the hottest (measuring with a hopefully accurate temperature gun).

And I also learned that the way in which a transceiver is used in a switch is to spoof the transceiver code, so the switch thinks the transceiver is something it is not. So when you are clicking “HP Aruba” or “Unifi” on those Amazon pages, what you’re doing is getting a transceiver that has been encoded to register in those switches as a valid component. But here’s the kicker, some switches don’t support the RJ45 transceiver, so the unit gets encoded as a LR or optical module.

This can have unintended side effects, too. If a switch claims it only works with certain kinds of optical modules and a 3rd party has encoded their module to “fake it,” you can end up in situations where the switch is doing things that would be OK for the intended module but not okay for a spoofed module.

One of the most helpful reviews I found was from an Amazon review of a Fiber Mall SFP+ transceiver by a person named K:

I have previously purchased and used 10GBase-T SFP+ 30m (Marvell 88X3310) modules from two other vendors (Wiitek & MikroTik), as well as a 10GBase-T SFP+ 80m (Broadcom BCM84891) module from one other vendor (10Gtek). These modules have been used in MikroTik hardware; specifically the MikroTik CCR2004-1G-12S+2XS (“CCR2004”) and the MikroTik CRS317-1G-16S+RM (“CRS317”).
I purchased two of these Fiber Mall 10GBase-T SFP+ 80m (Broadcom BCM84891) modules in the hopes of getting a Broadcom BCM84891-based module with working temperature/voltage reporting, as that is something that is missing from both the Wiitek Marvell 88X3310-based and 10GTek Broadcom BCM84891-based modules.
While the MikroTik Marvell 88X3310-based modules do have working temperature/voltage reporting, they are unfortunately extremely hot and power hungry (far more so than every other Marvell 88X3310-based module out there — which is particularly annoying given that the MikroTik Marvell 88X3310-based module is the only SFP+ 10GBase-T module with fully working NBASE-T support!).
Anyways, among all of these modules, these Fiber Mall Broadcom BCM84891-based modules are easily & by far my favorites so far! Why?
#1: They have working temperature/voltage reporting! This is a rarity on Marvell 88X3310-based modules, and the only other Broadcom BCM84891-based module I have been able to test so far also lacked working temperature/voltage reporting, so this is a very big advantage!
#2: They have had no trouble whatsoever with reliably achieving a 10GBase-T link on my longest run of cat5e (which cannot be said of any of the Marvell 88X3310-based modules, which struggle to manage a 5GBase-T link at the absolute most, and will typically settle at only 1000Base-T on that run).
(Please note that I have been able to successfully push 10 Gbps over a cat5e run approximately ~3x this distance by going from my port 1 on my Intel X550-AT2 back to port 2 on my Intel X550-AT2, so the issues with achieving full speed links on this run with Marvell 88X3310-based modules are presumably due primarily to a combination of power limitations coupled with the inferior power efficiency of the Marvell 88X3310-based modules.)
#3: Broadcom BCM84891-based modules are dramatically superior to Marvell 88X3310-based modules in certain applications.
Yes, all of the Broadcom BCM84891-based modules on the market are priced at a significant (35–100%) price premium over the Marvell 88X3310-based modules, however these particular Broadcom BCM84891-based modules are one of the cheapest Broadcom BCM84891-based modules available.
Furthermore, the Broadcom BCM84891 is well worth the price premium over the Marvell 88X3310 in situations involving:
• Achieving 10G links on ≥30m runs of cat6a cabling
• Achieving 10G links on runs of cat5e/cat6 cabling that are too long for Marvell 88X3310-based modules to reliably achieve 10G link speeds over. Note that in this case “too long” is often MUCH lower than 30m — for example, my use case is a link that should be somewhere in the ~15–20m range, and I’ve seen many other users online reporting similar issues (and similar resolutions!) at a very wide range of different link lengths.
• Situations where module power consumption is a significant consideration. Note that Broadcom BCM84891-based modules are generally specified for a power draw of ~1.6–2.0 W for ≤30m links (or 2.0–2.5 W for ≤80m), while Marvell 88X3310-based modules are generally specified for a power draw of ~2.4–3.4 W for ≤30m links — this can be a rather dramatic (~17–41%) difference in power consumption for ≤30m links!
• Situations where module temperature is a significant consideration
• Situations where the increased fan noise resulting from the elevated module temperatures & power consumption would be undesirable, or where supplementary cooling would otherwise be required (e.g. with fully-passively-cooled hardware such as a MikroTik CRS305 or CRS309).
One example of a situation like the last example is my use case for my CRS317. I have it located in my bedroom/office, and so having it frequently kick its (normally quietly idling) fans up to max speed (which is very loud!) for an extended period of time to try to cool down the overheating Marvell 88X3310-based MikroTik modules operating at 10G is obviously a significant annoyance.
I even had to relocate the position of the MikroTik modules to ensure they were placed directly in front of the rear fans in order to increase airflow to them. While this significantly reduced how often & how long the fan would spin up to max speed, it unfortunately did not do so by anywhere near enough to make it tolerable.
Yet the Fiber Mall Broadcom BCM84891-based module that is acting as the switch’s uplink port (which is also in the absolute worst SFP+ port on the switch from an airflow point of view!) is running FAR cooler than the MikroTik module that sitting directly in the path of the fans (and therefore receiving FAR more airflow from them), even though it is operating at the exact same link speed, and is (at least currently) passing the exact same amount of traffic through it!
Of course, if you’re working with cat6a links under 30m with actively cooled host hardware located in an area where noise is of no concern (e.g. datacenter, MDF, IDF), and have sufficient port group power budgets, then the cheaper Marvell 88X3310-based modules are obviously going to be a better fit for you.
#4: The price is right — and it’s available right here on Amazon.com with fast and free Prime shipping!
The only possible downside I can think of is that the multi-packs sold on Amazon currently aren’t Prime-eligible and have extremely long shipping windows, however I suspect part of the reason for the lengthy shipping window is pandemic-related shipping delays, and it’s possible that the seller may begin to stock those (and/or other multi-pack options) in US warehouses down the road. Regardless, this minor downside is not something that I care much about, but I thought it was still worth mentioning.
Overall, I am EXTREMELY satisfied with these modules, and plan to purchase more of them in the near future!

Long review but was worth it for me to understand the different module chipsets, too.

My other big takeaway is that the Aruba 1930 is not picky on SFP+ modules. This is wonderful for me as a new owner of one!

On placement of Copper SFP+ transceivers, I found this web site helpful:

Particularly, this layout (from their site):

I found another Reddit thread touching on the Broadcom vs not topic: https://www.reddit.com/r/networking/comments/d9hwpj/there_are_now_80_meter_10g_rj45_sfp_modules_whats/

excerpt image:

Further evidence of the particular chipset BCM84891:

I was also able to find this chipset, supposedly, on Amazon for non-Aruba (but many other brands) and fairly cheap: https://amzn.to/3DDIcoB

I’ve picked up a couple of different 30m/80m adapters to test with:

  1. 10GTek 80m (Broadcom)
  2. Fiber Mall 80m (Broadcom)
  3. Wiitek 30m (Marvell)
  4. 10GTek 30m (Marvell)
  5. Ipolex 30m (Marvell)

In the image above, from left to right: Wiitek, Ipolex, 10GTek 30m, Fiber Mall (80m).

I have now had a chance to test all of these modules.

For each transceiver, I used the following test setup:

  • Unraid running an x540-T2, single link
  • An Aruba 1930 using the top 2 SFP+ 10Gbps ports
  • A Mac Mini with the 10GbE Aquantia
  • Run LibreSpeed from Unraid and do tests between the two machines via the switch
  • Run iPerf3 and do standard single-thread test in both directions (once with a default run and once with a -R).

TLDR; 10GTek 80m SFP+ to RJ45 performed perfectly and was priced within 10% of 30m modules. The Fiber all 80m performed the same as the 10GTek 80m module but is 2x the price as of writing this. The Wiitek did similarly well and is slightly cheaper and rated for 30m. But I think it important to make sure your switch recognizes the device fully (see below) and if you can get it coded for your device I would do that.

I believe the Ipolex 30m and the 10GTek 30m devices are the same. They have all of the same markings and label themselves “ASF-10G-T” They both would not negotiate >5Gbps over a (very likely) ~30m Cat6A run. They could not push data over that run >2Gbps. They received data at 5Gbps. On my shorter Cat5e run, they negotiated at 10Gbps and achieved the full 10Gbps throughput well.

The Wiitek 30m module might be a different chipset than the Ipolex and 10GTek 30m variants or built differently. It did negotiate at a full 10Gbps over the same Cat6A run that the other two struggled on. And it achieved full 10Gbps throughput. It also achieved full 10Gbps negotiation and throughput on the Cat5e run.

The 80m adapters (Fiber mall and 10GTek) performed perfectly on all connection configurations and speed tests. They both look identical concerning manufacturing and board markings:

10GTek 80m module:

As I dug more into the Aruba’s handling of the SFP+ port, I noticed an oddity. In the UI you can hover over a green-lit port to learn about its details:

The UI when you hover over an inactive/down port:
image

The UI when you hover over an active/up port (notice the SFP+ to RJ45 transceiver module claims it is a BASE-SR):
image

When I looked at the 10GTek 80m transceiver (not coded for Aruba, just a generic module), it will negotiate but you might see this UI:
image

I noticed immediately following this UI, if you refresh, you will not longer see any popup on hover. All green network ports have no user interaction. This seems worrisome to me. So I pulled the 10GTek 80m module and put in one of the Ipolex 30m modules and noticed that it, like the Wiitek, also showed BASE-SR:
image

I am not quite sure why this happens but my suspicion is that 10GTek 80m adapter was not coded properly. The Fiber Mall 80m, if I recall right, did show BASE-SR. I have already returned all of the test modules so unfortunately I can not reconfirm some of these suspicions.

I’ll stick with using the Wiitek for now.

Please leave blank for additional content