This is the first article of The Networking Hardware Vendors Assessment 2021 series. In this series, I’m going to talk about some major networking hardware vendors, their hardware and software, their achievements and what to expect if you buy their hardware in 2022.
MikroTik has always been the definitive choice if my purchase target is the cost effectiveness. This is not to say their product is good (the “good” good), but:
- They produce some products that perfectly match some weird usage I needed
- They pack a lot functionalities into all their products so you can kind of expect a high baseline of what a low-end MikroTik device can do (not in terms of performance, obviously)
This is the MikroTik way to find its position in the market.
Hardware
CCR2000 Series
Support for Tile architecture been dropped by Linux in early 2018, MikroTik has been building their new gears on several ARM (aarch64) CPUs. This shed some light on the performance of their high-end routers when being used as BGP border routers. Thus said, high-end CCR1 series has always been used in the role of BRAS due to its ability to terminate a lot PPPoE sessions and process a lot queues parallelly thank to its high core count. There are still a lot of things in software to optimize (e.g. interrupt affinity) for CCR2 to be even considered for a full replacement of CCR1 series.
RB5000 Series
I’ve been a long time user of RB2011/RB3011/RB4011. A form factor of 1U makes it too weird for homes, and its bad performance has always been a drawback every time I try to use it in a more “pro” environment. RB5009 is MikroTik’s response to this ever-growing niche market. It’s small enough to be put into a home, and it has decent performance, plus a SFP+ port! Well, 2 SFP+ ports might be better so I can set up LACP, but RB5009 comes with a form factor so much better than its predecessors. (What, you say the 2.5Gbps port? I avoid copper cables as much as possible. 2.5G/5G/10G copper solutions are too expensive and have a bad user experience. Get yourself some second hand SFP+ modules as they are better in a magnitude. )
hAP ac3
This product makes no sense to me. It’s so much larger than hAP ac2, and the SFP port on the hAP ac is nowhere to be found on the hAP ac3. The LTE model is so expensive that I’ll certainly choose a cheap Huawei USB LTE modem plus a cheap USB hub plugged into a old model.
The hardware quality does not meet my expectation too. The antenna connectors are so loose. The base mounting clip is impossible to bend. The spare user LED and the beeper is missing.
I would choose hAP ac2 over hAP ac3, unless one need the extra 128MiB system memory to run some serious BGP sessions on a home wireless router. (And forget running Docker on routers; if I need a hyperconverged infrastructure, I’d get some real infrastructure instead. )
CRS300 Series
MikroTik finally realized they need to use serious switch chips to make a proper switch. And when the hardware is somehow ready for heavy work, their software still need a long way to go. Models like CRS326 and CRS354 looks fine for an enterprise, but when you really try to use it, you will discover a lot missing pieces in the software.
On the other side, low-cost passive-cooled 10G switches like CRS305 and CRS309 are very sweet for a homelab. (Well, if your homelab includes MPLS VPN, RDMA storage applications, network security experiments or providing access for untrusted devices, then choose a more serious switch.)
Old Devices
The new shiny RouterOS 7 comes with a lot more features, but it also rendered the 16MiB flash that comes with most low-end devices too small for everyday use. You’ll need to NetInstall every time you want to upgrade and this is not the best user experience. I’m not sure if this problem will be resolved in software or not, so we can only wait and see.
Another thing I noticed during the hardware platform transition is that some new packages (e.g. ZeroTier, Docker) are only available for ARM platforms. I hope when RouterOS 7 enters stable channel, these packages will be available for other major hardware platforms. (Update: NO. )
Software
Sometimes you can feel that MikroTik has no bar on software quality at all. The web console randomly reorders typed characters when used under high latency. Configuration will not be applied correctly if you click the items in the WinBox too quickly (yes, I’m a rhythm game player). MPLS L3VPN nexthops are always displayed incorrectly in the GUI. WinBox never properly supported IME, nor do it disable IME in the input boxes. Considering their devices’ price, I think these issues are just the hidden cost you have to live with.
After all there will be a RouterOS 7, right? The new version we have been waiting for over 10 years. Every time we ask for a new feature in the forums, there will be a reply stating it will be available in RouterOS 7. So RouterOS 7 must be able to run true VRFs, terminate VXLANs, connect to OpenVPN servers with UDP, do NAT for IPv6, open a beer can, light a fire, and launch missiles, right?
Then the RouterOS 7 is live. It comes with really a lot of new features, but they are just features that are not even properly designed, and sometimes they don’t integrate with other parts of the OS. I can feel their software PM doesn’t use — or even need these features in their life; these features are just collected through the forum or copied from other vendors, without thinking if their implementation make any sense to the user. For example, there are a lot people asking for management services to be available in VRF since 2010, and on RouterOS 6, it is not possible without some ugly and easy-to-break hack in the iptables (firewall). RouterOS 7 tried to expose management services to the VRFs, but 1) you can only select one VRF per service; 2) all host outbound services go through main routing table; 3) non-management services like DNS, TFTP, SMB and SNMP are still broken. At this situation, we can only hope during the beta period of RouterOS 7, these problems get discovered and resolved.
Use Cases
Note: All the use cases are evaluated under RouterOS 6 long-term version unless stated.
Routing
MikroTik devices are generally suitable to act as small scale, non-serious routers and BRASes. Avoid using them for large scale networks or run BGP at AS border. WinBox is too fragile so you have to be slow when editing BGP configurations; and since RouterOS doesn’t have atomic commit and rollback, if you accidentally dragged a route filter to another position, you might squirt a lot routes to other routers.
MPLS L3VPN is always available and always buggy. Always evaluate your use case inside a lab before using them. If you need to import/export routes between main table and a L3VPN VRF table, use a jumper cable to connect 2 spare ethernet ports together and run some routing protocol on them.
RouterOS 6.x doesn’t support any means to apply routing policy to IPv6, or do NAT on IPv6. You need RouterOS 7 on that.
Switching
It is unsuitable for any serious switching usage. Essential features missing on all models:
- DHCP snoopin: display database and enforce lease expiry
- ARP snooping
- ND snooping
- RA snooping
- IP source guard
- DCBx
And remember you need to look through a lot tables across their wiki to make sure if the model supports hardware offloading of a certain feature. This is always a pain.
If you need cheap dumb layer 2 switches, or if you need to run VLANs at home without the need first hop security, go ahead and buy one MikroTik switch, as it is a lot better than a real dumb switch. If you need L3 switching features or mLAG, or want to purchase now and upgrade to RouterOS 7 later, buy CRS3xx only. DO NOT buy CRS1xx or CRS2xx.
As for OpenFlow, I never used it so I can’t say much about it.
Firewall
Firewall in RouterOS is basically iptables + ip set + ip rules, so if you came from the Linux side, you’ll be pretty familiar with its logic. There will be some modules missing, but the basic functionalities are there, and it is Turing complete.
For seasoned enterprise firewall users, setting up ZBFW on RouterOS with a lot zones can be uncomfortable. You need to create interface lists, create a lot rules with “connetion-state=new”, and walk through a lot chains. Good luck with that. Also remember to exclude IPsec traffic for NAT.
CPE and VPN Headend
MikroTik routers are good CPEs for small scale network if you are happy to accept outdated insecure ciphers. Their PPTP, L2TP, OpenVPN clients are easy to configure, with dynamic domain resolution capabilities. But no DMVPN-like solutions exist on RouterOS 6, unless you use a lot script to automate the routers.
RouterOS 7.0 supports WireGuard and ZeroTier One, which makes the situation a lot better. It’s nice to see MikroTik adopting new things from the open source world.
Don’t try to make it MPLS L3VPN PE devices while expecting it to simultaneously deliver Internet access to the end user. Just don’t. The embedded DNS server will not work inside a VRF. And route import/export are sometimes impossible that I’d prefer using a jumper cable to do route leaking, which makes it impossible to have collided IP ranges in different VRFs.
VPN Concentrator and Access Server
If you are happy about outdated cipher and OpenVPN TCP mode, it can act like a good VPN concentrator. It comes with a free DDNS service which had multiple days of outage this year, so don’t rely too much on it. And you cannot start multiple VPN servers of the same type. And the dynamic interface thing will make you crazy when any of your client has a flapping Internet connection. In the end you will choose to statically create pseudo interfaces for every client on every protocol, and it is a pain to set up.
RouterOS 7 will bring new protocols and performance enhancements I believe, but don’t expect the pitfalls inside its design to become better.
If you really need something that works, use Linux.
Wireless Access
MikroTik does a good job in long-range point to point link, but for indoor AP, it’s another story. Unicode SSID doesn’t exist as WinBox does not properly support IME. CAPsMAN might work for some of you, but it never supported 802.11K/V/R. The whole idea of kicking clients under a certain RSSI to force it to roam is just wireless harassment to my mobile devices.
RouterOS 7 brings 802.11ac wave 2 and presumably 802.11ax (WiFi 6) support, but it is unusable in its current form. The performance enhancements are there, but the configuration logic does not work at all.
Conclusion
In the following years, you can still expect MikroTik to keep its high cost effectiveness, and you will certainly get some free features after the final beta phase of RouterOS 7. But don’t expect a toy-grade thing to magically grow up becoming a true enterprise thing in one night. If MikroTik devices work for you, they will continue work for you; if they doesn’t work for you, don’t take the bet on RouterOS 7.