Traditional point-to-point site-to-site VPN protocols require extensive setup in certain use cases. For example, if you want shortcuts between branch offices rather than let every packet go through the HQ, then you need to set up a cartesian product of tunnels by hand, which is time-consuming and error prone. So, people want something better, something easy to set up and maintain, and dynamic enough. While there are already a lot dynamic point-to-multipoint or full-mesh site-to-site VPN implementations (e.g. Tinc VPN, ZeroTier) on Linux, you don’t have many choices on these commercial black boxes.

Cisco DMVPN (Cisco Dynamic Multipoint VPN) is one solution to this. Huawei also had their DMVPN-compatible solution called DSVPN (Dynamic Smart VPN). Since the protocol is largely compatible, I’ll just reference it as DMVPN.

On the DMVPN Protocol

Strictly speaking, DMVPN is not one protocol, but is stitched from 4 protocols:

• Multipoint GRE (mGRE)
• Next Hop Resolution Protocol (NHRP) inside mGRE
• (Optional) IPSec encryption outside mGRE
• (Optional) another routing protocol, typically OSPF

Multipoint GRE

Multipoint GRE is just plain GRE but with a mapping of inside destination IPs to outside destination IPs. It doesn’t have a single remote tunnel endpoint IP, instead we need to have some external protocol to fill its IP mapping (like the ARP protocol to the Ethernet).

Next Hop Resolution Protocol

NHRP is a protocol which can automatically fill the mGRE tunnel’s IP mapping, it’s the ARP for mGRE. It works in a straightforward way: some (usually 1 or 2) devices are designated as hubs (NHS, Next Hop Server), while others are all spokes. Hubs does not have any mapping information while NHRP daemon is started, and spokes have statically configured mapping of all hubs’ IP mapping information. When spokes’ NHRP daemon is started, the daemon sends registration packets to the hubs, announcing its existence, its own external and internal IPs and optionally some other routes. When spokes try to connect to each other, they query the packet destination’s external IP addresses with the hub and try to directly connect them with GRE. If the other end is not reachable, packet is routed to the hub then to the destination.

IPSec

IPSec is a protocol mainly used to transparently encrypt IP packets. I’m not going to talk about it since it is not the point of this article.

Routing Protocol

NHRP is not suitable for distributing a lot routes with it. For mid-sized or larger networks, we usually only use NHRP to resolve tunnel endpoint IPs, and use another routing protocol (RIP, OSPF or BGP) for routing the client networks.

Security

Since DMVPN can run without IPSec, if the NHRP daemon implementation is flawed, there is possibility that one plain unprotected tunnel can register successfully thus resulting in a downgrade attack.

Lab Setup

There are already example setups on the wild Internet using a Cisco router as the hub and other Cisco/Huawei devices as the spokes (see the references section). Today we try using Huawei USG firewall as the hub. All configurations are provided in 2 parts, one for plaintext mGRE/NHRP, and one for adding IPSec protection to the existing tunnel.

Security warnings first:

• DO NOT use the plaintext config in production! If you want to capture NHRP packets in a lab environment, use this config. It is not secure for production traffic.
• DO NOT blindly copy the IPSec ciphers for production. Use strong GCM ciphers and enable PFS if your devices support them.

And some notes on cross-vendor compatibility:

• Huawei devices send L3MTU as OSPF MTU, and other vendors send L2MTU, so we ignore MTU by default in OSPF config
• The maximum MTU of Huawei USG firewalls is 1500, so we just use 1500 for every device

Hub – Huawei USG

Test gear: Huawei USG6320, software version V500R005C00SPC200

If you config DSVPN from the USG web interface, here is a quick lookup table for you:

Huawei (English)	Huawei (中文)	Other Vendors
Reverse Routes from Branches	总部从分支学习反向路由	send static routes with NHRP
Mutual Route Learning	分支节点相互学习路由	no shortcut
Route Aggregation to HQ	分支节点路由汇聚到总部	shortcut

Plain Text

With IPSec

Security

Even if the DMVPN tunnel is configured to use IPSec, a cleartext NHRP request will still result in a NHRP neighbor entry. You are not able to send any packets through the tunnel, though. (This might lead to buffer overflow vulnerabilities from unsolicited NHRP peers.)

Caveats

• NHRP redirect & multicast config is only available in CLI
• OSPF network type can only be broadcast (non-configurable)
• OSPF MTU ignore is not user-configurable
• MTU is not user-configurable (1500 hardcoded)
• Huawei never officially said DSVPN is compatible with DMVPN, so the compatibility is not guaranteed
• If an IP address of the same subnet as the one configured on the DMVPN interface doesn’t have a NHRP entry, an USG spoke will send it to the HUB

Huawei extensions to NHRP protocol (use undo nhrp identity enable to disable):

• 1024 (0x0400): 0-byte padding
• 1025 (0x0401): device identity (string, length 1-31 bytes, default value is hostname)
• 1026 (0x0402): 0-byte padding
• 1027 (0x0403): device serial number (string, 20 bytes)

BTW, one free advice for you: if you have two DSVPN instances mapped to one OSPF process from the CLI, do not try to edit DSVPN config from the web; this will lead to unexpected results.

Spoke 1 – Cisco IOS XE

Test gear: Cisco CSR1000v, software IOS XE 16.9.4

Plain Text

With IPSec

Security

Not tested.

Caveats

NHRP network-id or NHRP domain is just a process ID; it is only used locally, but if you don’t configure it, NHRP process will be down forever.

The tunnel’s L2MTU will be hardcoded to 9976 and you cannot configure it. (Note that the transport MTU will be the correct value i.e. 1476, and you can still manually set the L3MTU.) So we explicitly set OSPF to ignore MTU mismatches.

On older devices, the config line

should be replaced with the old grammar

Spoke 2 – VyOS

Test gear: VyOS 1.2.6 (with patch) / VyOS 1.3 nightly (official)

Plain Text

With IPSec

Security

Current OpenNHRP’s IPSec configuration works by first generating a IPSec config without local/remote IPs to StrongSwan, then after a neighbor is discovered, use swanctl with local/remote IP override to create a dynamic IPSec peer. If the IPSec tunnel is down for some reason, it won’t be restarted, resulting in packets being sent in cleartext. Due to this reason, I do not recommend using VyOS for DMVPN in production networks.

Caveats

OpenNHRP implementation from VyOS 1.2.6 is not compatible with Huawei’s NHRP implementation. If you really want to use VyOS together with Huawei devices, you need to apply this patch. First we need to compile it. On a Debian 8 (jessie):

Then install it onto VyOS:

The patch will be available for the current running VyOS image.

Spoke 3 – Huawei AR Router

Test gear: Huawei AR6121-S, VRP 5.170 (V300R019C10SPC300)

Plain Text

With IPSec

Security

Not tested.

Caveats

For an AR router to connect to other routers successfully with IPSec, ipsec authentication sha2 compatible enable might be required.

If IPSec tunnel mode is required (e.g. spoke is behind NAT), manually switch to tunnel mode:

Unlike an USG, an AR does not support encapsulation mode auto negotiation.

If an IP address of the same subnet as the one configured on the DMVPN interface doesn’t have a NHRP entry, an AR spoke will drop it, which would cause some problems in certain topologies.

Notes

References:

• Introduction to Multipoint GRE and NHRP
• Establishing a DSVPN over the IPSec tunnel between AR and Cisco
• Dynamic Multipoint VPN Configuration Guide, Cisco IOS Release 15M&T
• DSVPN
• IP Addressing: NHRP Configuration Guide, Cisco IOS XE Release 3S
• DMVPN for R&S CCIE Candidates – Johnny Bass – BRKCCIE-3003
• 配置IPSec安全提议