Optimising VPN Algorithms and Ciphers for FortiGate Firewalls

A practical FortiGate IPsec guide for choosing IKEv2, AES-GCM, DH groups, lifetimes, tunnel behaviour, and optional hybrid PQC on FortiOS 7.6.1 and later.

OuchItBurnsWhenIP · Published 25 Nov 2025 · Updated 17 Oct 2026

IPsec cipher profiles, key exchange and tunnel settings for route-based FortiGate-to-FortiGate VPNs on FortiOS 7.4 and later. The examples use IKEv2, AES-GCM and ECP groups. Optional hybrid post-quantum key exchange requires FortiOS 7.6.1 or later and compatible peers.

These examples target non-regulated environments; apply any mandated cryptographic policy separately. For remote access, see FortiClient dial-up IPsec VPN on FortiGate.

Cipher profiles

Both profiles use IKEv2 and enable Phase 2 perfect forward secrecy (PFS).

Setting	Option 1: AES-256 / DH21	Option 2: AES-128 / DH19
Phase 1 proposal	`aes256gcm-prfsha384`	`aes128gcm-prfsha256`
DH group	21 — ECP P-521	19 — ECP P-256
Phase 1 lifetime	86400 seconds (24 hours)	86400 seconds (24 hours)
Phase 2 proposal	`aes256gcm`	`aes128gcm`
PFS group	21	19
Phase 2 lifetime	43200 seconds (12 hours)	43200 seconds (12 hours)

Option 1 uses larger keys and a more computationally expensive DH group. Option 2 uses a key exchange with approximately 128-bit classical security. DH20 (ECP P-384) provides approximately 192-bit classical security and is another option for either profile, subject to peer support and policy.

IKEv2 and AES-GCM

IKEv2 needs fewer exchanges, supports EAP authentication and is required for RFC 9370 additional key exchanges. Use IKEv1 only where required by a legacy peer.

AES-GCM provides authenticated encryption: encryption and integrity are combined, without a separate HMAC proposal. AES-CBC requires a separate integrity algorithm and remains an interoperability option for peers that cannot use GCM.

DH groups and PFS

Match key exchange strength to the cipher, peer capabilities and available processing capacity. The strength estimates below refer to classical cryptanalysis.

Group	Name	Approximate strength	Deployment considerations
1	MODP 768	Weak	Do not use
2	MODP 1024	Weak	Do not use
5	MODP 1536	Weak	Do not use
14	MODP 2048	Below 128-bit	Legacy fallback
15	MODP 3072	128-bit	Alternative where ECP is unavailable
16	MODP 4096	Above 128-bit	Higher computational cost than ECP
19	ECP P-256	128-bit	Used in Option 2
20	ECP P-384	192-bit	Intermediate strength between DH19 and DH21
21	ECP P-521	256-bit	Used in Option 1
27	BrainpoolP224r1	Below 128-bit	Avoid for new deployments
28	BrainpoolP256r1	128-bit	Check peer support and policy requirements
29	BrainpoolP384r1	192-bit	Check peer support and policy requirements
30	BrainpoolP512r1	256-bit	Check peer support and policy requirements
31	Curve25519	128-bit	Verify support and negotiation on both peers
32	Curve448	224-bit	Verify support and negotiation on both peers

ECP groups 19, 20 and 21 are defined in RFC 5903; Curve25519 and Curve448 are defined in RFC 8031. Brainpool groups 27–30 have different strengths and should not be treated as interchangeable.

Phase 1 and Phase 2 DH groups do not have to match. The examples use matching groups to simplify configuration. PFS adds a fresh DH exchange when rekeying child SAs, improving key separation at the cost of additional computation.

Hardware and throughput

NP6 does not offload AES-GCM. On platforms with CP9, the content processor can accelerate GCM encryption; this still differs from the NP6 fast path. NP6XLite (SoC4) and NP7 support AES-GCM network-processor offload. See Fortinet’s GCM hardware guidance and processor capabilities. On NP6 appliances, AES-CBC with SHA-256 can preserve hardware acceleration. Check npu_flag in diagnose vpn tunnel list while traffic is flowing: 03 indicates offload in both directions. See Fortinet's IPsec offload guide.

For FortiGate-VM, throughput depends on vCPU allocation, CPU cryptographic acceleration and hypervisor overcommit. Compare cipher performance under the expected traffic load.

Lifetimes and tunnel settings

The examples use a 24-hour Phase 1 lifetime and a 12-hour Phase 2 lifetime. Keep Phase 2 at or below Phase 1 for these profiles; shorten lifetimes where policy requires it, accounting for the additional rekey load.

Dead peer detection

Keep DPD enabled. on-demand checks peer availability when traffic needs to be sent; on-idle checks idle peers and can detect failures on quiet links.

Automatic negotiation

For static site-to-site peers, enabling auto-negotiate in both phases brings up the tunnel without waiting for user traffic. This suits branch-to-hub connections and static or dynamic routing over IPsec.

Routing

Use phase1-interface and phase2-interface for route-based tunnels. set add-route enable can install routes on simple spokes; explicit static routes or BGP provide more control in larger deployments. BGP supports route summarisation and dynamic failover over the IPsec interfaces.

Hybrid PQC on FortiOS 7.6.1+

Hybrid key exchange combines classical DH with additional key exchanges. Both FortiGate peers must support the required IKEv2 extensions and KEM parameters.

RFC 9370 defines multiple IKEv2 key exchanges. FIPS 203 defines the ML-KEM parameter sets used by FortiOS:

CLI group	Algorithm
`35`	ML-KEM-512
`36`	ML-KEM-768
`37`	ML-KEM-1024

FortiOS supports up to seven additional exchange rounds (ADDKE1–ADDKE7), with up to three KE groups per round. The example below uses one additional exchange, ML-KEM-768, alongside DH20, AES-256-GCM and PRF SHA-384. DH21 and ML-KEM-1024 are alternatives with higher computational cost.

Configure additional KE in both Phase 1 and Phase 2, and enable childless-ike in Phase 1.

config vpn ipsec phase1-interface
    edit "vpn-pqc"
        set interface "wan1"
        set remote-gw <REMOTE_IP>
        set ike-version 2
        set authmethod psk
        set psksecret <YOUR_PSK_HERE>
        set proposal aes256gcm-prfsha384
        set dhgrp 20
        set addke1 36
        set childless-ike enable
        set keylife 86400
        set dpd on-demand
        set auto-negotiate enable
    next
end

config vpn ipsec phase2-interface
    edit "vpn-pqc-p2"
        set phase1name "vpn-pqc"
        set proposal aes256gcm
        set pfs enable
        set dhgrp 20
        set addke1 36
        set keylifeseconds 43200
        set auto-negotiate enable
    next
end

This example is for FortiGate-to-FortiGate tunnels. The FortiClient 7.4.x workflow covered in the remote-access guide uses classical IKEv2; its documented client settings do not expose ADDKE or ML-KEM controls.

Peer compatibility

Both peers need a common proposal and DH group. Retain IKEv2 and AES-GCM where supported. AES-CBC with SHA-256 provides an alternative for older peers or NP6 offload. Weak DH groups remain a security limitation even when needed for interoperability.

FortiClient remote access also requires a dial-up responder, mode-config, client address assignment, user-group authentication and split-tunnel settings. Those are covered in the FortiClient guide.

Site-to-site configuration

Replace wan1, <REMOTE_IP> and <YOUR_PSK_HERE> with deployment values. Configure the corresponding peer and routing for the required networks.

Option 1: AES-256 / DH21

Phase 1

config vpn ipsec phase1-interface
    edit "vpn-option1"
        set interface "wan1"
        set remote-gw <REMOTE_IP>
        set ike-version 2
        set authmethod psk
        set psksecret <YOUR_PSK_HERE>
        set proposal aes256gcm-prfsha384
        set dhgrp 21
        set keylife 86400
        set dpd on-demand
        set auto-negotiate enable
    next
end

Phase 2

config vpn ipsec phase2-interface
    edit "vpn-option1-p2"
        set phase1name "vpn-option1"
        set proposal aes256gcm
        set pfs enable
        set dhgrp 21
        set keylifeseconds 43200
        set auto-negotiate enable
    next
end

Option 2: AES-128 / DH19

Phase 1

config vpn ipsec phase1-interface
    edit "vpn-option2"
        set interface "wan1"
        set remote-gw <REMOTE_IP>
        set ike-version 2
        set authmethod psk
        set psksecret <YOUR_PSK_HERE>
        set proposal aes128gcm-prfsha256
        set dhgrp 19
        set keylife 86400
        set dpd on-demand
        set auto-negotiate enable
    next
end

Phase 2

config vpn ipsec phase2-interface
    edit "vpn-option2-p2"
        set phase1name "vpn-option2"
        set proposal aes128gcm
        set pfs enable
        set dhgrp 19
        set keylifeseconds 43200
        set auto-negotiate enable
    next
end

References

RFC 7296: IKEv2 protocol
RFC 8247: IKEv2 algorithm implementation guidance
RFC 8221: ESP and AH algorithms
RFC 5903: ECP groups 19, 20 and 21
RFC 8031: Curve25519 and Curve448 for IKEv2
RFC 9370: multiple key exchanges in IKEv2
NIST SP 800-57 Part 1 Rev. 5: key management and security strength
FIPS 203: ML-KEM parameter sets
FortiOS 7.6.6 PQC for IPsec key exchange: FortiGate hybrid PQC configuration