VPN: Protocols

Spread the love



Point-to-Point Protocol (PPP) is a layer 2 protocol used in point-to-point connections (dial-up, ISDN) to encapsulate any protocol at upper layers. It is an extension to HDLC.

A PPP packet has the following format:

where the most significant fields are:

  • leading and trailing flags: they are required for data framing;
  • Address and Control fields: they are inherited from HDLC but they are completely meaningless in point-to-point connections; they were kept to make it simple to upgrade the processing algorithms in HDLC devices;
  • Protocol field: it specifies the protocol at upper layer for the encapsulated packet.

‘01111110’ is the sequence delimiting the frame, but in order to send that sequence as data some stuffing rules are required. In PPP stuffing is at byte level: the escape sequence ‘01111101’ is inserted both when a byte equal to the delimiting byte appears in the data, and when a byte equal to the escape byte itself appears:

PPP byte stuffing.svg

Link Control Protocol (LCP) has the task of opening and closing PPP connections, negotiating some options (such as frame maximum length, authentication protocol).


An IP packet can not encapsulate directly a protocol at layer 3 or lower, because the ‘Protocol’ field in the IPv4 header can specify only protocols at upper layers (such as TCP, UDP, ICMP).

Generic Routing Encapsulation (GRE) is a protocol to encapsulate any protocol (including IP and other protocols at lower layers) into IP: the GRE header is inserted between the encapsulating IP header and the encapsulated packet, and the ‘Protocol’ field in the encapsulating IP header is set to 47.

The GRE header has the following format:

where the most significant fields are:

  • C, R, K, S flags (1 bit each one): they indicate the presence/absence of optional fields;
  • strict source routing (s) flag (1 bit): if set to 1, the packet is dropped if, when the source route list (‘Routing’ field) ends, the destination has not been reached yet (similar to TTL);
  • Recur field (3 bits): it specifies the allowed maximum number of additional encapsulations (currently not supported);
  • Version field (3 bits): it specifies the version of the GRE protocol (0 in this case);
  • Protocol type field (16 bits): it specifies the protocol of the encapsulated packet;

Enhanced GRE

The format of the Enhanced GRE header introduces the new ‘Acknowledgment number’ field:

5 8 13 16 32
C R K S s Recur A Flags Version (1) Protocol type
Key (payload length) Key (call ID)
Sequence number (optional)
Acknowledgment number (optional)

where the most significant fields are:

  • Key field (32 bits):
    • payload length (16 bits): it specifies the packet length excluding the GRE header (in bytes);
    • call ID (16 bits): it specifies the session ID for the packet;
  • Acknowledgment number field (32 bits): the sender puts in the packet the last packet sequence number it has received from the destination (cumulative ACK).
The new ‘Acknowledgment number’ field, in combination with the ‘Sequence number’ field, allows some additional mechanisms:

  • flow control: sliding windows avoid destination flooding, and they are moved when ACK packets are received;
  • out-of-order packet detection: Enhanced GRE was designed for PPP encapsulation, and PPP is a protocol for point-to-point connections where it is not expected packets arrive out-of-order → out-of-order packets are always discarded;
  • lost packet detection: when a timeout expires, that is no ACKs have been received, the packet is detected as lost, but the detected lost packets are not re-transmitted;
  • congestion control: when too many timeouts are detected, the packet transmission is slowed down not to congest the network.


L2TP original reference scenario: provider provisioned deployment mode.

Layer 2 Tunneling Protocol (L2TP) is a protocol to tunnel any layer 2 protocol (PPP in this discussion) into IP. L2TP was originally designed for provider provisioned access VPNs and it was standardized by IETF; later it was extended to customer provisioned access VPNs by simply implementing the LAC functionality into the user machine.

In the L2TP original reference scenario, a remote user wants to send a packet through a point-to-point (PPP) connection to an internal server inside the corporate network. An L2TP tunnel to the target L2TP Network Server (LNS) and an L2TP session inside the tunnel are required to emulate the PPP connection over the service provider network.

When the PPP frame arrives at the L2TP Access Concentrator (LAC), if an L2TP tunnel has not been established yet to the LNS, before opening an L2TP session the LAC has to establish a tunnel to the LNS: the LNS authenticates itself to the LAC by using a challenge-based mechanism similar to Challenge Handshake Authentication Protocol (CHAP), and a new control connection is created.

Each L2TP session uses two channels inside the tunnel:

  • data channel: to carry data messages, that is the PPP frames;
  • control channel: to exchange control messages, aimed to manage the communication (such as verifying that packets arrive, retransmitting lost packets, checking the right order of packets).
In contrast to data messages, control messages are exchanged in a reliable way: lost control messages are always retransmitted.

Multiple sessions may coexist within the same tunnel, sharing the same control connection, to distinguish multiple flows of PPP frames from multiple remote users: each tunnel is identified by a tunnel ID, each session is identified by a session ID.


Besides the authentication ensured during the tunnel establishing step, L2TP does not provide any security mechanism by itself: in fact it does not make sense to use mechanisms like encryption for L2TP packets travelling along the tunnel, because the service provider’s LAC still can look at the layer 2 frames → any security mechanism has to be implemented end-to-end, that is between the user and the final destination inside the corporate network.
Optionally it is possible to use IPsec through the tunnel: it provides a strong security, but it is complicated to be implemented.

A packet inside the L2TP tunnel includes several encapsulated headers:

MAC header IP header UDP header L2TP header PPP header PPP payload
PPP header

It identifies the point-to-point connection between the remote user and the internal server inside the corporate network.

L2TP header

It identifies the L2TP tunnel:

8 16 32
T L 0 S 0 O P 0 Version (2) Length
Tunnel ID Session ID
Ns Nr
Offset size Offset pad :::

where the most significant fields are:

  • T flag (1 bit): if set to 0 the packet is a data message, if set to 1 it is a control message;
  • Tunnel ID field (16 bits): it identifies the L2TP tunnel;
  • Session ID field (16 bits): it identifies the L2TP session, that is the data channel in the tunnel;
  • Ns field (16 bits): it contains the sequence number of the data/control message;
  • Nr field (16 bits): it contains the sequence number of the next control message to be received for a reliable control connection.
UDP header

Why is a layer 4 protocol such as UDP used to move layer 2 frames? This can be explained by considering firewalls over a general network: if a packet does not contain a layer 4 encapsulation, it is easier to get discarded by firewalls.
Another possible reason is that layer 4 can be accessed through sockets, while the operating system is in charge of layer 3. Since L2TP wants to be a solution against PPTP (proposed by operating system vendors), L2TP designers wanted to make it accessible to the applications and not to be tight to the will of the operating system vendors.
It is also possible to use a different layer 4 protocol (such as ATM, Frame Relay).

IP header

It contains the IP addresses of the tunnel endpoints.
In the original reference scenario, the IP addresses correspond to the LAC and the LNS.


PPTP original reference scenario: customer provisioned deployment mode.

Point-to-Point Tunneling Protocol (PPTP) is a protocol to tunnel the PPP protocol into IP. PPTP was originally designed for customer provisioned access VPNs and it was developed by major operating system vendors; later it was extended to provider provisioned access VPNs by introducing a PPTP Access Concentrator (PAC) with functionality similar to the LAC one.

The PPTP original reference scenario differentiates itself from the L2TP one for the fact that the PPTP tunnel is established between the remote user itself and the PPTP Network Server (PNS).

In contrast to L2TP, PPTP provides weak (optional) security mechanisms: the standard covers the usage of specific Microsoft-proprietary security protocols such as MPPE for encryption and MS CHAP for authentication, so there is not any negotiation of protocols.

Data channel

The encapsulated PPP frames go through the data channel:

IP header GRE header PPP header PPP payload

PPTP uses the Enhanced GRE protocol to encapsulate the PPP frames into IP. The PPP payload can be optionally encrypted by MPPE.

Control channel

The PPTP control messages go through the control channel:

IP header TCP header PPTP control message

Control messages are required for tunnel data session setup, management and tear-down, and they are sent to the well-known TCP port 1723 of the PNS.


The functionality of the Internet Protocol Security (IPsec) protocol suite in IPv4 is very similar to the one in IPv6.

The main difference is that in IPv6 IPsec is an extension header included in the standard, while in IPv4 it is a new additional protocol encapsulated into IP (for AH the ‘Protocol’ field is set to 51, for ESP it is set to 50):

Authentication Header (AH).
IPv4 header AH header TCP/UDP header payload
authenticated data
Encapsulating Security Payload (ESP).
IPv4 header ESP header
(for encryption)
TCP/UDP header payload ESP header
encrypted data
authenticated data


Secure Sockets Layer (SSL), today called Transport Layer Security (TLS), is a cryptographic protocol which is designed to provide communication security between a client and a server:

  1. the client opens a TCP connection on port 443 with the server, and sends a challenge for server authentication;
  2. the server sends to the client a Hello message containing its certificate and the response to the challenge encrypted by the server’s private key;
  3. the client verifies the certificate by looking into a list of certificates given by a trusty certification authority, then it decrypts the response to the challenge by using the server’s public key;
  4. the client decides a private key for the secure communication and sends it to the server by encrypting it through the server’s public key → from this point on all record messages will be encrypted by using that private key (which should be periodically renegotiated);
  5. often the server asks the user to type its username and password on a web page for client authentication (at application layer).

Leave a Reply

Your email address will not be published. Required fields are marked *