Updated April 4, 2023
Introduction to Encapsulation Security Payload
Encapsulating Security Payload is an IPsec Protocol that provides confidentiality and integrity features. The encapsulating security payload protocol also defines a new header to be inserted into the IP packets. ESP (Encapsulating Security Payload) processing also includes the transformation of the protected data into its encrypted format, i.e. unreadable format. Under normal circumstances, the encapsulating security payload will be inside the AH (authentication header). That means first; the encryption process takes place after the authentication is done. ESP is based on symmetric-key cryptography techniques. ESP can be used in isolation or can be combined with an Authentication header. In this topic, we are going to learn about Encapsulation Security Payload.
How encapsulating security payload works?
On the receipt of IP packets that IPsec processed, the receiver processes the authentication header first, if it is present. Based on the output result, the receiver identifies whether the content of the packet is correct, i.e. original, or it has been altered during transmission. If the packet’s content is original, the receiver extracts the key and algorithms associated with the encapsulating security payload and then decrypts the contents.
Encapsulating Security Payload Packet
The encapsulating security payload packet consists of seven fields in which four fields are of fixed length, and three fields are of variable length. Let’s discuss these fields one by one.
- SPI: It stands for Security Parameter Index consists of 32-bit fields. It can be used in combination with source and destination address as well as the IPsec protocol used to uniquely identify the SA (Security Association) for the traffic to which a datagram belongs.
- Sequence number: It is a 32-bit field which is used to prevent replay attack.
- Payload data: It is a variable-length field containing the transport-layer segment or IP packet, protected by encryption.
- Padding: It contains padding s bits, if any. These are used by the encryption algorithm or for aligning the padding length field so that it begins at the third byte within the 4-byte word.
- Padding length: It is an 8-bit field that specifies the number of padding bytes in the immediately preceding field.
- Next header: It is an 8-bit field that is used to identify the type of encapsulated data in the payload. For example, a value of 6in this field indicates that the payload contains TCP data.
- Authentication data: This field is a variable-length field containing authentication data called ICV (Integrity Check Value) for the diagram. This is calculated over the length of the encapsulating security payload packet minus the authentication data field.
Modes of Operation
There are two modes in which encapsulating security payload is done: transport mode and another is tunnel mode. Let’s discuss them one by one in detail.
1. ESP transport mode
Transport mode encapsulating security payload is used to encrypt and optionally authentic the data carried by the IP. Here the encapsulating security payload is inserted into the IP packet immediately before the transport layer header, and an encapsulating security payload trailer is added after the IP packet. If authentication is also used, the encapsulating security payload authentication data field is added after the encapsulating security trailer. The entire transport layer segment and the encapsulating security payload trailer are encrypted. The entire ciphertext, along with encapsulating security payload header, is authenticated.
- On a sender’s side, the block of data containing the ESP trailer and entire transport layer segment is encrypted, and then the plain text is replaced with its corresponding ciphertext to form the IP packet. Authentication is appended if selected. This packet is now ready for transmission.
- The packet is routed to the destination address. The intermediate routers need to look at the IP header and any IP extension header, but not at the ciphertext.
- At the receiver’s end, the IP header plus any plain text IP extension headers are examined. The remaining portion of the packet is then decrypted to retrieve the original plain text transport layer segment.
2. ESP tunnel mode
The tunnel mode encapsulating security payload is used to encrypt the entire IP packet. Here, the ESP header is fixed to the packet, and then the packet and the ESP trailer are encrypted. The IP header contains the destination address as well as intermediate routing information. Therefore this packet cannot be transmitted as it is. Otherwise, the delivery of the packet would be impossible. Therefore a new packet is added, which contains sufficient information for the routing.
- On the sender’s side, the sender prepares the Inner IP packet with the destination address as the internal destination. This packet is pre-fixed with an ESP header, and then this packet and ESP trailer are encrypted and then authentication data is added. A new IP header is added at the start of this block. This forms the outer IP packet.
- The outer packet is routed to the destination firewall. Each intermediate router needs to check and process the outer IP header along with the other outer IP extension headers. If need not know about the ciphertext.
- The destination firewall processes the outer IP header plus any extension headers and recovers the plain text from the ciphertext at the receiver site. The packet is then sent to the actual destination host.
Recommended Articles
This is a guide to Encapsulation Security Payload. Here we discuss the encapsulating security payload with its working and two modes of operation, i.e. tunnel mode and transport mode. You may also have a look at the following articles to learn more –
