The transport layer TCP in a network stack provides end-to-end reliable delivery to the application layer. A reliable connection implies that data will be delivered in-order, error-free and without loss or data duplication. Developers will occasionally face situations where TCP connections do not work as expected. Rather than just blaming “the network” for some unspecified fault and hoping that the problem will resolve itself, a developer with a clear understanding of the low-level details of TCP connections will be able to accurately diagnose the trouble. In this article, we delve into the three phases when a client establishes a TCP connection with a server: connection setup, data transfer and tear down. In each of these phases, the client and server exchange messages, and the TCP connection transitions through different states. We will explain the conditions that define each of these TCP states, how the connection transitions from one state to another, and the implications for debugging when issues arise during connection setup.
Internet usage today is dominated by web browsers and mobile apps accessing online content. HTTP [1] is the underlying protocol used by browsers, and most mobile apps also make use of REST (Representational State Transfer)/HTTP as well as other protocols. All such communications make use of TCP [2] as the underlying protocol. TCP provides end-to-end reliable delivery between applications on two machines (end points of communications), located anywhere in the world. For example, when a user types google.com to search, the web browser establishes a TCP connection between the user’s browser and one of Google’s servers.
At its core, the Internet only provides the IP layer, which iTCP at the transport layer is implemented only in end-systems. Intermediate internet routers do not implement TCP and are “unaware” of TCP data packets unless they make use of Network Address Port Translation (NAPT). Reliable delivery of TCP implies that whenever it delivers data to the application, the delivered data is uncorrupted, in-order, without any packet loss or duplicates. A common belief is that TCP reliability also implies guaranteed delivery – it is important to note that this is a misconception. For example, when an application (browser) hands over the data to the transport layer at the sending side and the transport layer acknowledges its successful receipt (e.g., TCP send() returns success to the application), it only means that if the transport layer successfully delivers the received data to the other end of the communication, it will be in-order, error-free and without any loss or duplication. If TCP is unable to deliver the data to the other end (e.g., if the underlying network connecting the two machines breaks down and there is no alternative route available), this does not violate the definition of TCP reliability. TCP is also characterized as a streaming service, which implies that it treats the entire input data from the sender as a stream of bytes [3].
TCP communication involves three phases: (1) connection setup, (2) data transfer, and (3) tear down. For each TCP connection, the two machines communicating via this connection independently maintain information about the present state of this connection from their own perspective (as explained below).
Each TCP connection can be in one of the 11 possible states listed in Table 1. When the TCP connection operates normally (i.e., there is no packet loss or corruption), the machines at the two ends correctly update the state of the TCP connection, and an application developer or someone maintaining network operations does not need to be aware of the state-level details. However, if the network encounters abnormalities such as link breakdown or intermediate router crashes that result in packet loss and/or retransmission in Phases 1 or 31 , an understanding of these details is critical to diagnose the problem and resolve it.
Table 1: TCP Connection state in various phase of communication
The key purpose of this article is to provide this understanding. In the experience of the authors, most developers are only familiar with the network programming interfaces (known as socket() programming) provided by the TCP stack for their chosen programming language. When such developers encounter abnormalities as noted above, they are often unable to resolve such issues. A naïve yet common “solution” is to close (discard) the current connection and start afresh with a new connection, in the hope that the issue does not recur. Apart from being wasteful (in terms of bandwidth and computing resources) in the near-term, this approach leaves the problem unresolved and can prove costly for enterprises in the long term. This article will focus on abnormalities that occur during Phase 1, and we will defer the discussion related to Phase 2 and Phase 3 to the next article.
1Such abnormalities have limited impact in Phase 2. If they occur, packets will be retransmitted and acknowledged at some point in time later, but the TCP connection state remains in the ESTABLISHED throughout this phase
As noted earlier, each of the two machines communicating via a TCP connection maintains information about the state of the connection from the perspective of that machine. The rules that describe how and when these states are updated (independently by the two machines) is best explained as a Finite State Machine (FSM) [4], shown in Figure 1. Depending on which machine is initiating a new connection (client) vs. listening for new connections (server), sending or receiving data over a connection (either the client or the server), or initiating the closure of a connection (either the client or the server), the two machines may follow different transitions (represented as arrows) in the FSM to update the state of the connection. The states of the client while initiating the connection (known as active open) are shown in blue, whereas the server states while waiting for a new connection to be opened (known as passive open) are shown in orange. Similarly, the active close states (for the side that initiates connection closure) are shown in red and the passive close states (for the other side) are shown in magenta. The CLOSED state (duplicated at the bottom to avoid cluttering the diagram) is shown in black, indicating no TCP connection state. While data is being transmitted, both machines will maintain the TCP connection in the ESTABLISHED state shown in green.
Figure 1: TCP State transition
Each machine maintains the TCP connection in its present state. When an event occurs at a machine, it transitions to an appropriate state as indicated by arrows. (In some cases, the next state may be the same state and this is indicated by a loop in the FSM). Each arrow is accompanied by two lines of text. The top line indicates the event that must occur for this transition to be triggered, and the bottom line indicates the action that this machine will take in response to the triggering event (in addition to following the arrow to the next state). This action could involve sending a message to the machine at the other end of the TCP connection, and such messages allow the two machines to maintain a coherent view of the TCP connection. For example, when a client application initiates a connection with the server by creating a network socket and invoking connect() (socket call to connect to server), this triggers an “appl: active open” event on the client machine. In response to this event, the state of this connection on the client machine moves from CLOSED to SYN_SENT, and the client machine additionally takes the action “send: SYN” (see the Figure 1) that sends a message to the server
A machine can run many processes concurrently, and each process can have many TCP connections. The state of every TCP connection in a machine can be viewed using the network utility command netstat [5][6] , as shown in Figure 2. (Note that netstat never shows TCP connections in the CLOSED state, since no system resources are associated with such TCP connections.) Each TCP connection is identified by a tuple <source IP, source port, destination IP, destination port>. When netstat is run, TCP connections will typically be in one of the three stable states LISTEN, ESTABLISHED, or TIME_WAIT (Figure 2). Under normal TCP communications, the other seven states shown by netstat are transient i.e. the machine rapidly updates TCP connection states to one of the stable states. Hence, if we observe a TCP connection remaining in one of these transient states (for more than a second), it indicates the occurrence of some abnormality, which should be carefully investigated and resolved (e.g. a network or application issue, or an improperly configured firewall that is blocking incoming or outgoing TCP messages that it should not).
Figure 2: An example of states of all TCP connections
We will now explore some of these abnormal conditions that enable TCP connections to exist in one of the transient states for long durations during Phase 1 and Phase 3. This exploration is supplemented with experiential learning exercises that will enable the reader to observe these states. A typical setup (consisting of two systems) for conducting such experiments is shown in Figure 3.
Figure 3: Basic setup of conducting TCP learning exercises.
The section of the TCP state transition diagram corresponding to connection setup is shown in Figure 4. A client can connect to a server only when the server is running and waiting to accept connections i.e. in the passive open or LISTEN state. To move to this state, the server application must bind to a chosen and well defined port by invoking the socket calls bind() followed by listen(). This moves the state of the TCP connection on the server to the desired LISTEN state. To setup the TCP connection, the client application must initiate a 3-way handshake, starting by creating a network socket and invoking the connect() socket call (which moves the state of this connection on the client machine from CLOSED to SYN_SENT). If the server machine receives the resulting SYN from the client (first part of the handshake), it moves the connection state from LISTEN to SYN_RECV and transmits a SYN|ACK to the client (the second part of the handshake). Again, if the client machine receives this SYN|ACK, it moves the connection to the ESTABLISHED state and transmits an ACK back to the server (the third part of the handshake). If the server machine receives this ACK, it also moves the connection state to ESTABLISHED, at which point the TCP connection enters Phase 2. (A TCP connection typically spends most of its time in this state.)
Figure 4: TCP state transition during connection setup.
In the preceding paragraph, we have assumed that the two machines are able to communicate successfully over the TCP connection. However, in the presence of abnormalities, the 3-way handshake may not successfully complete. For instance, a firewall between the client and the server may drop either the SYN or ACK packet from the client to the server, or the SYN|ACK response packet from the server to the client. Let us examine each of these situations.
Suppose the state of a TCP connection on the client machine is SYN_SENT i.e., the client application has sent the SYN message to the server and is awaiting the SYN|ACK response. First, suppose that the SYN message is dropped by the firewall and the client application is not informed of this packet drop (as is typical of a firewall). Since the server does not receive the SYN request, it will continue to maintain the TCP connection in the (stable) LISTEN state. In contrast, the client will maintain the TCP connection in the transient SYN_SENT state. Second, consider the situation where the server successfully receives the SYN message and responds with SYN|ACK (while moving the TCP connection to the SYN_RECV state), but this response is dropped by either the server-side or client-side firewall. In this situation, the TCP connection state is transient on both the client and server machines (SYN_SENT and SYN_RECV respectively). Third, suppose the client machine receives the SYN|ACK response but the ACK it sends the server (while moving the TCP connection to the stable ESTABLISHED state) is dropped. Here, the TCP connection on the server remains in the transient state SYN_RECV.
Each of these three scenarios can occur in an operational network. When the connection setup does not complete successfully, it will simply timeout and throw an error to the client application. Note that it is possible to distinguish between each of these cases by observing the TCP connection state on both the server and the client, but developers and network operators are generally unfamiliar with the fine-grained state of TCP connections, and they rarely know how to make inferences based on these observations that can lead to resolving the network/firewall issues.
We now describe hands-on exercises to develop this understanding. In addition to netstat, we will use two further tools: a network utility called netcat (nc) [7][8] that can be used to create both a TCP client and a TCP server (for the latter, run with the option –l), and a utility to implement firewalls functionality, called iptables [9]. The first such scenario is shown in Figure 5 (client-side, where the command prompt is set as C_m) and Figure 6 (server-side, where the command prompt is set as S_m). For clarity, command windows are split into an upper and a lower panel, and each command is preceded by date to show the order in which the commands were issued. (Times are shown in IST on the client and in PST on the server. Two different time zones reflect real life situations, where two machines can be geographically apart and operate in different time zones.)
Figure 5: Client connection state with FW dropping SYN
The first two commands are in the lower panel of Figure 5 (issued at 17:03:18 IST and 17:03:37 IST). The first command uses iptables to implement a client-side firewall that is configured to drop all outgoing SYN packets with the destination port 5555. The second command uses netstat to observe the client-side state of the TCP connection on port 5555. Since no such connection has been established yet, only the output of the date command is observed. The third command is in the upper panel of Figure 5 (issued at 17:03:41 IST), which runs a client application initiating a TCP connection to port 5555 on the server with IP address 10.211.55.10. The fourth and fifth commands are two repetitions of the second command, shown in the lower panel of Figure 5 (issued 15 seconds apart at 17:03:45 IST and 17:04:00 IST). Since the connection is now established, netstat produces output showing that the connection has remained in the transient state SYN_SENT for 15 seconds. An experienced developer would immediately note that this is unexpected and would investigate the firewall configuration (ideally at both the client and server side) to diagnose the problem. The commands for the server side are shown in Figure 6. The second command in the upper panel of Figure 6 enables the server to listen for TCP connections on port 5555. As the subsequent netstat commands in the lower panel of Figure 6 show, the server remains in the LISTEN state since no SYN is received.
Figure 6: Server connection setup state
When the firewall is implemented on the server side and is configured to drop SYN|ACK responses from the server to the client, the TCP connection state maintained by both the client and the server (SYN_SENT and SYN_RECV respectively) are transient, as shown in Figure 7 (client-side) and Figure 8 (server-side). We note that when a server is maintaining the state of a TCP connection in the SYN_RECV state and it does not receive the third part of the 3-way handshake message (i.e., the ACK) within a few seconds, it may consider this as a SYN flooding attack and forcibly close the connection. (A detailed description of SYN flooding attacks and possible solutions are given in [10].) For this reason, one may not actually witness the SYN_RECV state while carrying out this exercise. See Exercise 2 for further details.
Figure 7: Client connection state unaware of Firewall
Figure 8: Server connection state with FW dropping SYN|ACK pkts
As we have noted earlier, Phase 2 begins when the 3-way connection setup is completed successfully (i.e., the connection state maintained by both the client and the server is ESTABLISHED). When the application at one of the end points decides that no further data needs to be sent, it initiates the teardown phase (Phase 3) by sending a FIN message. The application at the other end point may decide that it has further data to send even after it receives this FIN message, and in this case it will continue to transmit data from the CLOSE_WAIT state. (It will enter teardown phase only after it has completed the data transfer). This situation is commonly known as TCP half-close, because one side has initiated the close as it is not sending any more data, but other side continues to send data. As a typical example, consider the situation where a client application sends a URL corresponding to a large file (e.g., a 10 MB image) to the server. After transmitting this URL data, the client application may initiate connection closure. However, the server will start the connection closure only after the entire image has been transmitted. Due to complex behavioral requirements such as this, a TCP connection must go through several intermediate states before it finally reaches the CLOSED state.
The successful transfer of data (the string hello) is shown in Figure 9 (server-side) and Figure 10 (client-side). The first command (issued at 00:25:34 PST) is shown in the upper panel of Figure 9, and runs a server application listening on port 5555. (The output of the date command is printed, and the string hello is printed later as explained below.) The second command (issued at 00:25:39 PST) is shown in the lower panel of Figure 9, and shows that the server is indeed listening on port 5555. The third command (issued at 13:55:54 IST) shown in the upper panel of Figure 10, initiates the TCP connection, and the fourth (issued at 00:26:01 PST) and fifth (issued at 13:56:11 IST) commands, respectively in the lower panel of Figure 9 and the lower panel of Figure 10, show that the connection is successfully established at both end points. Next, the user types the string hello in the nc program on the client side, shown in the upper panel of Figure 10. This string is received by the nc program on the server side, shown in the upper panel of Figure 9. Finally, the user initiates connection closure by typing Ctrl-C on the client side2 . The last command (issued at 00:26:32 PST) shown in the lower panel of Figure 9 shows that the TCP connection is closed on the server-side (i.e., there is no output by netstat). We now focus on the last command in the lower panel of Figure 10 (issued at 13:56:26 IST), which shows that the TCP connection state on the client-side is TIME_WAIT.
Figure 9: TCP States on server side
Figure 10: TCP States for Client
When Ctrl-C is typed on the client-side to terminate the application (nc program), the TCP connection state on the client side moves from ESTABLISHED to FIN_WAIT_1, and the client sends the server the FIN message. On receiving the FIN|ACK response from the server, the client-side state moves to TIME_WAIT. (The transmissions of FIN and FIN|ACK happen within a very short duration e.g. less than a second, and thus the intermediate state FIN_WAIT_1 is not visible in the lower panel of Figure 10. Similarly, the server-side rapidly transitions from ESTABLISHED to CLOSE_WAIT to LAST_ACK, after which the server-side TCP connection is CLOSED. Once again, these intermediate states are not captured in the lower panel of Figure 9.) The client-side remains in the TIME_WAIT state for a duration corresponding to the TCP maximum segment life (MSL) timeout, which is typically 2 minutes as recommended by RFC 793 [1]. See Exercise 1 for further details.
RFC 793 [1] defines TIME-WAIT as a state that “represents waiting for enough time to pass to be sure the remote TCP received the acknowledgment of its connection termination request”. The primary reason for this state is to prevent delayed delivery of a TCP segment3 from an earlier (closed) connection being misinterpreted as a part of a subsequent new (open) connection. This scenario, although unlikely, can occur under rare circumstances when the new connection makes use of the same port number and IP addresses as the earlier connection (i.e., the same socket tuple identification), and has TCP sequence numbers in the same range as used in the earlier connection. A schematic representing this possibility is shown in Figure 11. Here, machine M2 initiates the connection closure and goes into the TIME_WAIT state. If a new connection starts with the same <src IP, src port, dst IP, dst port> tuple, this connection should be rejected until data from the earlier connection that exists in the network can be delivered to M2. The life of data packets in a network is determined by the Time To Live (TTL) field in the IP header[11]., which corresponds to number of hops (i.e., routers in the network) through which this packet can pass through before it gets dropped. Since the time taken by a packet from one router to the next router in the path from source to destination corresponds to transmission delay plus propagation delay [12], TTL roughly corresponds to the maximum duration for which a packet can exist in the network either before it reaches its destination or dropped (when TTL becomes zero4 ). Thus, the TCP connection for the end point that initiates active close must wait for this much time, known as 2MSL time (twice the Maximum Segment Lifetime). When a machine sees a new connection being established with the same socket tuple corresponding to a connection in a TIME_WAIT state, it simply rejects the connection with an error “Address already in use”. This is shown in Figure 12. Here, the client C_m (lower panel) using port number 6666 initiates a TCP connection to the server S_m (upper panel) listening on port 5555. The connection is established and after exchange of the data string hello, the client closes the connection and enters the TIME_WAIT state. Thus, when the client machine initiates a new connection using same socket tuple (using source port 6666 and destination port 5555), the TCP implementation on the client gives the error “bind failed: Address already in use”. Since the client remains in the TIME_WAIT state for 2MSL time (2 minutes) this error will persist until the 2MSL timeout expires.
Figure 11: Use of TIME_WAIT state
Thus, when client initiates another TCP connection using the same source port number which results in tuple value identical to previous connection in TIME_WAIT state, TCP implementation rejects this connection request and returns an error (“Address already in use”).
Figure 12: Use of TCP TIME_WAIT State`
Further, as a safeguarding measure (possibly an overzealous step), if any program on this machine tries to use the port number 6666 during this 2MSL period, it also gets the error “Address already in use”. Two such examples, which result in this error, are shown in lower panel of Figure 12. The first example corresponds to using the source port 6666 to connect to a server application on port 7777 (irrespective of whether this server application is running or not), and the second example corresponds to starting a server application listening on port 6666 on the client machine C_m itself. From a correctness perspective, both these cases should have been allowed. Only the TCP connection corresponding precisely to the earlier tuple <10.211.55.2, 5555, 10.211.55.10, 6666> should have been rejected. This implementation does impose a restriction on the number of TCP connections that can be set up on the machine.
The TIME_WAIT state also serves another purpose. Imagine that the TIME_WAIT state did not exist. Thus, when a connection is in the FIN_WAIT_2 state, and it receives the FIN message from the other end (Figure 1), it sends the ACK message as a response and directly transitions to the CLOSED state. From the perspective of the TCP connection causing active close, the connection is completely closed. However, what if this ACK response is lost? In this case, the other end (following the path of passive close) will remain in the LAST_ACK state. After the timeout (to receive ACK of FIN) expires, it will send the FIN message again. When this message is received by other end (which has completed doing active close), it will not find any connection state and will respond with a TCP Reset error message implying no such connection exists. The connection following the transition path of passive close therefore gets an error message, even though the connection was closed properly – an unwanted situation.
The TIME_WAIT state may lead to scalability issues, and resources need to be planned accordingly. For instance, consider a high-performance web/application server that processes a high number of requests per second from web users. To serve these requests, suppose this server initiates TCP communication with other application servers, database servers, etc. for which it needs to open new connections. Further, this server closes these connections immediately after it gets the required data from these other servers. Since this web/app server is initiating active close, each such connection will transit to TIME_WAIT and remain in this state for 2MSL time before closing and releasing its full resources. The typical ephemeral port range (by default, a client uses source port number within this range when it initiates a connection) on a machine has around 300005 ports. Considering value of 2MSL timeout as 120 second and a server that sets up 250 TCP connections per second, it will very soon have 30000 connections in the TCP_WAIT state. Thus, it will not be able to initiate a new TCP connection as no local port number is available for setting up a new connection. One short-term approach is to reduce the value of 2MSL timeout, but that does not solve the problem since it may lead to the problem of TCP data segments from earlier connections appearing as genuine data segments of newly established later connections, which was the original problem addressed by the TIME_WAIT state. A better alternative could be to assign multiple IP addresses to the web server, and it should randomly pick an IP address and ephemeral port to initiate the connection.
We have explored the basic working of TCP state transition for a normal connection setup and specific network error conditions causing packet loss during TCP Connection Setup (Phase 1). We observed the connection state (SYN_SENT for acting open and SYN_RECV for passive open) at the client and server ends. The insight we have gained into these states will help in diagnosing situations where TCP connections do not work on expected lines, and will thus help us resolve issues accurately, and in a timely fashion.
There are several additional details that have not been covered in this article. Even during the connection setup phase, the transition path from SYN_SENT to SYN_RECV (shown by black color transitions in Figure 1) has not been discussed. This transition corresponds to simultaneous open, which is unlikely to occur – it is practically infeasible to create such a scenario in lab experiments. In the next article, we will discuss the events and state transitions in data transfer phase (Phase 2) and the teardown phase (Phase 3) and related issues. We will explore network behaviour and conditions that will enable us to witness individual states of transition for both active close and passive close, and the related challenges
3In general, data packets at TCP transport layer are called segments.
4Each router when forwards a packet decreases TTL by 1 and drops the packet when the value becomes 0. The max value of TTL can be 255 as it is an 8-bit field.
5This value is specified by system configuration parameter net.ipv4.ip_local_port_range and its default value is 32768-60999.
All the example exercises [13] discussed in this article make use of Linux built in commands related to network utility nc for data transfer and netstat for monitoring network connection state. No specific programs are required for these exercises. At times depending upon your Linux installation, the utility netstat may not be installed. In that case, install this utility by using the command sudo apt-get install net-tools. The netcat (nc) is available on all Linux and Apple Mac machines, (there are free open source versions available for Windows machines as well). All the steps in these exercises are defined for Ubuntu installation 16.04 LTS but would work on other versions of Ubuntu as well and other variants of Linux as well. All the exercise steps described consider that two machines, namely, Ciient (C_m) and Server (S_m) are connected via a network. An example of this connectivity is shown in Figure 3.
a. On S_m, run nc command in server mode to listen on a port. To listen on TCP port number 5555, run the commandnc –l 5555
b. On S_m, run netstat command to check the TCP connection state of this server. Thenetstat –nat | grep 5555
It should show the state of this TCP socket as LISTEN. Since we have not specified any IP address for the server, by default it will consider all the IP addresses available (displayed as 0.0.0.0) on that machine and will show the Local Address as 0.0.0.0.5555. Since no client has connected to this server, thus it will show client side address (listed under the column name Foreign Address) as *.*.
c. On C_m, use a client program to connect to our server application. For this you should know the IP address of the server S_m (It can be found by using the command ip addr show on the server). Assuming the server IP address is 10.11.255.10, run the command on C_mnc 10.211.55.10 5555
d. When network is working fine, the client will connect to server. On both server and client, the connection will show the state ESTABLISHED. Use the following command to check status of TCP connection on both client and server. On client, it will show the server’s IP address as foreign address, and on server it will show the client’s IP address as Foreign address.i. netstat –nat | grep 5555
e. After TCP connection is successfully established, whatever is entered on the terminal of client and server, it will be sent to other side and displayed. The connection will continue to remain in ESTABLISHED state indicating the connection setup is completed and connection is in data transfer stage.
f. On client machine, open the browser, and access any of your preferred website e.g. (http://acc.digital). Check the number of TCP connections which are in established state (netstat –nat | grep ESTABLISHED). It will show all the TCP connections for which setup is completed and data transfer can occur. For website acc.digital, it is likely to show both HTTP (port 80) and HTTPS (port 443) connections i.e.94.130.104.144.80 and 94.130.104.144.443as the Foreign Address of TCP connections in ESTABLISHED state.
g. Terminate the client program (enter Ctrl_C on the terminal window where nc client is running). Since it is client initiated close, TCP connection on client will move to TIME_WAIT state, whereas TCP connection will be fully closed on server and will not be listed under the output of netstat command.
h. Wait for about 2 minutes (equal to 2MSL timeout), and then check the status of TCP Connection. This TCP Connection is closed and will not be shown in output of netstat.
i. Rerun the steps of establishing the TCP connection between client and server (steps a to e), but after data transfer has taken place on this connection, terminate the server program rather than the client program. Now since this is server initiated close, the state of TCP connection on server will show in TIME_WAIT state and TCP connection on client is fully closed (will not show in the output of netstat command).
a. Create a firewall on client side to drop TCP connection setup packet (i.e. SYN packet) to the server. The rule should use the action Drop rather than Reject (since the latter will inform the sender that packet has been dropped and this might cause state change of TCP connection). Assuming (as before) that server program is listening on port 5555 for new connection, use the following command to invoke a firewall rule.
sudo iptables -I OUTPUT -p tcp --dport 5555 --tcp-flags
SYN,ACK,FIN,RST SYN -j DROP
This command inserts a rule in its firewall chain (-I OUTPUT), which defines that a TCP packets (-p tcp) with destination port number of 5555 (--dport 5555) initiating TCP connection setup (TCP Flag as SYN) should be silently dropped without informing the server (-j DROP).
b. As described in previous exercise (i.e. Exercise 1), run the server program (nc –l 5555) on server machine and run the client program on client machine connecting to server (nc 10.211.55.10 5555)
c. Analyze the TCP connection state on client. The state should show as SYN_SENT. This happens because TCP SYN packet is dropped by firewall which client’s TCP connection is unaware of and is waiting for SYN|ACK from server.
d. Analyze the TCP connection state on server. Since it has not received any SYN packet, it will show the TCP socket state as LISTEN only.
e. Analyze the number of packets dropped by firewall rule. Since TCP will retry sending of SYN packets, multiple TCP SYN packets will be sent by client, though it doubles the timeout wait value each time it receives no response. Thus, depending upon the time duration, nc program was running, count of number of packets dropped will vary. Use the following command to observe the number of packets dropped by firewallsudo iptables -L OUTPUT –vn
f. Close the client application and remove the firewall rule on client side. To remove (delete) the re-invoke the firewall rule as in step a) above, but with option –D, for examplesudo iptables -D OUTPUT -p tcp --dport 5555 --tcp-flags
SYN,ACK,FIN,RST SYN -j DROP
g. Create a firewall on server side to drop SYN|ACK packets i.e. the server should receive TCP SYN packets, but when server application sends SYN|ACK as response packet, this packet should be dropped. Since the rule is created on server side, the port number 5555 will become source port, and TCP flags to be checked should specify both SYN and ACK. An example command is
sudo iptables -I OUTPUT -p tcp --sport 5555 --tcp-flags
SYN,ACK,FIN,RST SYN,ACK -j DROP
h. Repeat the above step b) to start the server program and client program.
i. Analyze the connection state at both client and server. Since client has not received TCP SYN|ACK response, it will show the connection state as SYN_SENT. Similarly, server program has sent the SYN|ACK response (which is dropped by firewall) and thus has not received ACK (not completed the 3-way handshake), and thus TCP Connection state would be in SYN_RECV. Since TCP server programs are subject to TCP SYN flooding attack [10]., it is possible that this connection is closed in a short time and may not be seen in the output of netstat command. Thus, if you encounter this situation, repeat the experiment few times to witness this state.
[1] RFC 2616, “Hyper Text Transfer Protocol – HTTP/1.1”, Network Working Group; Fielding, Gettys et al. June 1999, https://tools.ietf.org/html/rfc2616, last accessed Nov 2018
[2] RFC 793, “Transmission Control Protocol “, Information Sciences Institute, USC, CA, Sep 1981, https://tools.ietf.org/html/rfc793. Last accessed Aug 2018.
[3] Ram Rustagi, Viraj Kumar, “Understanding Transport Layer Basics”, ACCS journal of Computing and Communications, Vol 2, Issue 3, Sep 2018. https://acc.digital/experiential-learning-of-networking-technologies-understanding-transport-layer-basics/, last accessed Nov 2018.
[4] Communicating Finite State Machines, Wikipedia; https://en.wikipedia.org/wiki/Communicating_finite-state_machine, accessed Nov 2018.
[5] netstat: man page, http://manpages.ubuntu.com/manpages/bionic/man8/netstat.8.html, Accessed Nov 2018.
[6] netstat usage examples: https://www.tecmint.com/20-netstat-commands-for-linux-network-management/
[7] Netcat (nc) command utility,
http://manpages.ubuntu.com/manpages/xenial/man1/nc.traditional.1.html, last accessed Aug 14, 2018.
[8] nc cheat-sheet, https://www.sans.org/security-resources/sec560/netcat_cheat_sheet_v1.pdf, last accessed Nov 2018.
[9] Iptables tutorial. https://www.frozentux.net/iptables-tutorial/iptables-tutorial.html, last accessed Nov 2018.
[10] https://www.cisco.com/c/en/us/about/press/internet-protocol-journal/back-issues/table-contents-34/syn-flooding-attacks.html. Last accessed Nov 2018.
[11] RFC 791, “Internet Protocol”, Information Sciences Institute, USC, CA, Sep 1981, https://tools.ietf.org/html/rfc791, last accessed Nov 2018
[12] Ram Rustagi, Viraj Kumar, “Understanding Network Delays”, ACCS journal of Computing and Communications, Vol 1, Issue 3, Dec 2017; https://acc.digital/experiential-learning-of-networking-technologies-understanding-network-delays/, last accessed Nov 2018.
[13] Example details for TCP Connection state during connection setup, https://github.com/rprustagi/EL-TCP-States.git. Last accessed Nov 2018.
