Having two interfaces in a module on the same subnet does not give you two interfaces into and out of the module. While you do get two interfaces into the module, STCP will use just one interface when sending packets out of the module. It does this regardless of which interface a client has connected to.

 

For example, given these two interfaces on the 172.16.1.0/24 subnet
%phx_vos#sdlmux.m16.10.11-2        IP: 172.16.1.116                     MAC: 00:00:a8:42:52:22
%phx_vos#sdlmux.m16.11.11-2        IP: 172.16.1.226                     MAC: 00:00:a8:43:52:22

 

Notice in figure 1 that the packet sent to 172.16.1.226 uses the destination MAC address of 00:00:a8:43:52:22 but the reply comes from 00:00:a8:42:52:22, the MAC address for 172.16.1.116 even though in the packet the reply is shown as coming from 172.16.1.226.

 

11:04:57.117 Xmit Ether Dst 00:00:a8:43:52:22  Src 00:00:a8:42:3b:6e Type 0800
+(IP)
IP   Ver/HL 45, ToS  0, Len   2c, ID 2045, Flg/Frg    0, TTL 3c,  Prtl  6
Cksum  0363, Src ac100122, Dst ac1001e2
TCP from 172.16.1.34.49183 to 172.16.1.226.telnet
. . .
11:04:57.130 Rcvd Ether Dst 00:00:a8:42:3b:6e  Src 00:00:a8:42:52:22 Type 0800
+(IP)
IP   Ver/HL 45, ToS  0, Len   2c, ID  7c8, Flg/Frg    0, TTL 3c,  Prtl  6
Cksum  1be0, Src ac1001e2, Dst ac100122
TCP from 172.16.1.226.telnet to 172.16.1.34.49183
. . .
Figure 1 – connection request to module (trace taken on client)
If the module initiates a connection instead of receiving one, it will always use the first (116) interface.
13:01:36.479 Rcvd Ether Dst 00:00:a8:42:3b:6e  Src 00:00:a8:42:52:22 Type 0800
+(IP)
IP   Ver/HL 45, ToS  0, Len   2c, ID 4a33, Flg/Frg    0, TTL 3c,  Prtl  6
Cksum  d9e2, Src ac100174, Dst ac100122
TCP from 172.16.1.116.49320 to 172.16.1.34.4680
. . .
Figure 2 – connection request from module to another server (trace taken on server)
Even if you bind to the second (226) IP address when you send out the packet it leaves via the first (116) interface
13:02:55.349 Rcvd Ether Dst 00:00:a8:42:3b:6e  Src 00:00:a8:42:52:22 Type 0800
+(IP)
IP   Ver/HL 45, ToS  0, Len   2c, ID 4b7a, Flg/Frg    0, TTL 3c,  Prtl  6
Cksum  d82d, Src ac1001e2, Dst ac100122
TCP from 172.16.1.226.49321 to 172.16.1.34.4680
. . .
Figure 3 – connection request from module to another server (trace taken on other server)
client application bound to the 172.16.1.226 IP interface
Only if the first (116) interface goes down will the module shift to using the second (226) interface. While connections to the second (226) interface will be maintained, connections to the first (116) interface will fail.
13:14:43.884 Xmit Ether Dst 00:00:a8:43:52:22  Src 00:00:a8:42:3b:6e Type 0800
+(IP)
IP   Ver/HL 45, ToS  0, Len   28, ID 5072, Flg/Frg    0, TTL 3c,  Prtl  6
Cksum  d339, Src ac100122, Dst ac1001e2
TCP from 172.16.1.34.49186 to 172.16.1.226.telnet
. . .
13:14:43.885 Rcvd Ether Dst 00:00:a8:42:3b:6e  Src 00:00:a8:42:52:22 Type 0800
+(IP)
IP   Ver/HL 45, ToS  0, Len   28, ID 53ef, Flg/Frg    0, TTL 3c,  Prtl  6
Cksum  cfbc, Src ac1001e2, Dst ac100122
TCP from 172.16.1.226.telnet to 172.16.1.34.49186
. . .
Interface 172.16.1.116 fails – note the change in source MAC address
13:15:56.718 Xmit Ether Dst 00:00:a8:43:52:22  Src 00:00:a8:42:3b:6e Type 0800
+(IP)
IP   Ver/HL 45, ToS  0, Len   29, ID 50e7, Flg/Frg    0, TTL 3c,  Prtl  6
Cksum  d2c3, Src ac100122, Dst ac1001e2
TCP from 172.16.1.34.49186 to 172.16.1.226.telnet
. . .
13:15:56.719 Rcvd Ether Dst 00:00:a8:42:3b:6e  Src 00:00:a8:43:52:22 Type 0800
+(IP)
IP   Ver/HL 45, ToS  0, Len   29, ID 5552, Flg/Frg    0, TTL 3c,  Prtl  6
Cksum  ce58, Src ac1001e2, Dst ac100122
TCP from 172.16.1.226.telnet to 172.16.1.34.49186
. . .
Figure 4 – Changes in MAC when first interface fails (trace taken on client)
How can you tell which interface will be used to send traffic?
There are several ways.  The first is to use packet_monitor or some other protocol analyzer on the remote host or the network to evaluate the source Mac addresses, see figures 1 thru 4.
Second is to use netstat to look at the interface, generate some traffic, run netstat again and compare. Only one interface should have incremented its “transmitted frames” counter.
netstat -interface #sdlmux.m16.10.11-2; netstat -interface #sdlmux.m16.11.11-2
. . .
MAC Statistics:
Received frames                          : 1598960
Received multicast and broadcast frames  : 1564503
Received octets                          : 258513766
Transmitted frames                       : 531
Transmitted octets                       : 50850
. . .
MAC Statistics:
Received frames                          : 1603838
Received multicast and broadcast frames  : 1569292
Received octets                          : 259291185
Transmitted frames                       : 15
Transmitted octets                       : 947
. . .
ready  13:29:48
ping 172.16.1.34
Pinging host 172.16.1.34 : 172.16.1.34
ICMP Echo Reply:TTL 60 time = 0 ms
ICMP Echo Reply:TTL 60 time = 0 ms
ICMP Echo Reply:TTL 60 time = 0 ms
ICMP Echo Reply:TTL 60 time = 0 ms
Host 172.16.1.34 replied to all 4 of the 4 pings
ready  13:29:55
netstat -interface #sdlmux.m16.10.11-2; netstat -interface #sdlmux.m16.11.11-2
. . .
MAC Statistics:
Received frames                          : 1603142
Received multicast and broadcast frames  : 1568591
Received octets                          : 259187870
Transmitted frames                       : 535
Transmitted octets                       : 51274
. . .
MAC Statistics:
Received frames                          : 1607492
Received multicast and broadcast frames  : 1572869
Received octets                          : 259880655
Transmitted frames                       : 15
Transmitted octets                       : 947
. . .
Figure 5 – Using netstat to compare transmitted frames counter
The third way is to connect to a service on another system and see what local source address is used. Unless you explicitly bind to another address the source IP address will correspond to the active interface.
netstat -numeric
Active connections
Proto Recv-Q Send-Q  Local Address      Foreign Address    (state)
. . .
tcp        0      0  172.16.1.116:49369 172.16.1.34:23     ESTABLISHED
. . .
Figure 6 – what interface is used for outgoing connections
It is my opinion that having multiple interfaces on the same subnet just confuses things and is usually not worth the confusion. If for some reason you need to IP addresses on the same subnet add an alias to the first interface.
ifconfig #sdlmux.m16.10.11-2 172.16.1.226 -add -alias
Adding IP address 172.16.1.226 to interface %phx_vos#sdlmux.m16.10.11-2
%phx_vos#sdlmux.m16.10.11-2: <UP, BROADCAST, RUNNING, NOFORWARDBROADCAST, KEEPAL
+IVE>
172.16.1.116 netmask 0xffffff00 broadcast 172.16.1.255
Number of additional address(es): 1
172.16.1.226
Figure 7 – adding an alias to an interface