Traceroute outcome from Bangalore to AWS Virginia and California

I recently noticed that my site (https://infoheap.com/) which was hosted on AWS North Virginia data center was very slow. I was accessing it from Airtel broadband (Bangalore, India). When I checked with a friend who is using a different broadband, he found the site loading as usual.

There was also a recent news about some internet cable getting cut in Egypt. Quoting from the article form techcrunch:

Egyptian authorities arrested three men off the coast of Alexandria for attempting to sever an undersea Internet cable. The SEA-ME-WE 4 main line was part of a cable network that spanned the Mediterranean and connects Southeast Asia, the Middle East and Western Europe and has 39 landing points, including Alexandria.

As per this amazon forums thread, there were no service wide issues in the US-East Region. As per AWS service health dashboard also things were fine.

I was also able to see from apache logs that latency to serve pages and images were fine. So it means that at least some users were being served and being served ok.

Some questions I had

1. Are things fine in AWS data center? Are servers up? Is webserver processing the request on time whenever it is receiving a request?
   Answer: This part seems to be ok as per apache log.
2. Now there may be routing issue from user to server. This could be related to Egypt cable incidence.
   Answer: It seems non-trivial to find answer for this.
3. Will it be better to shift servers to a different data center? May be califonia or may be Asia pacific (singapore)?
   Answer: Overall if your users are not specific to a country, I think its better to be hosted in US.
4. Is this problem only with Airtel broadband in India or other broadband providers also? May be a specific broadband providers is not using optimum routing algorithms. I’m assuming routing algorithms are based on cost and latency. If cost factor is more, we may have more latency when a link is broken or slow even if there are better links available.
   Answer: At least one ISP did not have this issue at a specific time. So seems like not all ISPs are using same route.

To do the traceroute experiment I migrated one of my site (infoheap.com) to AWS North California server. Then I did traceroute to two of my servers located in AWS North Virginia and North California

North Virginia Server

Traceroute from Bangalore, India (Airtel Broadband) To AWS North Virginia Data Center:

traceroute to funvector.com (54.235.193.178), 64 hops max, 52 byte packets
1 192.168.100.1 (192.168.100.1) 36.187 ms 2.725 ms 4.503 ms
2 192.168.1.1 (192.168.1.1) 4.161 ms 3.291 ms 3.985 ms
3 abts-kk-dynamic-001.0.167.122.airtelbroadband.in (122.167.0.1) 9.553 ms 10.295 ms 8.727 ms
4 abts-kk-static-077.32.166.122.airtelbroadband.in (122.166.32.77) 9.757 ms 9.045 ms 8.650 ms
5 abts-kk-static-009.32.166.122.airtelbroadband.in (122.166.32.9) 8.621 ms 10.134 ms 8.769 ms
6 122.175.255.29 (122.175.255.29) 15.662 ms 15.784 ms 15.060 ms
7 125.62.187.133 (125.62.187.133) 316.090 ms 350.472 ms 279.762 ms
8 ldn-b2-link.telia.net (213.248.71.17) 279.397 ms 279.198 ms 327.051 ms
9 ldn-bb1-link.telia.net (80.91.249.171) 409.387 ms
ldn-bb2-link.telia.net (80.91.245.28) 306.967 ms
ldn-bb2-link.telia.net (80.91.251.238) 348.587 ms
10 ash-bb3-link.telia.net (80.91.251.207) 306.917 ms
ash-bb4-link.telia.net (80.91.251.209) 351.922 ms
ash-bb4-link.telia.net (213.248.65.210) 307.314 ms
11 ash-b1-link.telia.net (213.155.136.39) 308.038 ms
ash-b1-link.telia.net (213.155.130.59) 353.300 ms
ash-b1-link.telia.net (213.155.130.73) 308.732 ms
12 vadata-ic-157234-ash-bb1.c.telia.net (62.115.9.74) 305.326 ms
vadata-ic-148786-ash-bb1.c.telia.net (213.155.129.222) 439.573 ms
vadata-ic-144300-ash-bb1.c.telia.net (80.239.196.2) 357.489 ms
13 * 72.21.220.47 (72.21.220.47) 526.907 ms
72.21.220.63 (72.21.220.63) 511.621 ms
14 205.251.245.55 (205.251.245.55) 307.959 ms
72.21.222.155 (72.21.222.155) 501.906 ms 412.302 ms
15 * * *
16 * * *
17 * * *
18 216.182.224.89 (216.182.224.89) 438.668 ms
   216.182.224.212 (216.182.224.212) 421.628 ms *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
31 * * *
32 * * *
33 * * *
34 * * *
35 * * *
36 * * *
37 * * *
38 * * *
39 * * *
40 * * *
41 * * *
42 * * *
43 * * *
44 * * *
45 * * *
46 * * *
47 * * *
48 * * *
49 * * *
50 * * *
51 * * *
52 * * *
53 * * *
54 * * *
55 * * *
56 * * *
57 * * *
58 * * *
59 * * *
60 * * *
61 * * *
62 * * *
63 * * *
64 * * *

This outcome suggested this route (using www.iplocation.net):

Bangalore ->New Delhi (Bharti Telesonic Infrastructure) -> London (Teliasonera International Carrier) -> Virginia (Telia International Carrier)

North California Server

Traceroute from Bangalore, India (Airtel Broadband) To AWS North Califonira data center

traceroute to infoheap.com (54.241.26.46), 64 hops max, 52 byte packets
1 192.168.100.1 (192.168.100.1) 6.917 ms 4.585 ms 2.664 ms
2 192.168.1.1 (192.168.1.1) 3.581 ms 4.783 ms 3.220 ms
3 abts-kk-dynamic-001.0.167.122.airtelbroadband.in (122.167.0.1) 9.073 ms 23.560 ms 9.647 ms
4 abts-kk-static-077.32.166.122.airtelbroadband.in (122.166.32.77) 8.536 ms 8.219 ms 9.471 ms
5 abts-kk-static-009.32.166.122.airtelbroadband.in (122.166.32.9) 8.797 ms 9.079 ms 8.967 ms
6 122.175.255.29 (122.175.255.29) 14.835 ms 14.971 ms 15.024 ms
7 125.62.187.113 (125.62.187.113) 326.104 ms 248.977 ms 305.568 ms
8 paix01-sfo4.amazon.com (198.32.176.36) 307.133 ms 353.113 ms 307.089 ms
9 205.251.229.14 (205.251.229.14) 308.263 ms 306.725 ms 307.236 ms
10 72.21.222.19 (72.21.222.19) 307.324 ms 244.368 ms 266.714 ms
11 216.182.236.108 (216.182.236.108) 306.909 ms 351.062 ms 307.238 ms
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
31 * * *
32 * * *
33 * * *
34 * * *
35 * * *
36 * * *
37 * * *
38 * * *
39 * * *
40 * * *
41 * * *
42 * * *
43 * * *
44 * * *
45 * * *
46 * * *
47 * * *
48 * * *
49 * * *
50 * * *
51 * * *
52 * * *
53 * * *
54 * * *
55 * * *
56 * * *
57 * * *
58 * * *
59 * * *
60 * * *
61 * * *
62 * * *
63 * * *
64 * * *

This outcome suggested this route (using www.iplocation.net):

Bangalore ->New Delhi (Bharti Telesonic Infrastructure) -> Palo Alto (Ep.net Llc.)

Conclusion

Overall I found the server I migrated to north california was responding better. I’ll watch it for some more days. This suggest that Airtel’s routing via London is not working well. Its nice to see Amazon support folks being proactive in following up with Airtel on this.

Its hard to draw any conclusion from this analysis which applies to all contexts. I think we should look at various factors like server availability in data center, routing path taken, local broadband provider, etc. and analyze the situation and take an action if needed.