Code
https://github.com/obrienlabs/magellan
http://jenkins.obrienlabs.cloud/job/magellan-mvn-java-snapshot/
- MAGELLAN-1Getting issue details... STATUS
There is something old school about knowing exactly where your packets are on the surface or below the ocean of the earth.
I know this might sound weird but not to someone who lives science as a normal part each day. I would like to make sure my REST calls propagate around the globe - not just to a server on the other side of the earth and back again like from us-east-1 to the Singapore ap-southeast-1 region but through us-west-2 through singapore - to europe and back to us-east-1.
Sort of when I fly - in order to maximize the number of takeoffs and landings - I try to get the most number of stops - a direct flight is not the best way to get somewhere when testing latency - as in packets on the cable from Oregon to China are better routed through Hawaii.
I liked Andrew Blum's "Tubes" book - which include details on the endpoints for sub-marine cables - https://www.amazon.com/Tubes-Journey-Internet-Andrew-Blum/dp/0061994952 - I also ran into a bit of this propagation delay working at Nokia.
This system will of course run a set of helm/kubernetes clusters (a cluster of 1 on each DC for now)
First principles will be install proxies at several regions in Amazon EC2 - https://aws.amazon.com/about-aws/global-infrastructure/
see https://www.submarinecablemap.com/
This is the path we will take (see the red circles on the Amazon AWS regions) - hopefully no backtracking and ideally it is roughly valid - as we do not actually cross the equator - I will also try through Sau Paulo.
through the following Amazon regions
AWS Regions
Propagation order
US East (N. Virginia)Canada (Central)US East (Ohio)US West (N. California)US West (Oregon)Asia Pacific (Tokyo)Asia Pacific (Seoul)
Asia Pacific (Singapore)Asia Pacific (Sydney) - skip
Asia Pacific (Mumbai)EU (Frankfurt)EU (Paris)
EU (Stockholm) - skipEU (London)EU (Ireland)South America (São Paulo) - skip
packet.global | DNS *.packet.global | |||||||||
DC | Identifier | Location | Region | EIP | TZ | GEO | Distance | City | ||
---|---|---|---|---|---|---|---|---|---|---|
0 | 0000 | US East (N. Virginia) | us-east-1 | 34.200.202.57 | p0/virginia | GMT-5 | New York | |||
Toronto | ca-central-1 | p1/toronto | Toronto | |||||||
US West Ohio | us-east-2 | p2/ohio | ||||||||
California | us-west-1 | p3/california | ||||||||
Oregon | us-west-2 | p4/oregon | ||||||||
ap-northeast-1 | p5 | Tokyo | ||||||||
ap-northeast-2 | p6/seoul | Seoul | ||||||||
ap-southeast-1 | p7/singapore | GMT+7 | Singapore | |||||||
ap-south-1 | p8/mumbai | Mumbai | ||||||||
p9 | Sydney | |||||||||
eu-central-a | p10/frankfurt | Frankfurt | ||||||||
eu-west-3 | p11/paris | Paris | ||||||||
p12 | Stockholm | |||||||||
eu-west-2 | p13/london | London | ||||||||
eu-west-1 | p14/ireland | Ireland | ||||||||
sa-east-1 | p15 | Sao Paulo | ||||||||
Base Cases
Opposite sides of the earth - Singapore - ap-southeast-1 to New York - us-east-1
It will never be shorter that 160 ms - which is c/(radius * pi /2) = as it it takes 16% of a second for light to pass to the other side (this is without optical repeater and endpoint delays)
packets within us-east-1
ubuntu@ip-172-31-68-232:~$ ping rke.obrienlabs.cloud PING rke.obrienlabs.cloud (3.216.18.230) 56(84) bytes of data. 64 bytes from ec2-3-216-18-230.compute-1.amazonaws.com (3.216.18.230): icmp_seq=1 ttl=63 time=0.786 ms
packets from singapore to us-east-1
they all take 225 ms except one 227 ubuntu@ip-172-31-46-121:~$ ping rke.obrienlabs.cloud PING rke.obrienlabs.cloud (3.216.18.230) 56(84) bytes of data. 64 bytes from ec2-3-216-18-230.compute-1.amazonaws.com (3.216.18.230): icmp_seq=1 ttl=30 time=225 ms 64 bytes from ec2-3-216-18-230.compute-1.amazonaws.com (3.216.18.230): icmp_seq=2 ttl=30 time=225 ms --- rke.obrienlabs.cloud ping statistics --- 58 packets transmitted, 58 received, 0% packet loss, time 57013ms rtt min/avg/max/mdev = 225.138/225.248/225.545/0.188 ms and back to singapore ubuntu@ip-172-31-68-232:~$ ping singapore.obriensec.com PING singapore.obriensec.com (18.138.217.106) 56(84) bytes of data. 64 bytes from ec2-18-138-217-106.ap-southeast-1.compute.amazonaws.com (18.138.217.106): icmp_seq=1 ttl=28 time=225 ms 64 bytes from ec2-18-138-217-106.ap-southeast-1.compute.amazonaws.com (18.138.217.106): icmp_seq=2 ttl=28 time=227 ms --- singapore.obriensec.com ping statistics --- 86 packets transmitted, 86 received, 0% packet loss, time 85106ms rtt min/avg/max/mdev = 225.188/225.350/227.678/0.687 ms
Architecture
Conceptual Architecture
Path Propagation
All paths are linear in sequence with the first bit reserved for off axis secondary routes. For example 00 is us-east and 10 is us-east but 01 is ca-central (north of us-east).
DevOps Architecture
- MAGELLAN-3Getting issue details... STATUS
https://hub.docker.com/repository/docker/obrienlabs/magellan-nbi
Deployment Diagram
Services Architecture
see
https://eclipsejpa.blogspot.com/2019/01/sending-packets-around-earth.html
Use Cases
Work Items
WI | Task | Jira / Wiki |
---|---|---|
WI 1 | OpenAPI specification | |
Application Architecture | ||
adjust readme | ||
Deployment Architecture | ||
adjust readme deployment guide | ||
POC | ||
1 | Terraform infrastructure | |
Helm/Kubernetes Charts | ||