Problems and Requirements of Addressing in Integrated Space-Terrestrial Network
Internet-Draft draft-li-istn-addressing-requirement-01
Yuanjie Li
Tsinghua University
Beijing
China
yuanjiel@tsinghua.edu.cn
Hewu Li
Tsinghua University
Beijing
China
lihewu@cernet.edu.cn
Jiayi Liu
Tsinghua University
Beijing
China
liu-jy21@mails.tsinghua.edu.cn
Lixin Liu
Tsinghua University
Beijing
China
liulx19@mails.tsinghua.edu.cn
Qian Wu
Tsinghua University
Beijing
China
wuqian@cernet.edu.cn
2022
General
Internet Engineering Task Force
template
This document presents a detailed analysis of the problems and requirements of network addressing in "Internet in space" for terrestrial users. It introduces the basics of satellite mega-constellations, terrestrial terminals/ground stations, and their inter-networking. Then it explicitly analyzes how space-terrestrial mobility yeilds challenges for the logical topology, addressing, and their impact on routing. The requirements of addressing in the space-terrestrial network are discussed in detail, including uniqueness, stability, locality, scalability, efficiency and backward compatibility with terrestrial Internet. The problems and requirements of network addressing in space-terrestrial networks are finally outlined.
Introduction
The future Internet is up in the sky. We have seen a rocket-fast deployment of mega-constellations with 100s-10,000s of low-earth-orbit (LEO) satellites, such as Starlink, Kuiper and OneWeb. These constellations promise competitive low latency and high capacity to terrestrial networks. They expand global high-speed Internet to remote areas that were not reachable by terrestrial networks, resulting in a tens-of-billions-of-dollar market with 3.7 billion users in rural areas[ITU-Measure], developing countries, aircraft, or oceans.
A salient feature for LEO mega-constellations is their high relative motions to the rotating earth. Unlike geosynchronous satellite or terrestrial networks, each LEO satellite moves fast (e.g., 28,080 km/h for Starlink), causing short-lived coverage for terrestrial users (less than 3 minutes). This yields diverse challenges for the traditional network designs.
This memo outlines the problems and requirements of addressing in integrated space-terrestrial network. It starts with the basics of satellite mega-constellations, terrestrial ground stations/terminals, and their inter-networking.
It analyzes how high space-terrestrial relative motions yields challenges for logical topology, addressing and their impacts on routing. Then it discusses the requirements of network addressing in space-terrestrial network for uniqueness, stability, locality, scalability, efficiency and backward compatibility with terrestrial Internet.
Terminology
GSO: Geosynchronous orbit (at the altitude of 35,786 km).
NGSO: Non-geosynchronous orbit.
LEO: Low Earth Orbit (at the altitude of 180-2,000 km).
MEO: Medium Earth Orbit (at the altitude of 2,000-35,786 km).
ISL: Inter Satellite Link.
NAT: Network Address Translation.
GS: Ground Station, a device on ground connecting the satellite.
FIB: Forwarding Information Base.
Basics of Space-Terrestrial Network
As shown in Figure 1, a space-terrestrial network for terrestrial users consists of the satellite or constellations, terrestrial terminals, and ground stations.
Satellites and Mega-Constellations
Satellites can be classified based on their relative motions to the earth. A satellite can operate at the geosynchronous orbit (GSO, at about 35,786 km altitude) or non-geosynchronous orbits, such as low earth orbits (LEO, <=2,000km) and medium earth orbits (MEO, between 2,000 km and 35,786 km). Satellites at higher altitudes offer broader coverage, while satellites at lower altitudes move faster.
Historically, communications in space were dominated by GSO satellites. As shown in Table 1, GSO offers excellent coverage at high altitudes, but at the cost of long space-terrestrial RTT (>=200ms) and low bandwidth (<=10Mbps, due to bit errors in long distance transmission). Instead, recent efforts seek to adopt satellites at lower non-geosynchronous orbits, with a special interest in low-earth orbits. Together with Ku (12–18 GHz) and Ka (26.5–40GHz) bands, these satellites promise competitive bandwidth and latency to terrestrial networks( [LOWLATENCY-ROUTING-SPACE], [SPACE-RACE], [NETWORK-TOPO-DESIGN]). Due to low coverage for each LEO satellite, a mega-constellation is necessary to retain global coverage. Table 2 exemplifies popular LEO mega-costellations in operation. They are enabled by recent advances in satellite miniaturization and rocket reusability.
A simplified space-terrestrial network architecture

         +-----------+                 +-----------+
         | Satellite |_______ISL_______| Satellite | (Space Segment)
      _ _|___________|              _ _|___________|
         /|           \                /|
        / |            \              / |
       /   (Bent pipe)  \ |          /
      /               _ _\|         /
+---------+               +------------+
|  Mobile |               |   Ground   |
| Station |               |  Station   |            (Ground Segment)
|_________|               |____________|

Differences between GSO and NGSO.
Orbit | Altitude (km) | RTT (ms)
GSO | GEO | 35,786 | 240
NGSO | MEO | 2,000-35,786 | 12-240
LEO | 500-2,000 | 2-12
Low-earth-orbit (LEO) satellite mega-constellations in operation.
Constellation | Num. satellites | Num. orbits | Altitude (km)
Starlink | 1584 | 72 | 540/550
720 | 36 | 570
348 | 6 | 560
172 | 4 | 560
Kuiper | 1156 | 34 | 630
1296 | 36 | 610
784 | 28 | 590
Telesat | 351 | 27 | 1015
1320 | 40 | 1325
Iridium | 66 | 6 | 780
Evolution of Space-Terrestrial Network
Terrestrial users access satellite networks via terminals (e.g., satellite phones, onboard dishes, IoT endpoints) or ground stations. Ground stations can serve as network gateways (e.g., carrier-grade NAT in Starlink and Kuiper) and remote satellite controllers (e.g., telemetry, tracking, orbital update commands, or centralized routing control).
"One-to-one" satellite communication
Early satellite communications favor the simple "bent-pipe-only" model (Figure 1), i.e., satellites only relay terrestrial users' radio signals to the fixed ground stations without ISLs or routing. This model has been popular in GSO satellites with broad coverage (2G GMR [GEO-MOBILE-RADIO-INTERFACE] [ETSI-TS-101], 3G BGAN [BGAN] [ETSI-TS-102], and DVB-S [SATELLITE-COMMUNICATIONS]), and recently adopted by LEO satellites in OneWeb (4G) [ONEWEB] and 5G NTN [STUDY-NR-SUPPORT] [SOLUTION-NR-NTN]. However, this model suffers from low LEO satellite coverage. To access the network, both terrestrial users and ground stations must reside inside the satellite's coverage. Due to each LEO satellite's low coverage, most users in remote areas with sparse or no ground stations cannot be served. As shown in Table 3, under current Starlink (no ISLs so far) and ground station deployments ([STARLINK-GS-FOUND], [AZURE-GS], [AWS-GS], [GOOGLE-DATA-CENTER], [AZURE-CLOUD-STARLINK], [STARLINK-GS-MAP]), 27%–52% global populations cannot be served by the "one-to-one" model (depending on how many satellites each ground station can simultaneously associate to). Most under-served users are from remote areas (e.g., Africa), thus causing revenue loss for operators.
The ground station aggregates traffic from all satellite users and becomes the single-point bottleneck. In reality, Starlink’s LEO satellites generate 5 TB telemetry data per day for the ground stations to process [REDDIT]. With limited space-terrestrial radio link capacity, its ground stations have limited the LEO network’s total capacity [COMPARISON-THREE-CONSTELLATION]. Similarly, each OneWeb's ground station must process 10,000 terminal handovers per second [ONEWEB-GS]. Deploying dense ground stations in these remote areas could mitigate above two problems. However, it is expensive and lowers commercial competitive advantages to terrestrial networks.
"One-to-many" networked ground stations
To this end, networked space-terrestrial infrastructure is crucial for global coverage and single-point bottleneck elimination. To date, inter-satellite links (ISLs) are under early adoption([BEIDOU-TEST], [TheVerge-STARLINK-SPEED]). The recent "burn on re-entry" regulations from FCC also slows down the adoption of ISLs[SPACEX-CLAIM]. As a near-term remedy, routing with distributed ground station networks is adopted. There are two variants. The ground station-as-gateway is adopted by Starlink and Kuiper. Each ground station is a carrier-grade NAT that offers private IP[RFC0791] for terrestrial users. The ground station-as-relay [USE-GROUND-RELAY] mitigates ISLs with ground station-assisted routing, but is vulnerable to intermittent space-terrestrial links in Ku/Ka-bands. Fundamentally, the "one-to-many" model heavily relies on global deployments of ground station networks, thus offsetting LEO satellites' advantages and competitive edges to terrestrial networks.
"Many-to-many" networked satellites
To unleash LEO mega-constellations' potentials and long-term success, the networked LEO satellites are under rapid deployments[KUIPER][STARLINK]. Today's networked LEO satellites typically have a microwave space-terrestrial radio interface and at least 4 laser/microwave inter-satellite links (ISLs) [LOWLATENCY-ROUTING-SPACE][USE-GROUND-RELAY] (2 for intra-orbit satellites, 2 for inter-orbit satellites). With this capability, recent work has explored topology design [NETWORK-TOPO-DESIGN], low-latency routing [LOWLATENCY-ROUTING-SPACE][SPACE-RACE], inter-domain routing [Giuliari20Internet], orbital computing [ORBITAL-EDGE-COM][IN-ORBIT-COM], and security [ICARUS] in LEO networks. We take a forward-looking view to simplifying LEO networks in the first place and helps these efforts fulfill their merits in space.
Global population that could access Starlink in its current "bent-pipe-only" model.
| Global | Africa | Oceania | South America | Asia | European | North America
1-SAT association | 48.71% | 19.52% | 42.85% | 49.63% | 43.49% | 91.00% | 87.50%
2-SAT association | 57.30% | 24.37% | 56.58% | 53.90% | 55.91% | 94.33% | 91.23%
4-SAT association | 67.04% | 26.13% | 60.31% | 63.16% | 71.34% | 95.46% | 95.04%
8-SAT association | 73.04% | 29.17% | 60.68% | 65.65% | 80.28% | 96.91% | 98.86%
Problems in Space-Terrestrial Network Addressing
In terrestrial and GEO satellite networks, the logical network topology, addresses, and routes are mostly stationary due to fixed infrastructure. Instead, LEO mega-constellations hardly enjoy this luxury, whose satellites move at high speeds (about 28,080 km/h). The earth’s rotation further complicates the relative motions between space and ground. In this section, we will analyze how high relative motion between space and ground challenges addressing due to topology instability, and its impact on routing[INTERNET-IN-SPACE].
Unstable Space-Terrestrial Topology
High physical mobility incurs frequent link churns between space and terrestrial nodes, thus causing frequent logical network topology changes. For all mega-constellations in Table 2, the topology changes every 10s of seconds[INTERNET-IN-SPACE]. The link churn populates with more satellites and ground stations.
In terrestrial mobile networks (e.g., 4G/5G), such physical link churn can be masked by handoffs without incurring logical topology changes. This method works based on two premises. First, all link churns occur at the last-hop radio due to user mobility, without affecting the infrastructure topology. Second, all cellular infrastructure nodes are fixed, resulting in a stable logical topology as “anchors”.
However, neither premise holds in non-geosynchronous constellations. Instead, infrastructure mobility between satellites and ground stations becomes a norm rather than an exception. This voids cellular handoffs’ merits to avoid propagation of physical link churns to logical network topology: They are designed for user mobility only, and heavily rely on the fixed infrastructure as “anchors.” Therefore, 5G NTN lists satellite handoffs as an unsolved problem ([STUDY-NR-SUPPORT], [SOLUTION-NR-NTN]), and the latest 3GPP 5G release 17 defers its mobility support for satellites [TEC-SPECI-GROUP-MEETING] due to significant architectural changes. While Starlink uses handoffs to migrate physical links between satellites and ground stations (every 15s [STARLINK-CGNAT]), its logical topology and routing are still be repeatedly updated at high costs.
Inconsistent “Locations” for Space/Terrestrial Nodes
Each space/terrestrial node has two notions of “locations”: The logical location in its topological address, and the physical location in reality. With repetitive topology changes, a static network address can hardly ensure its logical location in the topology is consistent with the fast-moving node’s physical location in reality. Then to correctly forward data, a network should choose one of the following designs:
a. Dynamic address updates
A node can repetitively re-bind its physical location to its logical network address, thus incurring frequent address updates or re-binding. Under high mobility, this could severely disrupt user experiences or incur heavy signaling overhead. Table 4 and Table 5 project the address update frequency when using legacy IP addresses[RFC0791] for logical interfaces. In this scheme, the terrestrial users’ logical IP[RFC0791] address changes if it re-associates to a new satellite (thus new interfaces and subnets) to retain its Internet access. Due to high LEO satellite mobility, each user is forced to change its logical IP address[RFC0791] every 133–510s. Every second, we observe 2,082–7,961 global users per second should change their IP addresses.
Frequency of each user's logical IP address update.
Starlink | Telesat | Kuiper | Iridium
Every 133s | Every 510s | Every 510s | Every 458s
Number of terrestrial users that change logical IP address per second.
Starlink | Telesat | Kuiper | Iridium
7961 | 2082 | 5673 | 2379
b. Static address binding to a fixed gateway
This is adopted by the cellular networks and Starlink [STARLINK-CGNAT] and Kuiper’s[KUIPER-CGNAT] initial rollouts. Each user gets a static address from the remote ground station (via carrier-grade NAT), which masks the external address changes and redirects users’ traffic. This mitigates user address updates, but cannot avoid gateway’s external address updates when changing satellite interfaces (detailed below). It also incurs detours and long routing latencies for remote users from ground stations (e.g., 18,000 km detours and 370 ms extra delays in [lai2021icnp]).
Impact on Routing: Frequent Routing Updates
The inconsistent locations in addressing further impact the network routing. As space and terrestrial infrastructure nodes physically move fast, the logical routing in cyberspace expires frequently. It must be updated frequently, thus threatening various routing schemes:
a. Distributed routing: Repetitive re-convergence.
In distributed routing, network nodes distribute topology information to others, locally compute forwarding tables, and eventually reach a global consensus on routing paths (i.e., convergence). Before global routing convergence, there is no guaranteed network reachability. With high mobility, each LEO satellite can only offer very short-lived access for a ground station(<=3 minutes in Starlink). Frequent topology updates cause repetitive routing re-convergence and thus low network usability. For intra-domain routing (e.g., OSPF[RFC2328], IS-IS[RFC1142]), most mega-constellations suffer from low network usability. Even the the size of constellation is samll, the network needs more than fifty seconds to converge after each handoff while using OSPF[TIMESLOT-DIVISION]. For inter-domain routing (e.g., BGP[RFC4271]), [Giuliari20Internet] and [NETWORK-IN-HEAVEN] show frequent logical topology changes cause BGP[RFC4271] re-peering, thus sharpening the instability of global Internet routing.
b. Centralized routing: Repetitive global updates.
In the centralized routing, a ground station predicts the temporal evolution of topology based on satellites’ orbital patterns, divides it into a series of semi-static topology snapshots, schedules the forthcoming global routing tables for each snapshot, and remotely updates the routing tables to all satellites (e.g., via SDN[RFC7426], MPLS[RFC3031], or SRv6[RFC8754]).
LEO mega-constellations pose stresses on centralized routing’s scalability, stability, and complexity. Due to frequent link churns, the topology snapshots and FIBs explode with more satellites, ground stations, and faster mobility.
Moreover, every satellite should locally load these new FIBs upon snapshot changes, which is vulnerable to transient global routing inconsistencies and thus black holes or loops.
Requirements of Addressing in Space-Terrestrial Network
Except from the basic properties like clusterability, network addressing in space-terrestrial network should also meet the following requirements:
Uniqueness
In integrated space-terrestrial networks, each user's address should be globally unique. This property calls for address allocation and duplicate address detection mechanisms.
Stability
This property ensures that the addressing of each node does not change with the movement of users or satellites. With this property, location-based routing will be more stable, avoiding routing convergence caused by the high dynamics of integrated space-terrestrial network. It also reduces the frequency of network addresses updates and reduces the impact on users' network services.
Locality
For any two users or satellites, this property ensures that if their addresses are closer, their actual physical distances should be closer. It not only guarantees the unified logical and physical locations, but also simplifies the design and implementation of location-based routing.
Scalability
The addressing of integrated space-terrestrial network should be designed to accommodate as many satellites and terrestrial nodes as possible. What’s more, it should scale to more satellites and terrestrial users if needed.
Efficiency
In order to avoid frequent addresses updates, the design of addressing in space-terrestrial network should not require static address binding to remote gateways (e.g., carrier-grade NAT). The addressing method should ensure consistent cyber-physical locations, thus easing physically shortest paths without detours.
Backward Compatibility with Terrestrial Internet
To ensure seamless expansion of the global Internet ecosystem, the addressing in space-terrestrial network should be backward compatible with terrestrial Internet. It should be compatible the standard IPv6 addressing formats, and facilitates inter-networking to external networks without modifying terrestrial infrastructure. For backward compatibility with IPv4, we recommend adopting 4over6 transition for integrated space-terrestrial networks.
IANA Considerations
This memo includes no request to IANA.
Security Considerations
The present memo does not introduce any new technology and/or mechanism and as such does not introduce any security threat to the TCP/IP protocol suite.
References
Normative References
[RFC4271]

A Border Gateway Protocol 4 (BGP-4)
DOI 10.17487/RFC4271
RFC 4271
Y. Rekhter
Tony Li
Susan Hares
January 2006
[RFC2328]

OSPF Version 2
DOI 10.17487/RFC2328
RFC 2328
STD 54
J. Moy
April 1998
[RFC0791]

Internet Protocol
DOI 10.17487/RFC0791
RFC 791
STD 5
J. Postel
September 1981
[RFC1142]

OSI IS-IS Intra-domain Routing Protocol
DOI 10.17487/RFC1142
RFC 1142
D. Oran
February 1990
[RFC8754]

IPv6 Segment Routing Header (SRH)
DOI 10.17487/RFC8754
RFC 8754
C. Filsfils, Ed.
D. Dukes, Ed.
S. Previdi
J. Leddy
S. Matsushima
D. Voyer
March 2020
[RFC7426]

Software-Defined Networking (SDN): Layers and Architecture Terminology
DOI 10.17487/RFC7426
RFC 7426
E. Haleplidis, Ed.
K. Pentikousis, Ed.
S. Denazis
J. Hadi Salim
D. Meyer
O. Koufopavlou
January 2015
[RFC3031]

Multiprotocol Label Switching Architecture
DOI 10.17487/RFC3031
RFC 3031
E. Rosen
A. Viswanathan
R. Callon
January 2001
Informative References
[STARLINK]

SpaceX Starlink
[KUIPER]

Amazon receives FCC approval for project Kuiper satellite constellation.
[ONEWEB]

OneWeb constellation
[LOWLATENCY-ROUTING-SPACE]

Delay is Not an Option: Low Latency Routing in Space
Proceedings of the 17th ACM Workshop on Hot Topics in Networks. 2018
M. Handley
[SPACE-RACE]

Gearing up for the 21st century space race
Proceedings of the 17th ACM Workshop on Hot Topics in Networks. 2018
D. Bhattacherjee
W. Aqeel
I. Bozkurt
A. Aguirre
B. Chandrasekaran
P.B. Godfrey
G. Laughlin
B. Maggs
A. Singla
[NETWORK-TOPO-DESIGN]

Network Topology Design at 27,000 km/hour
Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies. 2019
D. Bhattacherjee
A. Singla
[COMPARISON-THREE-CONSTELLATION]

A technical comparison of three low earth orbit satellite constellation systems to provide global broadband
Acta Astronautica, 2019
I.D. Portillo
B.G. Cameron
E.F. Crawley
[ORBITAL-EDGE-COM]

Orbital edge computing: Nanosatellite constellations as a new class of computer system
Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, 2020
D. Bradley
B. Lucia
[IN-ORBIT-COM]

In-orbit computing: An outlandish thought experiment?
Proceedings of the 19th ACM Workshop on Hot Topics in Networks, 2020
D. Bhattacherjee
S. Kassing
M. Licciardello
[ICARUS]

{ICARUS}: Attacking low Earth orbit satellite networks
2021 USENIX Annual Technical Conference (USENIX ATC 21), 2021
G. Giuliari
T. Ciussani
A. Perrig
A. Singla
[STARLINK-CGNAT]

Petition of Starlink Services, LLC for Designation as an Eligible Telecommunication Carrier
[KUIPER-CGNAT]

Amazon Kuiper
[GEO-MOBILE-RADIO-INTERFACE]

GEO-Mobile Radio Interface
[ETSI-TS-101]

ETSI. TS 101 376-1-3: GEO-Mobile Radio Interface Specifications; Part 1: General specifications; Sub-part 3: General System Description
[BGAN]

Broadband Global Area Network (BGAN)
[ETSI-TS-102]

ETSI. TS 102 744-3-6: Satellite Earth Stations and Systems (SES); Part 3: Control Plane and User Plane Specifications; Sub-part 6: Adaptation Layer Operation
[SATELLITE-COMMUNICATIONS]

Satellite communications. McGraw-Hill Education
D. Roddy
[STUDY-NR-SUPPORT]

Study on New Radio (NR) to support nonterrestrial networks
[SOLUTION-NR-NTN]

Solutions for NR to support non-terrestrial networks (NTN)
[Giuliari20Internet]

Internet backbones in space
ACM SIGCOMM Computer Communication Review, 2020, 50(1): 25-37
G. Giuliari
T. Klenze
M. Legner
D. Basin
A. Perrig
A. Singla
[BEIDOU-TEST]

China tests inter-satellite links of BeiDou navigation system
[TheVerge-STARLINK-SPEED]

With latest Starlink launch, SpaceX touts 100 Mbps download speeds and "space lasers" (though the system still has a ways to go)
[SPACEX-CLAIM]

Spacex claims to have redesigned its starlink satellites to eliminate casualty risks
[USE-GROUND-RELAY]

Using ground relays for low-latency wide-area routing in megaconstellations
Proceedings of the 18th ACM Workshop on Hot Topics in Networks. 2019: 125-132
M. Handley
[TEC-SPECI-GROUP-MEETING]

Technical Specification Group Meeting #91E
[NETWORK-IN-HEAVEN]

Networking in heaven as on earth
Proceedings of the 17th ACM Workshop on Hot Topics in Networks. 2018: 22-28
T. Klenze
G. Giuliari
C. Pappas
A. Perrig
D. Basin
[TIMESLOT-DIVISION]

A Timeslot Division Strategy for Availability in Integrated Satellite and Terrestrial Network
2021 IEEE Wireless Communications and Networking Conference (WCNC). IEEE, 2021: 1-7
J. Li
H. Li
J. Liu
Z. Lai
Q. Wu
X. Wang
[lai2021icnp]

Cooperatively constructing cost-effective content distribution networks upon emerging low earth orbit satellites and clouds
2021 IEEE 29th International Conference on Network Protocols (ICNP). IEEE, 2021
Z. Lai
H. Li
Q. Zhang
Q. Wu
J. Wu
[ITU-Measure]

ITU. Measuring digital development: Facts and figures 2020, 2020
[STARLINK-GS-FOUND]

Tesmanian. SpaceX Starlink Gateway Stations Found In The United States and Abroad, 2021
[AZURE-GS]

Microsoft Azure. New Azure Orbital, ground station as a service, now in preview, 2020
[AWS-GS]

Amazon AWS Ground station: Easily control satellites and ingest data with fully managed Ground Station as a Service, 2021
[GOOGLE-DATA-CENTER]

ZDNet. SpaceX to put Starlink ground stations in Google data centres, 2021
[AZURE-CLOUD-STARLINK]

CNBC. Microsoft partners with SpaceX to connect Azure cloud to Musk’s Starlink satellite Internet, 2020
[STARLINK-GS-MAP]

SpaceX Starlink Ground Station Map, 2021
[REDDIT]

Online discussion with Starlink Engineers about satellites’ capability, 2021
[ONEWEB-GS]

OneWeb Gateways Require Complex Hughes Engineering, 2021
[INTERNET-IN-SPACE]

"Internet in Space" for Terrestrial Users via Cyber-Physical Convergence
Proceedings of the 20th ACM Workshop on Hot Topics in Networks. 2021
Y. Li
H. Li
L. Liu
W. Liu
J. Liu
J. Wu
Q. Wu
J. Liu
Z. Lai