Architecture and Load Balancing

A file conversion REST API differs from a regular REST API in one important way: HTTP requests and responses are large. That difference shapes how ConvertAPI's infrastructure routes traffic.

Topology

To manage our infrastructure and facilitate zero downtime deployments, we employ a Kubernetes cluster. However, the traditional setup with a single load balancer distributing the load to worker nodes was not suitable for our needs.

Given that our cluster is spread across the globe (see server locations), a single entry point would introduce considerable latency and reduce traffic throughput. Instead, we run identical and independent worker nodes that handle traffic directly, without any intermediate hops. This makes the service more responsive, faster, and fault-tolerant. The diagram below shows how the pieces fit together - how our system interacts with clients, file servers, and service workers:

Load balancing with GeoDNS

Load balancing is implemented with GeoDNS: the service domain name v2.convertapi.com resolves to the worker node geographically closest to the client, and the client connects directly to the hardware that processes its conversion.

Bypassing intermediary devices means nothing can slow the conversion down in transit or be exploited as an attack vector. Worker nodes are continuously monitored, and a faulty node is excluded from domain name resolution automatically.

Fault tolerance and DNS TTL

Every architectural choice has trade-offs, and GeoDNS balancing is no exception. When a worker node unexpectedly goes offline, a client may keep using the faulty node's IP address for a short period because of DNS caching. Within that window, known as the TTL (Time To Live), requests to the faulty address can time out.

Network issues can also arise on the client's side or anywhere along the path between the client and the worker node. The mitigation is the same: retry failed conversions. Incorporating request retries is good practice for any remote API call - see throttle and fallback for the recommended pattern.