Understanding Pagination

Learn the difference between offset and cursor-based pagination, and when to use each approach.

Why Pagination Matters

The Vail ReRBN database contains hundreds of thousands of spots. Returning all matching results in a single response would be slow, consume excessive bandwidth, and potentially crash clients. Pagination allows you to retrieve data in manageable chunks.

Two Pagination Methods

Offset Pagination

Best for: Historical browsing, page-by-page navigation

Database (newest to oldest):

1
2
3
4
5
6
7
8
9
10
Request 1
offset=0, limit=3
1
2
3
Request 2
offset=3, limit=3
4
5
6

Pros

  • Simple to understand
  • Can jump to any page
  • Good for static datasets

Cons

  • Can miss/duplicate spots if data changes
  • Slower for large offsets
  • Not ideal for real-time data

Cursor-Based Pagination (after_id)

Best for: Real-time monitoring, continuous polling

Timeline (old → new):

ID:100
ID:101
ID:102
Cursor
ID:103
ID:104
ID:105
Initial Request
limit=3
ID:100
ID:101
ID:102
Next Request (5 sec later)
after_id=102, limit=3
ID:103
ID:104
ID:105

Pros

  • Never misses new data
  • Perfect for real-time streaming
  • Consistent performance

Cons

  • Can't jump to arbitrary positions
  • Must track last seen ID
  • Only moves forward in time

Which Method Should I Use?

Scenario Use offset Use after_id
Real-time monitoring / Live feed ❌ ✅
Historical browsing / Data browser ✅ ❌
Continuous polling ❌ ✅
Need page numbers (e.g., "Page 5 of 10") ✅ ❌

Code Examples

Offset Pagination

Offset pagination 
# Page 1
curl "https://vailrerbn.com/api/v1/spots?limit=100&offset=0"

# Page 2
curl "https://vailrerbn.com/api/v1/spots?limit=100&offset=100"

# Page 3
curl "https://vailrerbn.com/api/v1/spots?limit=100&offset=200"
Offset pagination 
import requests

page_size = 100
page = 1

while True:
    offset = (page - 1) * page_size

    response = requests.get(
        "https://vailrerbn.com/api/v1/spots",
        params={
            "limit": page_size,
            "offset": offset
        }
    )
    data = response.json()

    print(f"Page {page}: {len(data['spots'])} spots")

    # Process spots
    for spot in data["spots"]:
        print(f"  {spot['callsign']} on {spot['frequency']} kHz")

    # Check if we have more pages
    if offset + len(data["spots"]) >= data["total"]:
        print("No more pages")
        break

    page += 1
Offset pagination 
const pageSize = 100;
let page = 1;

while (true) {
  const offset = (page - 1) * pageSize;

  const response = await fetch(
    `https://vailrerbn.com/api/v1/spots?limit=${pageSize}&offset=${offset}`
  );
  const data = await response.json();

  console.log(`Page ${page}: ${data.spots.length} spots`);

  // Process spots
  for (const spot of data.spots) {
    console.log(`  ${spot.callsign} on ${spot.frequency} kHz`);
  }

  // Check if we have more pages
  if (offset + data.spots.length >= data.total) {
    console.log('No more pages');
    break;
  }

  page++;
}
Offset pagination 
<?php
$pageSize = 100;
$page = 1;

while (true) {
    $offset = ($page - 1) * $pageSize;

    $url = "https://vailrerbn.com/api/v1/spots?" . http_build_query([
        'limit' => $pageSize,
        'offset' => $offset
    ]);
    $response = file_get_contents($url);
    $data = json_decode($response, true);

    echo "Page $page: " . count($data['spots']) . " spots\n";

    // Process spots
    foreach ($data['spots'] as $spot) {
        echo "  {$spot['callsign']} on {$spot['frequency']} kHz\n";
    }

    // Check if we have more pages
    if ($offset + count($data['spots']) >= $data['total']) {
        echo "No more pages\n";
        break;
    }

    $page++;
}
Offset pagination 
require 'net/http'
require 'json'

page_size = 100
page = 1

loop do
  offset = (page - 1) * page_size

  uri = URI('https://vailrerbn.com/api/v1/spots')
  uri.query = URI.encode_www_form({
    limit: page_size,
    offset: offset
  })

  response = Net::HTTP.get(uri)
  data = JSON.parse(response)

  puts "Page #{page}: #{data['spots'].length} spots"

  # Process spots
  data['spots'].each do |spot|
    puts "  #{spot['callsign']} on #{spot['frequency']} kHz"
  end

  # Check if we have more pages
  if offset + data['spots'].length >= data['total']
    puts 'No more pages'
    break
  end

  page += 1
end

Browse through historical spots page by page

Cursor-Based Pagination (Real-Time)

Real-time monitoring with after_id 
# Not practical with curl - use Python or JavaScript for continuous polling
Real-time monitoring with after_id 
import requests
import time

last_id = 0

print("Monitoring for W1AW spots (Ctrl+C to stop)...")

while True:
    params = {"limit": 500, "call": "W1AW"}
    if last_id > 0:
        params["after_id"] = last_id

    response = requests.get("https://vailrerbn.com/api/v1/spots", params=params)
    data = response.json()

    for spot in data["spots"]:
        grid = spot.get('grid', '????')
        print(f"{spot['timestamp']} - {spot['callsign']} ({grid}) on {spot['frequency']} kHz by {spot['spotter']}")
        last_id = max(last_id, spot["id"])

    time.sleep(5)  # Poll every 5 seconds
Real-time monitoring with after_id 
let lastId = 0;

console.log('Monitoring for W1AW spots (Ctrl+C to stop)...');

async function pollSpots() {
  const params = new URLSearchParams({
    limit: 500,
    call: 'W1AW'
  });

  if (lastId > 0) {
    params.set('after_id', lastId);
  }

  const response = await fetch(`https://vailrerbn.com/api/v1/spots?${params}`);
  const data = await response.json();

  for (const spot of data.spots) {
    const grid = spot.grid || '????';
    console.log(`${spot.timestamp} - ${spot.callsign} (${grid}) on ${spot.frequency} kHz by ${spot.spotter}`);
    lastId = Math.max(lastId, spot.id);
  }
}

// Poll every 5 seconds
setInterval(pollSpots, 5000);
pollSpots(); // Initial call
Real-time monitoring with after_id 
<?php
$lastId = 0;

echo "Monitoring for W1AW spots (Ctrl+C to stop)...\n";

while (true) {
    $params = ['limit' => 500, 'call' => 'W1AW'];
    if ($lastId > 0) {
        $params['after_id'] = $lastId;
    }

    $url = "https://vailrerbn.com/api/v1/spots?" . http_build_query($params);
    $response = file_get_contents($url);
    $data = json_decode($response, true);

    foreach ($data['spots'] as $spot) {
        echo "{$spot['timestamp']} - {$spot['callsign']} on {$spot['frequency']} kHz by {$spot['spotter']}\n";
        $lastId = max($lastId, $spot['id']);
    }

    sleep(5); // Poll every 5 seconds
}
Real-time monitoring with after_id 
require 'net/http'
require 'json'

last_id = 0

puts 'Monitoring for W1AW spots (Ctrl+C to stop)...'

loop do
  uri = URI('https://vailrerbn.com/api/v1/spots')
  params = { limit: 500, call: 'W1AW' }
  params[:after_id] = last_id if last_id > 0
  uri.query = URI.encode_www_form(params)

  response = Net::HTTP.get(uri)
  data = JSON.parse(response)

  data['spots'].each do |spot|
    puts "#{spot['timestamp']} - #{spot['callsign']} on #{spot['frequency']} kHz by #{spot['spotter']}"
    last_id = [last_id, spot['id']].max
  end

  sleep 5 # Poll every 5 seconds
end

Continuously poll for new spots using cursor pagination