Add integration tests of offline extracts with C++ Valhalla

.github/workflows/integration.yml

@@ -0,0 +1,50 @@
+# Integration tests of offline extracts with C++ Valhalla. See scripts/integration/README.md.
+name: Integration
+
+on:
+  pull_request:
+    branches: [ "main" ]
+    paths:
+      - "valhalla-graphtile/src/extract/**"
+      # The tar layout and the index.bin code are in the tile providers.
+      - "valhalla-graphtile/src/tile_provider/**"
+      - "valinor-cli/**"
+      - "scripts/integration/**"
+      - ".github/workflows/integration.yml"
+  workflow_dispatch:
+
+env:
+  CARGO_TERM_COLOR: always
+  # The Valhalla release under test. Without this pin, the harness uses `latest`.
+  # When you change it, examine the differences in the responses and logs before you merge.
+  VALHALLA_VERSION: "3.8.2"
+
+jobs:
+  integration:
+    # Linux only: the job tests the Docker image, not the platform.
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v6
+
+    - uses: astral-sh/setup-uv@v10.3.0
+
+    - name: Check the harness
+      working-directory: scripts/integration
+      run: |
+        uv run --locked ruff check
+        uv run --locked ruff format --check
+        uv run --locked pytest
+
+    - name: Run the integration tests with C++ Valhalla
+      run: uv run --locked --project scripts/integration valinor-integration
+
+    - name: Upload harness artifacts on failure
+      if: failure()
+      uses: actions/upload-artifact@v4
+      with:
+        name: integration-workdir
+        path: |
+          target/integration/responses
+          target/integration/valhalla-*.log
+          target/integration/valhalla.json

Cargo.toml

@@ -54,6 +54,7 @@ serde = { version = "1.0.229", features = ["derive"] }
 serde_with = "3.24.0"
 serde_json = "1.0.154"
 tar = "0.4.46"
+tempfile = "3.27.0"
 thiserror = "2.0.21"
 tokio = { version = "1.54.1", features = ["macros", "rt-multi-thread", "fs", "io-util", "sync"] }
 tracing = "0.1.44"

scripts/integration/.gitignore

@@ -0,0 +1,3 @@
+.venv/
+__pycache__/
+.pytest_cache/

scripts/integration/README.md

@@ -0,0 +1,94 @@
+# Integration tests with C++ Valhalla
+
+This harness makes sure that a sub-extract from `valinor-cli extract`
+works in stock C++ Valhalla the same way as the full extract.
+
+The harness does these steps:
+
+1. It writes a sub-extract of the full extract with `valinor-cli extract`.
+   The tiles are in [`sub-extract-tiles.txt`](sub-extract-tiles.txt).
+   The sub-extract must be a proper subset of the full extract,
+   or the comparison would prove nothing.
+2. It starts two `valhalla_service` containers with docker-py:
+   one for the full extract and one for the sub-extract.
+   The image's own `valhalla_build_config` makes the config.
+   The harness sets `mjolnir.shortcut_caching` to `false`,
+   because Valhalla stops at startup with a partial extract when it is `true`.
+3. It sends each request of the [Bruno collection](bruno/) to both servers.
+   It saves each response, and it compares the two responses of each request
+   (see [`checks.py`](src/valinor_integration/checks.py) for the fields that it compares).
+4. It makes sure that the logs of both servers contain no `[ERROR]` lines.
+
+The default full extract is the Andorra fixture
+(`valhalla-graphtile/fixtures/andorra-tiles.tar`).
+
+## Requirements
+
+- [uv](https://docs.astral.sh/uv/)
+- Docker (Docker Desktop, colima, or a Docker daemon on Linux)
+- cargo, if you do not set `VALINOR_CLI`
+
+## Run the test
+
+From the root of the repository:
+
+```bash
+uv run --project scripts/integration valinor-integration
+```
+
+The command prints `integration: PASS`, or a list of failures with a diff for each difference.
+The work directory (`target/integration` by default) contains:
+
+- `sub-extract.tar`: the sub-extract.
+- `valhalla.json`: the Valhalla config.
+- `responses/full/` and `responses/sub/`: the raw response of each request.
+- `valhalla-full.log` and `valhalla-sub.log`: the logs of the two servers.
+
+Use `--help` for the options (a different full extract, work directory, tile list, or collection).
+
+## Send requests by hand with Bruno
+
+```bash
+uv run --project scripts/integration valinor-integration serve
+```
+
+This serves the full extract at `http://localhost:8002`
+and the sub-extract at `http://localhost:8003` until you push Ctrl-C.
+Open `scripts/integration/bruno` in [Bruno](https://www.usebruno.com/),
+and select the `full` or the `sub` environment.
+
+## Environment variables
+
+All variables are optional.
+
+| Variable | Default | Description |
+| --- | --- | --- |
+| `VALHALLA_VERSION` | `latest` | The tag of the Valhalla image, such as `3.8.2`. CI sets it in [`integration.yml`](../../.github/workflows/integration.yml). If it is not set, the harness uses `latest` and logs a warning. |
+| `VALHALLA_IMAGE` | `ghcr.io/valhalla/valhalla` | The image repository. |
+| `VALINOR_CLI` | (none) | The path of a `valinor-cli` binary. If it is not set, the harness runs `cargo run --release -p valinor-cli`. |
+| `VALHALLA_PORT` | `8002` | The port of the full extract in `serve` mode. The sub-extract uses the next port. If you change it, change the Bruno environments too. |
+
+## Add a request
+
+1. Add the request to the Bruno collection.
+   You can do this in the Bruno app, or you can copy a file in `bruno/route/`.
+   Keep all coordinates far inside the tiles of the sub-extract,
+   so that the request never needs a tile which the sub-extract does not contain.
+2. If the request uses a new endpoint, add a check for the endpoint to
+   [`checks.py`](src/valinor_integration/checks.py).
+   The harness refuses a request for an endpoint that has no check.
+
+The harness reads a subset of the Bruno YAML format:
+the method, the URL (with `{{baseUrl}}`), headers, and a JSON body.
+It refuses requests with Bruno scripts or assertions,
+because it does not run them; put such checks in `checks.py`.
+
+## Unit tests of the harness
+
+```bash
+uv run --project scripts/integration pytest
+```
+
+```bash
+uv run --project scripts/integration ruff check
+```

scripts/integration/bruno/environments/full.yml

@@ -0,0 +1,4 @@
+name: full
+variables:
+  - name: baseUrl
+    value: http://localhost:8002

scripts/integration/bruno/environments/sub.yml

@@ -0,0 +1,4 @@
+name: sub
+variables:
+  - name: baseUrl
+    value: http://localhost:8003

scripts/integration/bruno/locate/andorra-la-vella-center.yml

@@ -0,0 +1,28 @@
+info:
+  name: andorra-la-vella-center
+  type: http
+  seq: 1
+
+http:
+  method: POST
+  url: "{{baseUrl}}/locate"
+  body:
+    type: json
+    data: |-
+      {
+        "locations": [
+          {
+            "lat": 42.5063,
+            "lon": 1.5218
+          }
+        ],
+        "costing": "auto",
+        "verbose": true
+      }
+  auth: inherit
+
+settings:
+  encodeUrl: true
+  timeout: 0
+  followRedirects: true
+  maxRedirects: 5

scripts/integration/bruno/locate/folder.yml

@@ -0,0 +1,8 @@
+info:
+  name: locate
+  type: folder
+  seq: 2
+
+request:
+  auth: inherit
+

scripts/integration/bruno/opencollection.yml

@@ -0,0 +1,10 @@
+opencollection: 1.0.0
+
+info:
+  name: Valinor integration
+bundled: false
+extensions:
+  bruno:
+    ignore:
+      - node_modules
+      - .git

scripts/integration/bruno/route/andorra-la-vella-center-pedestrian.yml

@@ -0,0 +1,32 @@
+info:
+  name: andorra-la-vella-center-pedestrian
+  type: http
+  seq: 5
+
+http:
+  method: POST
+  url: "{{baseUrl}}/route"
+  body:
+    type: json
+    data: |-
+      {
+        "locations": [
+          {
+            "lat": 42.5063,
+            "lon": 1.5218
+          },
+          {
+            "lat": 42.5103,
+            "lon": 1.5336
+          }
+        ],
+        "costing": "pedestrian",
+        "units": "kilometers"
+      }
+  auth: inherit
+
+settings:
+  encodeUrl: true
+  timeout: 0
+  followRedirects: true
+  maxRedirects: 5

scripts/integration/bruno/route/andorra-la-vella-to-encamp-auto.yml

@@ -0,0 +1,32 @@
+info:
+  name: andorra-la-vella-to-encamp-auto
+  type: http
+  seq: 1
+
+http:
+  method: POST
+  url: "{{baseUrl}}/route"
+  body:
+    type: json
+    data: |-
+      {
+        "locations": [
+          {
+            "lat": 42.5063,
+            "lon": 1.5218
+          },
+          {
+            "lat": 42.536,
+            "lon": 1.5836
+          }
+        ],
+        "costing": "auto",
+        "units": "kilometers"
+      }
+  auth: inherit
+
+settings:
+  encodeUrl: true
+  timeout: 0
+  followRedirects: true
+  maxRedirects: 5

scripts/integration/bruno/route/andorra-la-vella-to-la-massana-auto.yml

@@ -0,0 +1,32 @@
+info:
+  name: andorra-la-vella-to-la-massana-auto
+  type: http
+  seq: 2
+
+http:
+  method: POST
+  url: "{{baseUrl}}/route"
+  body:
+    type: json
+    data: |-
+      {
+        "locations": [
+          {
+            "lat": 42.5078,
+            "lon": 1.5211
+          },
+          {
+            "lat": 42.5449,
+            "lon": 1.5148
+          }
+        ],
+        "costing": "auto",
+        "units": "kilometers"
+      }
+  auth: inherit
+
+settings:
+  encodeUrl: true
+  timeout: 0
+  followRedirects: true
+  maxRedirects: 5

scripts/integration/bruno/route/encamp-to-canillo-auto.yml

@@ -0,0 +1,32 @@
+info:
+  name: encamp-to-canillo-auto
+  type: http
+  seq: 3
+
+http:
+  method: POST
+  url: "{{baseUrl}}/route"
+  body:
+    type: json
+    data: |-
+      {
+        "locations": [
+          {
+            "lat": 42.536,
+            "lon": 1.5836
+          },
+          {
+            "lat": 42.5676,
+            "lon": 1.5976
+          }
+        ],
+        "costing": "auto",
+        "units": "kilometers"
+      }
+  auth: inherit
+
+settings:
+  encodeUrl: true
+  timeout: 0
+  followRedirects: true
+  maxRedirects: 5

scripts/integration/bruno/route/folder.yml

@@ -0,0 +1,8 @@
+info:
+  name: route
+  type: folder
+  seq: 1
+
+request:
+  auth: inherit
+