High-Level Architecture¶

The architecture consists of three main components:

• the Experience API,
• the Process API,
• and the System API.

Each component has its own responsibilities and interacts with the other components to fulfill user/client requests, following the Integration Hub's XAPI → PAPI → SAPI pattern. Each layer is implemented as a separate AWS Lambda function, built on an internal framework called ipaas-framework.

A single request from Sapio passes synchronously down through all three layers and back, as illustrated in the diagram below.

• XAPI (inthub-xapi-sapio-studies) – the entry point Sapio calls.
• PAPI (inthub-papi-mlwh-studies) – a routing/orchestration layer with no business logic of its own.
• SAPI (inthub-sapi-sequencescape) – a thin adapter that talks to the real Sequencescape application over HTTPS.

Supported Operations¶

The architecture below is shared by every operation * only the validation rules and downstream target change per operation.

Architecture Diagram¶

flowchart TD

    A[Client/ Sapio]

    subgraph XAPI["Experience API"]
        direction TD
        X1[Validate<br/>Request]
        X2[Authenticate<br/>& Authorise]
        X3[Normalise<br/>Parameters]
        X4[Route to<br/>Process API]
    end

    subgraph PAPI["Process API"]
        direction TD
        P1[Build<br/>SQL Query]
        P2[Apply<br/>Pagination]
        P3[Transform<br/>Request Model]
        P4[Route to<br/>System API]
    end

    subgraph SAPI_SS["System API: SequenceScape"]
        direction TD
        SS1[Build HTTPS<br/>Request to<br/>SequenceScape API]
        SS2[Execute<br/>Query]
        SS3[Return<br/>JSON<br/>Response]
    end

    subgraph SAPI_MLWH["System API: MLWH"]
        direction TD
        SM1[Execute<br/>Query]
        SM2[Map Database<br/>Rows]
        SM3[Return<br/>JSON<br/>Response]
    end

    DB[(MLWH<br/>Database)]

    A --> XAPI
    XAPI --> PAPI
    PAPI --> SAPI_MLWH
    PAPI --> SAPI_SS
    SAPI_MLWH --> DB

• Each layer is responsible for validating, transforming, and routing the request to the next layer, until it reaches the Sequencescape or MLWH application.
• Every hop is a direct, blocking Lambda invocation (RequestResponse), so the response flows back up through the same three layers in reverse.

Components¶

XAPI — inthub-xapi-sapio-studies¶

• Purpose: exposes the public-facing surface Sapio calls: GET /sapio/studies/v1 (lookup by name) and POST /sapio/studies/v1 (create a study record).
• Responsibilities: validates the inbound request shape, invokes PAPI, and reshapes PAPI's response (JMESPath transform) into the shape Sapio expects.
• Dependencies: the PAPI Lambda, which is the only downstream target for this route.

PAPI — inthub-papi-mlwh-studies¶

• Purpose: a routing layer — it contains no domain logic, only maps an inbound route to the correct downstream SAPI Lambda.
• Responsibilities: for the Sapio-facing route (/sapio/studies/v1), forwards directly to the SAPI/Sequencescape adapter.
• Dependencies: the SAPI/Sequencescape and SAPI/MLWH Lambdas, which are the downstream targets for this route.

SAPI — inthub-sapi-sequencescape¶

• Purpose: adapter between PAPI and the real Sequencescape Rails application — not Sequencescape itself.
• Responsibilities: builds an authenticated HTTPS request to Sequencescape's /api/v2/sapio/studies JSON:API endpoint, and normalises both success and error responses back into a consistent shape for PAPI.
• Dependencies: the Sequencescape application, which is the downstream target for this route.

Data Flow¶

Every operation shares the same shape: a single blocking round trip down through XAPI → PAPI → SAPI and back, with no retry or fallback path today. * A failure or timeout at any layer surfaces as an error at every layer above it. What differs per operation is the validation applied and the downstream target. Each operation gets its own breakdown below.

GET — look up a study by name¶

1. Sapio sends GET /sapio/studies/v1?name=... to the shared API Gateway.
2. The custom Lambda authorizer validates the caller before the request reaches XAPI.
3. XAPI validates the name query parameter, then synchronously invokes PAPI with the request.
4. PAPI routes the request to the SAPI (Sequencescape) Lambda.
5. SAPI calls Sequencescape's /api/v2/sapio/studies over HTTPS, authenticating with a secret-stored client-ID header and CA certificate.
6. Sequencescape's response travels back up unchanged in shape until SAPI normalises it, then PAPI and XAPI each pass it back untouched to the caller.

POST — create a study record¶

1. Sapio sends POST /sapio/studies/v1 with a JSON body to the shared API Gateway.
2. The custom Lambda authorizer validates the caller before the request reaches XAPI.
3. Tracking ID is generated in XAPI and attached onto the body for downstream logging and tracing.
4. The enriched body (all fields + event_uuid) is published to SNS, and XAPI returns a 202 Accepted response to Sapio immediately. The request is now asynchronous, and the caller does not wait for the downstream processing to complete.
5. PAPI is then invoked asynchronously by the SNS subscription, and it routes the request to the Sequencescape SAPI.
6. The Sequencescape SAPI calls Sequencescape's /api/v2/sapio/studies by extracting only name and uuid from the body.
7. On Sequencescape success, PAPI then invokes the MLWH SAPI to create a matching record in the sequencing data warehouse.
8. On MLWH success, PAPI invokes the Sapio SAPI to confirm back to Sapio that the study now exists in Sequencescape and MLWH — Sapio is the originator of the request and the recipient of this confirmation, not a third place the study record is created.
9. If Sequencescape, MLWH, or this confirmation back to Sapio fails, the failure is recorded, an email alert is sent to the configured user group, and the record is pushed to a dead-letter queue for reprocessing. The caller already received its 202 before any of this happens, so failures are only visible through these channels, not in the original response.

Because the POST flow branches on success/failure at each of the three downstream systems, and because Sapio's own response comes back long before that branching happens, it's worth seeing as a diagram rather than prose alone:

sequenceDiagram
    participant Sapio
    participant XAPI
    participant PAPI
    participant Sequencescape
    participant MLWH
    participant Notifications as Notification Service

    Sapio->>XAPI: POST /sapio/studies/v1
    XAPI->>XAPI: Validate, generate event_uuid
    XAPI-->>Sapio: 202 Accepted (event_uuid)
    XAPI->>PAPI: Publish via SNS (async)

    PAPI->>Sequencescape: Create study
    alt Sequencescape succeeds
        PAPI->>MLWH: Insert study row
        alt MLWH succeeds
            PAPI->>Sapio: Confirm study created
            alt Confirmation fails
                PAPI->>Notifications: Send failure alert
                PAPI->>PAPI: Push to dead-letter queue
            end
        else MLWH fails
            PAPI->>Notifications: Send failure alert
            PAPI->>PAPI: Push to dead-letter queue
        end
    else Sequencescape fails
        PAPI->>Notifications: Send failure alert
        PAPI->>PAPI: Push to dead-letter queue
    end

External Dependencies¶

• Sequencescape – the system of record for study data, called by SAPI over HTTPS.
• Shared API Gateway + custom Lambda authorizer – owned by another Integration Hub component; provides the only application-level authentication in the whole chain.
• ipaas-framework – the internal Integration Hub framework all three Lambdas are built on.
• AWS Secrets Manager – holds the Sequencescape client-ID and CA certificate used by SAPI.
• AWS SSM Parameter Store – resolves shared infra values (VPC networking, authorizer ID, Lambda layer ARN) at deploy time.