Event API Firehose

Configuration guide for the Event API Firehose feature, which allows you to export and stream usage tracking data via AWS S3 or custom webhooks.

Overview

Required plan

This feature or setting is available to all customers on any Recurly Engage subscription plan.

Prerequisites & limitations

  • You must have Company or App Administrator permissions in Recurly Engage.
  • add additional prerequisites or limitations if any

Definition

The Event API Firehose feature pushes usage tracking events in JSON format to external systems—either by writing to an S3 bucket or by POSTing to a custom web API endpoint—enabling real-time or batch consumption of event data.

Key benefits

  • Real-time insights: Stream usage events as they occur for immediate data-driven decisions.
  • Scalable storage: Offload event logs to S3 for long-term retention and BI integration.
  • Flexible integration: Send data to any HTTP endpoint or cloud storage service to fit your architecture.

Key details

S3 Event Data Logs

Usage tracking data that has been enabled in Recurly Engage can be pushed to external systems. Anything added in the Usage Tracker section is sent as an event once the integration is set up.

There are two ways to receive usage data. You must provide the following values to your account manager:

AWS S3 Requirements – Pull data using Recurly Engage:

  • Go to Settings > User Traits.
  • Select “Click here for AWS S3 credentials”.
  • Copy the AWS Bucket, Access Key, and Secret Key, then navigate to the exports folder. All filenames start with the usages prefix. Pull this data on demand.

AWS S3 data specifications

All data is JSON formatted with the following fields. To enable optional fields, ask your account manager:

FieldDescriptionNotes
user_idPrimary user identifier
anonymous_user_idIdentifier when userId is unavailable
eventName of event
platformType of integrationAlways redfast for usage
typeType of trackerValues: page, track, custom
properties.idUsage Tracker id
properties.valuesConfigured values in Recurly Engage Console
properties.optionsAdditional optionsSpecifies if regex is used for page trackers
tsTimestamp when activity occurred (epoch)
app_idRecurly Engage app idOptional
traitsKey/value pairs of user attributes ingested from external and in-app sourcesOptional
segmentsList of segment objects (id + name) at event timeOptional
pathsList of personalization path objects (id, name, device_type, zone)Optional

Event types

TypeDescription
ImpressionUser was shown the prompt
TimeoutUser did not respond before timer expired
DismissUser dismissed by clicking X or outside the window
DeclineUser clicked on the decline link
ClickUser accepted the prompt (phased out in favor of Goal)
GoalUser accepted the prompt
Custom_Goals_[Activity Type]User completed a defined custom goal on a prompt
ExcludeUser excluded due to holdout or user limits
HoldoutUser in holdout group
Pipeline_ImpressionUser shown the prompt and is in a pipeline stage
Pipeline_TransitionUser moved from one pipeline stage to another

Custom Web API Firehose

This option provides near-real-time delivery. Recurly Engage can send single or batched events within sub-minute windows to an endpoint you specify. Provide your account manager with:

  • URL
  • API Key / Access Token
  • Request type (GET, POST, PUT)

Payload format

The payload is a JSON array of event objects. In S3 each event is one object per row.

[  
  { /* event object as shown below */ },  
  { /* ... */ }  
]

Example event payload

{
  "app_id": "74f72649-0327-4911-bd21-a4cd533cec1c",
  "user_id": "123",
  "anonymous_user_id": "2528a84d8fa8…",
  "event": "Home Page",
  "platform": "redfast",
  "type": "page",
  "ts": 1630476856,
  "properties": {
    "id": "8e282c72-07da-4c2f-bf91-85df633828af",
    "values": [{ "url_hash": "", "url_path": "/", "query_params": "" }],
    "options": { "use_regex": false }
  },
  "traits": { /* user trait key/value pairs */ },
  "segments": [{ "id": "...", "name": "Engaged" }, /* ... */],
  "paths": [{ "id": "...", "name": "Prompt A", "device_type": "web", "zone": "banner" }, /* ... */]
}

API data specifications

FieldDescriptionNotes
app_idRecurly Engage app id
user_idPrimary user identifier
anonymous_user_idIdentifier when userId is unavailable
eventName of event
platformAlways redfast
typepage, track, or custom
properties.idUsage Tracker id
properties.valuesConfigured tracker values
properties.optionsRegex or other options
tsEpoch timestamp
traitsUser attributes (key/value pairs)Optional
segmentsSegment list (id + name)Optional
pathsPersonalization path list (id, name, device_type, zone)Optional

Usage types

TypeDescription
pagePage view tracker
trackButton or CSS class click
customCustom event (e.g., transaction complete)

API Push example

curl 'example.backendurl.com/v1/api/ingest/' \
  -X POST \
  -H 'Content-Type: application/json' \
  -H 'your-api-key: abcdef' \
  --data-raw '[ { /* event objects */ } ]'