Kinesis Feed

1. Venuenext's ordering app publishes to a kinesis stream in VN's AWS account.
2.  A VN-managed lambda publishes those messages to a stream in your account (that you'd setup).
3.  You would write a lambda and hook it up to your own kinesis stream...and you'd be able to do whatever you want inside your own lambda.

VN likes this setup because you have complete control over your Lambda's business logic, in its own lambda, in its own AWS account.
Also - you can control scale by dialing in the appropriate kinesis shard count.

Controlling Access

There's some IAM configuration to be done to give the VN lambda permission to publish to the stream in your account.
This is something we've done before. Without going deep in the weeds....

You would create a "kinesis writer" IAM role. That role's trust policy will trust the ARN of the VN-lambda role.
You can imagine the psudo-code of the VN lambda looking something like:

credentials = Aws::AssumeRoleCredentials.new(role_arn: "THE-ARN-OF-THE-WRITER-ROLE-YOU-CREATED")
kinesis_client = Aws::Kinesis::Client.new(credentials)

kinesis_client.put_record(stream_name: "your-stream-name", data: <JSON PAYLOAD HERE>})

The special part is that our lambda assumes the role you've prepped in your AWS account before publishing.
Because that role trusts us, it our lambda to publish to a stream in your account.

Implementation Order

You could:

1. Create a kinesis stream in your AWS account.
2. Create a "Kinesis Writer" role. (no trust policy with VN's role for now).
3.  Tell VN:
     a.  The ARN of the kinesis writer role.
     b.  The region of the stream.

Then VN can:

1.  Provision the lambda that will publish to your stream.
2.  Provide you with the ARN of the lambda's role...so you can add it to the "kinesis writer" role's trust policy.

Venuenext’s ordering system (named “Stadium”) publishes messages as orders’ states change. We can integrate those messages to clients’ systems.

How?

A lambda, hosted on VN’s AWS account, consumes the messages published by our ordering system. It can publish those messages to a kinesis stream (in the client’s account). Clients may hook up as many lambdas as they’d like to the stream in their own account.

Requirements

Venuenext

A kinesis stream for the ecosystem/env combination (e.g. mars-dev or levis-prd). A purpose-built lambda whose only responsibility is to publish to the client-hosted kinesis stream.

Client

A client-hosted lambda to consume from a kinesis stream in its own account.

Identity Management / Permissions

Clients must create the following:

• A kinesis stream.
• A consumer lambda bound to that stream, with whatever business logic they want.
• An IAM role or user, used by that lambda to read from the stream and give access to any other AWS resources it needs to acomplish its job. (e.g. S3 permission if the lambda is writing files to S3.)
• A “writer role”, which VN’s lambda will assume, so it can publish across AWS accounts, to the client-hosted kinesis stream. The writer role will grant sts:AssumeRole on the ARN of VN’s lambda.

Sample

{
    "_id": "01egndytjh3xmmt154hxa6dz4m",
    "_v": 1,
    "o": "cubs",
    "u": "1d1c41ee-a875-43dd-ad9c-4375256190de",
    "s": "stadium",
    "e": "order.state.change",
    "p": {
        "uuid": "b120fdbf-9b4e-44a8-99fd-4e21e3709c1f",
        "venue_uuid": "e13b6c22-fdcf-4446-ba25-c240bc9bc40b",
				"confirmation_number": "1234",
        "state": "completion_pending",
        "created_at": "2020-08-26T01:02:46Z",
        "total": 7500,
        "tax": 250,
        "discount": "0",
        "tip": 1250,
        "device_uuid": "817c505d-d620-467b-b8a7-77ba9cd2fd30",
        "revenue_center_name": "Suite 101",
        "revenue_center_external_ref_id": "external-suite-101",
        "revenue_center_uuid": "c30550a3-4f87-4a7d-8fd9-22afc2bdbbdb",
        "menu_name": "Day of Event",         "menu_external_ref_id": "external-day-of-event",
        "service_type": "POS",
        "employee_name": null,
        "employee_id": null,
        "transaction_history": [{
            "state": "authorized",
            "time": "2020-08-26T01:02:47Z"
        }],
        "state_history": [{
            "state": "tabbed",
            "time": "2020-08-26T01:02:46Z"
        }, {
            "state": "processing",
            "time": "2020-08-26T02:28:12Z"
        }, {
            "state": "completion_pending",
            "time": "2020-08-26T02:28:13Z"
        }],
        "line_items": [{
            "uuid": "6d268953-85c8-4db4-8091-33676e890c6f",
            "name": "Sausage Sampler",
            "price": 1000,
            "sku": "5001",
            "created_at": "2020-08-26T01:02:46Z",
            "refunded_at": null,
            "removed_at": null,
            "modified_line_item_uuid": null,
            "promotions": [{"discount_amount_in_cents": 0, "uuid": "8e8c197e-9b2b-4f40-a528-ce28f51404d7"}]
        }, {
            "uuid": "8aed311c-3c2d-46d8-9d63-c6d218491c1d",
            "name": "Sausage Sampler",
            "price": 1000,
            "sku": "5001",
            "created_at": "2020-08-26T01:02:46Z",
            "refunded_at": null,
            "removed_at": null,
            "modified_line_item_uuid": null,
            "promotions": []
        }, {
            "uuid": "038418fc-59b8-4b91-b8f7-b5d71ef2bbe1",
            "name": "Sausage Sampler",
            "price": 1000,
            "sku": "5001",
            "created_at": "2020-08-26T01:02:46Z",
            "refunded_at": null,
            "removed_at": null,
            "modified_line_item_uuid": null,
            "promotions": []
        }, {
            "uuid": "a4b862dc-bb73-4ae0-a805-5ddd25e6d7a7",
            "name": "Sausage Sampler",
            "price": 1000,
            "sku": "5001",
            "created_at": "2020-08-26T01:02:46Z",
            "refunded_at": null,
            "removed_at": null,
            "modified_line_item_uuid": null,
            "promotions": []
        }, {
            "uuid": "d888d272-2260-4761-8809-5a7ab92a1ec1",
            "name": "Sausage Sampler",
            "price": 1000,
            "sku": "5001",
            "created_at": "2020-08-26T01:02:46Z",
            "refunded_at": null,
            "removed_at": null,
            "modified_line_item_uuid": null,
            "promotions": []
        }, {
            "uuid": "887c26fd-0669-4d78-9319-ab46dbf3b86d",
            "name": "Service Charge",
            "price": 1000,
            "sku": "8374626",
            "created_at": "2020-08-26T01:02:46Z",
            "refunded_at": null,
            "removed_at": null,
            "modified_line_item_uuid": null,
            "promotions": []
        }]
    }
}

Data Types

{
    "_id": standard ULID string
    "_v": short integer
    "o": 256 character string
    "u": 36 character UUID
    "s": short string, currently always “stadium”
    "e": 256 character string
    "p": {
        "uuid": 36 character UUID string
        "state": 256 character string
        "created_at": 256 character date string
        "total": 16-bit integer total in cents
        "tax": 16-bit integer tax in cents
        "discount": 256 character string discount amount
        "tip": 16-bit integer tax in cents
        "device_uuid": 36 character UUID string
        "revenue_center_name": 256 character string
        "revenue_center_external_ref_id": 256 character string
        "menu_name": 256 character string
        "menu_external_ref_id": 256 character string
        "service_type": short string
        "employee_name": 256 character string
        "employee_id": 256 character string
        "transaction_history": [{
            "state": 256 character string
            "time": 256 character date string
        }],
        "state_history": [{
            "state": 256 character string
            "time": 256 character date string
        }],
        "line_items": [{
            "uuid": 36 character UUID string
            "name": 256 character string
            "price": 16-bit integer price in cents
            "sku": 256 character string
            "created_at": 256 character date string
            "refunded_at": 256 character date string
            "removed_at": 256 character date string
            "modified_line_item_uuid": 36 character UUID string
        }]
    }
}

As outlined above, each payload contains a state history array (of objects). There are timestamps included with each state transition. If that array contains the same elements AND the most recent timestamps are the same, then it can be presumed that the order hasn’t had a state change.

Another way to do this “quickly” if one wanted a “marker” would be to hash a sorted (by date) copy of that array as a string. An incoming state change with the same transition hash would be a duplicate.