Event Streams
Service Description
Event streams provide notification of changes to authoritative data, for consumption and syncing with dependent downstream systems.
Key Concepts
- Client Integration
The Event Streams service can be accessed from a native Kafka client or via a RESTful API. NOTE: Certain event streams require granular business rule filtering for assurance purposes, and those may only be available via the RESTful API. Please refer to the "Client Availability" designation in the Service Catalog section. Additional information is available in the Events Streams - Client Integration Guide. - Pull Technology
The Event Streams service follows the publish/subscribe model in delivering events from source to client-facing topics. Clients are required to 'pull' events from those topics using the integration strategy of their choosing, per the options described in the Client Integration Guide. - Targeted Objects
The objects delivered via each event stream are configured to house the minimal set of attributes necessary to describe the event and to provide the data pertinent to the change. - Activity Types
- Create - A new record was created at the source system.
- Delete - A record was deleted from the source system.
- Replace - A record was updated at the source system.
- Significant Changes
A replace event will only be published when a significant attribute was changed within the impacted data object, at the source. The significant attributes for each event type are documented in the service catalog below. Other extraneous and/or metadata updates, that pertain only to the source system, will be ignored. All create and delete events are considered significant changes, and thus will trigger an event stream event. - Key Data
Producers and consumers of event stream data must have a shared understanding of the data attribute(s) that uniquely identifies the affected data. The key data for each event stream object is included in the service catalog below. For objects having a multi-attribute key, all key data must be matched to ensure downstream data integrity. - Authoritative Source
Each event stream object will represent a single authoritative source, or source of truth, for the data therein. Authoritative source for each object is reflected in the service catalog below. - Schema
Producers and consumers of event stream data must have a shared understanding of the data object structures and attributes used to transport it. To establish and preserve this understanding, the objects used within the Event Stream service will be defined with Avro schema and evolved using the controls therein. Properties definitions and the raw schema for each object are provided in the service catalog below. - Governance
All requests for integration with Event Stream services must be vetted and approved by the applicable data owners, stewards, and stakeholders around the University System. AITS will help facilitate this workflow for incoming requests.
Service Catalog
| Category | Object | Description | Client Availability | Authority |
|---|---|---|---|---|
| Employee | EmployeeEvent | Provides notification when an employee is added or updated in the authoritative source | Streams API | Banner |
| Employee | JobEvent | Provides notification when a job is added/edited/terminated for an employee in the authoritative source | Streams API | Banner |
| Finance | AccountingElementEvent | Provides notification of changes to chart-organization | Streams API; Native Kafka | Banner |
| Person | AddressEvent | Provides notification of changes to street address data in the authoritative source | Streams API; Native Kafka | Banner |
| Person | EmailEvent | Provides notification of changes to email address data in the authoritative source | Streams API; Native Kafka | Banner |
| Person | PersonEvent | Provides notification when a person's data is added or updated in the authoritative source | Streams API; Native Kafka | Banner |
| Person | PhoneEvent | Provides notification of changes to phone number data in the authoritative source | Streams API; Native Kafka | Banner |
| Student | CourseRegEvent | Provides notification of changes (add/drops) to course registration data. | Streams API; Native Kafka | Banner |
Event Metadata
Metadata attributes apply commonly and are included with each Event Stream event.
| Path | Type | Name | Occurrence | Description |
|---|---|---|---|---|
| metaData | Object-Array | metaData | Exactly One | Object containing metadata describing an event stream event |
| metaData.event | Object-Array | event | Exactly One | Object containing details about the type and timing of an event |
| metaData.event.activity | Element | activity | Exactly One | The type of data change that triggered the event (created, replaced or deleted). |
| metaData.event.dateTime | Element | dateTime | Exactly One | The date and time at which the data change event took place |
| metaData.event.id | Element | id | Exactly One | The immutable ID associated with the data change event, at the source. This is separate from the Kafka offset associated with the same event. |
| metaData.object | Object-Array | object | Exactly One | Object containing details about the data object that changed |
| metaData.object.name | Element | name | Exactly One | The name of the data object that changed (e.g. Person, Address, Employee, etc). |
| metaData.object.version | Element | version | Exactly One | The version of data object being presented with the event |
| metaData.source | Object-Array | object | Exactly One | Object containing details about the source of the data |
| metaData.source.application | Element | application | Exactly One | Application from which the data change originated |
| metaData.source.store | Element | store | Exactly One | Data store from which the data change event was captured |