What is a gateway?
A gateway is a long running/repeatable process whose tasks are to process and transform either the internally produced events or external events into the cloudevents specification compliant events and dispatch them to watchers(sensors and/or gateways).
A gateway has two components:
gateway-processor: Either generates the events internally or listens to external events. The implementation of gateway-processor is provided by the user which means the user can easily create a custom gateway.
gateway-transformer: Transforms the incoming events from gateway-processor into a cloudevents specification compliant events. The event is then dispatched to watchers.
Refer https://github.com/cloudevents/spec for more info on cloudevents specifications.
Core gateways come in 5 types:
||Listens to messages on a queue/topic|
||Listens to S3 Bucket Notifications|
||Produces events internally depending upon date/time schedules or intervals|
||Watches Kubernetes resources|
||Reacts to HTTP webhook notifications (Git, JIRA, Trello etc.)|
In order to take advantage of the various gateway types, you may need to install compatible message platforms (e.g. amqp, mmqp, NATS, etc..) and s3 api compatible object storage servers (e.g. Minio, Rook, CEPH, NetApp). See the artifact guide for installing object stores.
gateway-controller is responsible for managing the
|DeploySpec||Pod specification for gateway|
|ConfigMap||Name of the configmap containing gateway configuration/s|
|Type||Type of gateway|
|Version||To mark event version|
|ServiceSpec||Specifications of the service to expose the gateway|
|Watchers||Watchers are components which are interested listening to notifications from the gateway|
|RPCPort||Used to communicate between gRPC gateway client and gRPC gateway server|
|HTTPServerPort||Used to communicate between gateway client and server over http|
|DispatchMechanism||Messaging mechanism used to send events from gateway to watchers|
All core gateways use kubernetes configmap to keep track of current gateway configurations. Multiple configurations can be defined for a single gateway and each configuration will run in a separate go routine. The gateway watches updates to configmap which let us add new configuration at run time.
How to write a custom gateway?
Follow this tutorial to learn more Custom Gateways
Gateway can have zero configuration(idle) or many configurations. A configuration can be added or removed during the runtime.
Events produced can be based on a cron schedule or an interval duration. In addition, calendar gateway currently supports a
recurrence field in which to specify special exclusion dates for which this gateway will not produce an event.
calendar.fooConfig: |- interval: 10s calendar.barConfig: |- schedule: 30 * * * *
Webhook gateway expose a basic HTTP server endpoint/s. Users can register multiple REST API endpoint. See Request Methods in RFC7231 to define the HTTP REST endpoint.
# port can't be reconfigured. port: "12000" webhook.fooConfig: |- endpoint: "/foo" method: "POST" webhook.barConfig: |- endpoint: "/bar" method: "POST"
Resource gateway support watching Kubernetes resources. Users can specify
kind, and filters including prefix of the object name, labels, annotations, and createdBy time.
resource.fooConfig: |- namespace: argo-events group: "argoproj.io" version: "v1alpha1" kind: "Workflow" filter: prefix: scripts-bash labels: workflows.argoproj.io/phase: Succeeded
Artifact gateway support S3
bucket-notifications via Minio. Note that a supported notification target must be running, exposed, and configured in the Minio server. For more information, please refer to the artifact guide.
s3.fooConfig: |- s3EventConfig: bucket: foo endpoint: minio.argo-events:9000 event: s3:ObjectCreated:Put filter: prefix: "" suffix: "" insecure: true accessKey: key: accesskey name: minio secretKey: key: secretkey name: minio s3.barConfig: |- s3EventConfig: bucket: bar endpoint: minio.argo-events:9000 event: s3:ObjectCreated:Get filter: prefix: "xyz" suffix: "" insecure: true accessKey: key: accesskey name: minio secretKey: key: secretkey name: minio
Stream gateways contain a generic specification for messages received on a queue and/or though messaging server. The following are the
builtin supported stream gateways.
Nats is an open-sourced, lightweight, secure, and scalable messaging system for cloud native applications and microservices architecture. It is currently a hosted CNCF Project.
nats.fooConfig: |- url: nats://nats.argo-events:4222 attributes: subject: foo nats.barConfig: |- url: nats://nats.argo-events:4222 attributes: subject: bar
mqtt.fooConfig: |- url: tcp://mqtt.argo-events:1883 attributes: topic: foo mqtt.barConfig: |- url: tcp://mqtt.argo-events:1883 attributes: topic: bar
AMQP is a open standard messaging protocol (ISO/IEC 19464). There are a variety of broker implementations including, but not limited to the following:
- Apache ActiveMQ
- Apache Qpid
amqp.fooConfig: |- url: amqp://amqp.argo-events:5672 attributes: exchangeName: fooExchangeName exchangeType: fanout routingKey: fooRoutingKey amqp.barConfig: |- url: amqp://amqp.argo-events:5672 attributes: exchangeName: barExchangeName exchangeType: fanout routingKey: barRoutingKey
#### Kafka [Apache Kafka](https://kafka.apache.org/) is a distributed streaming platform. We use Shopify's [sarama](https://github.com/Shopify/sarama) client for consuming Kafka messages.
kafka.fooConfig: |- url: kafka.argo-events:9092 attributes: topic: foo partition: "0" kafka.barConfig: |- url: kafka.argo-events:9092 attributes: topic: bar partition: "1"
### Examples Explore [Gateway Examples](https://github.com/argoproj/argo-events/tree/eventing/examples/gateways)