Gateway

What is a gateway?

A gateway consumes events from event sources, transforms them into the cloudevents specification compliant events and dispatches them to sensors.


Gateway


Components

A gateway has two components:

  1. gateway-client: It creates one or more gRPC clients depending on event sources configurations, consumes events from server, transforms these events into cloudevents and dispatches them to sensors. Refer https://github.com/cloudevents/spec for more info on cloudevents specifications.
  2. gateway-server: It is a gRPC server that consumes events from event sources and streams them to gateway client.

Core gateways

  1. Calendar: Events produced are based on either a cron schedule or an interval duration. In addition, calendar gateway supports a recurrence field in which to specify special exclusion dates for which this gateway will not produce an event.

  2. Webhooks: Webhook gateway exposes REST API endpoints. The request received on these endpoints are treated as events. See Request Methods in RFC7231 to define the HTTP REST endpoint.

  3. Kubernetes Resources: Resource gateway supports watching Kubernetes resources. Users can specify group, version, kind, and filters including prefix of the object name, labels, annotations, and createdBy time.

  4. Artifacts: Artifact gateway supports S3 bucket-notifications via Minio. Note that a supported notification target must be running, exposed.

  5. Streams: Stream gateways contain a generic specification for messages received on a queue and/or though messaging server. The following are the stream gateways offered out of box:

    1. NATS: Nats is an open-sourced, lightweight, secure, and scalable messaging system for cloud native applications and microservices architecture. It is currently a hosted CNCF Project.

    2. MQTT: MMQP is a M2M "Internet of Things" connectivity protocol (ISO/IEC PRF 20922) designed to be extremely lightweight and ideal for mobile applications. Some broker implementations can be found here.

    3. Kafka: Apache Kafka is a distributed streaming platform. We use Shopify's sarama client for consuming Kafka messages.

    4. AMQP: AMQP is a open standard messaging protocol (ISO/IEC 19464). There are a variety of broker implementations including, but not limited to the following:

    5. Apache ActiveMQ

    6. Apache Qpid

    7. StormMQ

    8. RabbitMQ

You can find core gateways here

Community gateways

You can find gateways built by the community here. New gateway contributions are always welcome.

Managing Event Sources

  • The event sources configurations are managed using K8s configmap. Once the gateway resource is created with the configmap reference in it's spec, it starts watching the configmap. The gateway-client sends each event source configuration to gateway-server over gRPC. The gateway-server then parses the configuration to start consuming events from external event producing entity.

  • You can modify K8s configmap containing event sources configurations anytime and gateway-client will intelligently pick new/deleted configurations and send them over to gateway-server to either start or stop the event sources.

How to write a custom gateway?

To implement a custom gateway, you need to create a gRPC server and implement the service defined below. The framework code acts as a gRPC client consuming event stream from gateway server.



Sensor


  • Proto Definition

    1. The proto file is located at https://github.com/argoproj/argo-events/blob/master/gateways/eventing.proto

    2. If you choose to implement the gateway in go, then you can find generated client stubs at https://github.com/argoproj/argo-events/blob/master/gateways/eventing.pb.go

    3. To create stubs in other languages, head over to gRPC website https://grpc.io/

    4. Service

      /**
      
    5. Service for handling event sources. */ service Eventing { // StartEventSource starts an event source and returns stream of events. rpc StartEventSource(EventSource) returns (stream Event); // ValidateEventSource validates an event source. rpc ValidateEventSource(EventSource) returns (ValidEventSource); }

  • Available Environment Variables to Server

    | Field | Description | | ------------------------------- | ------------------------------------------------ | | GATEWAYNAMESPACE | K8s namespace of the gateway | | GATEWAYEVENTSOURCECONFIGMAP | K8s configmap containing event source | | GATEWAYNAME | name of the gateway | | GATEWAYCONTROLLERINSTANCEID | gateway controller instance id | | GATEWAYCONTROLLERNAME | gateway controller name | | GATEWAYSERVER_PORT | Port on which the gateway gRPC server should run |

  • Implementation

    You can follow existing implementations here

Examples

You can find gateway examples here