Events should be filterable by CloudEvents source as opposed to just type

[berndtj]

Feature Request

Detailed Description

Event types alone are not enough for routing through subscriptions. The alternative is filtering in a function and calling subsequent functions, however this is a waste of computation.

The method for defining filters should be spec'ed out and agreed upon.

Context

Possible Implementation

Complexity

• Low - Simple enhancement or bug fix, no architectural changes or refactoring
• Medium - Change requires some thought, but is relatively isolated
• High - Significant architectural change or large refactor

[zimengyang]

Pending Changes:

The reason to introduce channel is to provide single entry point for subscription for events coming from both event-driver and custom events.

• Add Channel concept which can be treated as a topic string in Transport.
  • Add channel entity model and http handler.
• Add source and channel fields in Subscription. Subscription will contain:
  • channel: required
  • source and event-type: optional and will be used as filter
• Add channel field to EventDriver.
  • [TODO] event-driver might be pointing to multiple channels in the future.
• Implement filters in subscriptions/manager/handler.
  • channel will be the top level filter
  • if source and event-type are provided, perform optional filtering.

Steps to Create Event Driver:

# 1. create channel, basically a string representing the topic
$ dispatch create channel <CHANNEL_NAME>

# 2. eventdriver given channel 
$ dispatch create eventdriver --channel <CHANNEL_NAME> ...

# 3. subscription given channel. optional source and event-type 
$ dispatch create subscription <FUNCTION_NAME> --channel <CHANNEL_NAME>  [--source <SOURCE>] [--event-type <EVENT_TYPE>]

[neosab]

The reason to introduce channel is to provide single entry point for subscription for events coming from both event-driver and custom events.

I am trying to see if we strongly needChannel in Dispatch either for the semantics or for the ease of implementation.

On the semantics

A subscription today is

from: <eventType>
to:  <function>

If we change it to be:

from : <eventDriver>
to: <function>
filters:
  eventType:
    - <x.y.z>
  source:
    - <source>

wouldn't this suffice?

IIUC, eventDriverType already has an --expose option for external sources to push events.

On the implementation side

For Solo:

I see what Channel is trying to achieve here but why can't we just use eventDriver (say uuid) for the topic string? Since the subscription has the eventDriver info, the handler can subscribe to the appropriate topic and run the filter and trigger the function. I know this different from the current implementation where eventtype itself is the topic name.

For knative:

Each eventDriver request can create an in-tree eventing.ContainerSource resource and an eventing.Channel resource implicitly. When a user creates a subscription from an eventDriver, we implicitly create a eventing.Subscription from the corresponding eventing.Channel to the function. W.r.t to filtering, we somehow implement it in the knative message dispatcher (e.g on a forked version of the Kafka channel dispatcher).

I know we discussed the possibility of a new CRD that creates and owns multiple eventing.Channels one for each event type. I don't think it scales well. Also, we can't address the case where a user wants a function to handle all event types :).

[kars7e]

If we change it to be:

from : <eventDriver>
to: <function>
filters:
  eventType:
    - <x.y.z>
  source:
    - <source>

wouldn't this suffice?

@neosab the initial idea was what you described, but we also have an events endpoint that accepts generic cloudevents. The idea behind channel was to introduce a single connection point for subscriptions, it's not related to knative and it's not there to be an abstraction over messaging (but of course it fits nicely). Is your suggestion to drop the events API? I'd like users to be able to manually generate events for testing/etc, but maybe we can work around that. If we only have event drivers as a way to get events into the system, then it's possible to couple them directly with subscriptions.