Make Kafka topics only visible to the specific host #206

New Issue

Closed

opened 2022-07-05 01:39:13 +02:00 by blacklight · 0 comments

blacklight commented

2022-07-05 01:39:13 +02:00

Owner

Created by: BlackLight

As of know all the nodes connected to a Kafka backend listen to whichever message arrives on the topics specified in config.yaml, and discard the message if the target hostname doesn't match theirs.

This is bad in terms of performance (messages are broadcast to all the nodes when only one actually needs to execute) and in terms of security (all the nodes can see all the messages).

Change the Kafka backend implementation so that each node only listens to the messages delivered on the <config_topic>.hostname topic.

Later on we'll implement:

A token-based (or async-key-based) auth mechanism so that only authenticated messages will actually be executed on the node;
A wildcard or multi-node message delivery system so that messages can be delivered to multiple nodes, groups of nodes or all the nodes.

*Created by: BlackLight* As of know all the nodes connected to a Kafka backend listen to whichever message arrives on the topics specified in config.yaml, and discard the message if the target hostname doesn't match theirs. This is bad in terms of performance (messages are broadcast to all the nodes when only one actually needs to execute) and in terms of security (all the nodes can see all the messages). Change the Kafka backend implementation so that each node only listens to the messages delivered on the ```<config_topic>.hostname``` topic. Later on we'll implement: * A token-based (or async-key-based) auth mechanism so that only authenticated messages will actually be executed on the node; * A wildcard or multi-node message delivery system so that messages can be delivered to multiple nodes, groups of nodes or all the nodes.