Repository navigation
If Kafka is turned on internal subs dont work #6778
Description
ISSUE TYPE
- Bug Report
COMPONENT NAME
API
CLOUDSTACK VERSION
4.17.0.1
SUMMARY
When kafka events are turned on, subscriptions internal to CloudStack can no longer receive events.
There appears to be code that is trying to use the events existing in the CloudStack code, but would stop working once kafka emitting is enabled.
This issue is to start a dialog about this functionality and/or directions.
A work around was created to resolve this issue, by creating an in-memory bus subscribe-able to anything internal to CloudStack and also emit events to any configured kafka endpoint simultaneously. Thus allowing event processing internally to CloudStack without having to subscribe to external endpoint.
Activity
DaanHoogland commentedon Sep 28, 2022
thanks for the report and the workaround @damonb123 Can you give a use case that would fail? Would for instance usage stop working?
mlsorensen commentedon Apr 12, 2023
I think what's going on is that most things that publish to the cloudstack event bus use code like:
So you can't for example have both RabbitMQ and Kafka enabled, or InMemory and Kafka. It just fetches one.
From what I can tell, all event buses are disabled by default and need XML defining a bean to enable, so it is probably ok to have just one bus usable at a time. You aren't going to break internal code by switching event buses. However it could be a nice enhancement to publish to any and all EventBus components that are loaded - it could be handy to use both InMemory and Kafka at the same time for different purposes.
Where one might also run into issues is if you have coded a plugin to do something like
memoryBus.subscribe(), but then switch the config to enable the Kafka event bus. The internal code will publish to only one, so you'd want to grab the one that is loaded rather than using the specific EventBus implementation directly.mlsorensen commentedon Apr 12, 2023
If everyone agrees, let's update the code to call all registered
EventBusimplementations by swapping forComponentContext.getComponentsOfType(EventBus.class)... preferably centralized in one place rather than adding loops everywhere we currently have the existing pattern above.DaanHoogland commentedon Aug 28, 2023
@mlsorensen did you create a PoC for this (or any other code we can discuss)?
mlsorensen commentedon Aug 28, 2023
@DaanHoogland - No, I haven't done anything other than identify the issue.
DaanHoogland commentedon Aug 28, 2023
ok, so your idea is that we create a list of channels and in have a default MessageBus.publish() iterate the list of registered message busses and call eache publish implementation?
Not sure if that is the best design. any thoughts anybody?
mlsorensen commentedon Aug 28, 2023
I’m not necessarily advocating for any specific implementation, but it would be good to support more than one registered EventBus implementation.
The most straightforward way is probably to just make the existing code aware that there can be more than one, and that would be fine for a small number.
If we want scalability (say we wanted dozens of implementations), maybe a sub/pub model would work better.
I have a hard time imagining anyone would need many implementations. I think we are realistically talking publishing events to maybe two or three types of event bus.
DaanHoogland commentedon Aug 29, 2023
Yes
but that would be hard to maintain, any change would be at all calling locations.
I was under the impression that we had that but it is obviously crippled. I think we should grow to that.
but two of three of varying types (plus internal) will make for a good justification of pub/sub. I will look into how much effort it will be to migrate to that.
1 remaining item
DaanHoogland commentedon Aug 31, 2023
@damonb123 can you explain about what you consider internal and not working in the light of what @NuxRo describes does work work him? I would like to reproduce the internal events not going through as that would be a potentialy costly bug for people using the usage service for instance.
I agree that it would be convenient to have a broadcast to which any and all message busses can be connected. For @NuxRo 's scenario there would be a workaround by subscribing one message bus to the other (chaining them).
DaanHoogland commentedon Sep 21, 2023
@mlsorensen, I didn't go for this as it is maybe a small number of implementations but scattered a bit around the code base. I think something like #7990 would be my take on this. Yhis way we can also allow proprietary/private implementations.