You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
When streaming, read packet queue grows without limit and workers are slower than producer. #77
When using synapsectl read I have observed 2 issues.
After stopping a read that has been running for a while, the CLI hangs for a long time before finally exiting.
The memory usage of the CLI seems to grow without bounds when streaming, then slowly reduces during the time the CLI hangs after stopping
Investigating this, it appears to be caused by the fact that the queues which hold new packets for various workers to consume (writing to disk primarily) have no size limit, and the worker thread for writing to disk is slower than the thread reading packets from the network.
This is not a problem for small channel counts with low data rates. But for large numbers of channels, the memory usage grows quickly (exceeding 1GB after 2 mins).
Activity