Repository navigation
Serialize File Content #574
Description
Right now we don't serialize file content. Just metadata. Along with this we'll want to allow users to configure the max message size to help users protect against large file upload attacks.
See this comment for implementation details.
Activity
Archmonger commentedon Apr 4, 2022
Just had a thought.
We could possibly leave file serialization to HTTP, and handle things on the backend.
For example, adding a HTTP endpoint
Not really sure how we'd be able to tell the WS side of things when an upload is a complete though...
Archmonger commentedon Apr 6, 2022
If we do go down this road of serializing via websockets, it's going to be extremely important to stream the file to disk rather than buffer it in memory. Otherwise, this would be a DDOS attack vector.
Ideally, "file stuff" should be handled by individual frameworks... but that design of that might not be technologically feasible.
Archmonger commentedon Dec 4, 2022
PyZMQ might be a decent option to communicate between separate ASGI processes.
rmorshea commentedon Dec 19, 2022
The initial implementation should probably use form elements to upload files over HTTP. We can overwrite the default
onSubmitbehavior for forms to submit without a redirect. Unfortunately though, I don't think we can supply a built-in solution that will allow for inter-process communication - that will have to be supplied on a case-by-case basis depending on the user's deployment.With that said, I have some thoughts on how this could be accomplished via the websocket in a more natural way, but it would require some pretty deep changes to how we send messages between the client and the server. Right now, there is no real messaging protocol - the server always sends layout updates and the client always sends layout events. We'd need to change this in order to allow the client and server to send different types of messages to each other. For example, all messages between the client and server would probably take the form:
{ "type": str, # some unique name "version": str, # allow client and server to sort out compatibility ... # any other data }Archmonger commentedon Jun 17, 2023
We are likely going to develop two file upload handlers.
For small files, we stream the file directly to memory.
For large files, we stream the file to disk (buffered in memory).
In order to support this without having this become an attack vector, we should implement a few safeguards:
MAX_BUFFER_SIZE: The maximum amount of bytes to buffer within memory before throttling responsesMAX_FILE_SIZE: The maximum size of a file that can be saved to the diskUPLOAD_DIR: Directory where files get saveddjango.core.files.uploadhandler.MemoryFileUploadHandlerdjango.core.files.uploadhandler.TemporaryFileUploadHandlerArchmonger commentedon Jun 21, 2023
We might want to transfer this issue to the
reactpy-routerrepo