Repository navigation
Description
Steps to reproduce
- activate 'workflow external scripts' app
- create a rule that calls an external script
- trigger the event that will call the external script
- wait until the next cron.php schedule
Expected behaviour
Scripts should be started upon next cron.php schedule, and should be allowed to finish
Actual behaviour
External script gets killed by SIGTERM some milliseconds after start
Workflow Script app
Workflow Script app version: (see apps admin page: /index.php/settings/apps)
1.2.0
Server configuration
Operating system: Ubuntu/RedHat/...
Debian 8
Web server: Apache/Nginx
Apache 2.4,
Database: MySQL/Maria/SQLite/PostgreSQL
Postgresql 9.5
PHP version:
7.3.15 FPM (deb.sury.org)
Nextcloud Version: (see admin page)
17.0.3
Where did you install Nextcloud from:
tarball, updated since long ago
Signing status:
ok
List of activated apps:
Enabled:
- accessibility: 1.3.0
- activity: 2.10.1
- admin_audit: 1.7.0
- bruteforcesettings: 1.5.0
- checksum: 0.4.4
- circles: 0.17.10
- cloud_federation_api: 1.0.0
- comments: 1.7.0
- dav: 1.13.0
- deck: 0.8.0
- dicomviewer: 1.2.1
- end_to_end_encryption: 1.3.0
- federatedfilesharing: 1.7.0
- federation: 1.7.0
- files: 1.12.0
- files_accesscontrol: 1.7.0
- files_automatedtagging: 1.7.0
- files_downloadactivity: 1.6.0
- files_fulltextsearch: 1.3.6
- files_pdfviewer: 1.6.0
- files_sharing: 1.9.0
- files_trashbin: 1.7.0
- files_versions: 1.10.0
- files_videoplayer: 1.6.0
- firstrunwizard: 2.6.0
- fulltextsearch: 1.3.8
- fulltextsearch_elasticsearch: 1.4.1
- gallery: 18.4.0
- groupfolders: 5.0.5
- logreader: 2.2.0
- lookup_server_connector: 1.5.0
- metadata: 0.11.1
- music: 0.13.0
- nextcloud_announcements: 1.6.0
- notifications: 2.5.0
- oauth2: 1.5.0
- password_policy: 1.7.0
- previewgenerator: 2.2.0
- privacy: 1.1.0
- provisioning_api: 1.7.0
- radio: 0.6.6
- richdocuments: 3.5.2
- serverinfo: 1.7.0
- sharebymail: 1.7.0
- sharelisting: 0.2.0
- spreed: 7.0.2
- support: 1.0.1
- systemtags: 1.7.0
- text: 1.1.1
- theming: 1.8.0
- theming_customcss: 1.5.0
- twofactor_backupcodes: 1.6.0
- twofactor_totp: 4.1.2
- updatenotification: 1.7.0
- viewer: 1.2.0
- workflow_script: 1.2.0
- workflowengine: 1.7.0
Nextcloud configuration:
Are you using external storage, if yes which one: local/smb/sftp/...
no
Are you using encryption: yes/no
no
Server log (data/nextcloud.log)
no errors
Remarks
I've written a shell script that only logs the signals it receives (only those that can be trapped at all). Some milliseconds after start the script receives a SIGTERM. I suspect systemd's cron implementation is sending this signal when the main PHP process (of cron.php) exits. This will be very quick given the fact that external scripts are called asynchronously, see lib/BackgroundJobs/Launcher.php#71. 90s later, systemd sends SIGKILL to all processes in that process group (and this is logged in systemd's journal).
With a similar configuration but with a native cron this doesn't occur: external scripts survive the termination of cron.php, which is the expected behaviour.
Please advise what to do in this situation.
Activity
blizzz commentedon Mar 2, 2020
We don't set a timeout, and, by heart, running the script should not finish the php process, but it waits for the execution first. Maybe I remember wrongly, as I wouldn't know how or why it would be killed.
If it is native cron vs systemd, then the fix has to happen somewhere on the systemd layer.
it25fg commentedon Mar 26, 2020
I have tested further. Contributing factors could be:-
The behaviour I have seen is that cron.php lasts for no more than 2 secs, starts the script, and my script gets killed (no output, no log message) exactly when cron.php finishes. The only way to guard my script is to include a BEGIN block that sets $sig{CHLD,PIPE,TERM,INT} to 'IGNORE'. Then, it's at least immune against these signals, and behaves this way:-
As soon as I remove the '&' from Launcher.php#71 the sequence changes to what you described:
--- continues until script has finished ---
-> script prints its last log message, script and cron.php vanish.
This would be my temporary solution if I encounter this on any Nextcloud installation. So you can close this issue: I have a workaround. With more and more systemd coverage I'd expect that others run into such problem too.
ldidry commentedon Jul 16, 2020
I confirm that using cron instead of systemd timer allows the external script to run until it’s normally finished.
Thanks @it25fg for the hint, I was becoming crazy.