could we not use a flag like the "no_wf" that says if it is OK or not to drop this session for a reboot? I.E. average off-site users by default can't fire up a session that will stop it rebooting, local users can (and local addresses do, by default). If the owner (or trusted user) is off-site via public IP and wants to force a non dying session they append (for example) &no_rbt_[pswd] That should work with scripts.
could we not use a flag like the "no_wf" that says if it is OK or not to drop this session for a reboot? I.E. average off-site users by default can't fire up a session that will stop it rebooting, local users can (and local addresses do, by default). If the owner (or trusted user) is off-site via public IP and wants to force a non dying session they append (for example) &no_rbt_[pswd] That should work with scripts.
Session and restart enforcement sounds more like a server policy than a user decision. If a trusted user connects via public IP then they can log in with their credentials to get a privileged session status, so an extra parameter isn't really needed for that, is it?
1) Checking if an update is available doesn't "kill" any sessions. You can try it manually in the update tab as proof. 2) Right now if an update is available, and there are user connections, the update will not begin until the last user disconnects. 3) "User connections" doesn't currently count "internal connections" which right now is only WSPR auto-run. Kiwirecorder et al are considered normal user connections. So only WSPR auto-run will be interrupted by an update (and then restarted). 4) Checking for, and if necessary building, an update occurs on each Kiwi server restart.
I just now spent some time re-testing all this. I changed the code slightly to shift the 0200Z check to 5 minutes in the future. Then tried connecting with kiwirecorder and then WSPR auto-run. It all worked fine. Kiwirecorder connections were not closed by an update check or pending update build. So I'm not sure what you guys are talking about.
And if you have both a WSPR auto-run and kiwirecorder connection active, and there is a pending update, the correct thing happens too: as soon as kiwirecorder disconnects WSPR get booted and the update proceeds.
I have implemented version 0.5c of kiwiwspr.sh which includes a watchdog daemon that checks the status of the kiwirecorder.py sessions each odd 2 minutes and restarts or performs scheduled configuration changes on them as needed. The watchdog log shows that one or more (and sometimes all) of them dead at random times. I will need to modify kiwiwspr.sh to save the error output of kiwirecorder.py, but looking at the restart times strongly suggests those restarts are happening at semi-random times. So they are unlikely generated by your update check. I'm sorry if I mislead you into a unneeded diagnostic task.
I have kwiiwspr.sh running on Raspberry Pis at two sites: KPH and AI6VN/KH6. At AI6VN/KH6 kiwirecorder.py never fails while at KPH it it fails frequently, sometimes several times per hour. I just noticed that at KPH the Raspberry Pi server running kiwiwspr.sh had 100% cpu activity due to a rogue zombie process I created last week while debugging Internet connection issues. After killing that process I seem to have reduced or eliminated the dying kiwirecorder.py's, although will take another day to be sure of the level of improvement. So I would theorize that your server runs a program at 0602 that consumes CPU or other resources on your server. If you don't know of a crontab job at that time, you could schedule 'top -b > top.log' to run 0658 to 0606 and look at what was running then. In my systems, the first 8-14 lines of top output are COMMAND python until each even 2 minutes when one or more 'wsprd' commands are at the head of the list for about 45 seconds
FWIW all the kiwirecorder tasks also die around 0623 UTC on my system. I do not use my system 24/7 but when I do I have experienced the failure each night around the same time.
Being able to run 8 simultaneous instances of WSPR on one Kiwi is a fantastic feature. Thanks so much to all who made it possible.
I have just finished version 0.5d which adds support for a watchdog and scheduled band configuration changes. It also incorporates many command line changes which I hope make it easier to use. However it requires changes to the CAPTURE_JOBS[] array in the kwiconfig.conf file and you must add at least one WSPR_SCHEDULE[] entry. It is running well at 3 sites, but as always save your current kwiwspr.sh and kwiwspr.conf file in case this new version doesn't work for you. It is a very lightweight program and you can easily spot all 14 LF/MF/HF WSPR bands on a Raspberry Pi I welcome comments and bug reports
installing now but may have an issue. I ran kiwiwspr.sh and let it make a conf file then without an edit ran kiwiwspr.sh -j a,all but it bombed. I had intended to stop it edit for myself and start again. See attached file
errors I get when I run it.... I edited calls and IP# only KIWI name unchanged Well I tried to upload errors.txt but board is thwarting me sorry about the dupes... [fixed -jks]
Thanks for the testing I found and fixed a bug in the auto-created kiwiwspr.conf. Delete the kiwiwspr.conf and run the attached 0.5e to auto-create a kiwiwspr.conf which doesn't crash kiwiwspr.sh There are no other changes or fixes in 0.5e
As I said in the post above, I agree with John that the problem is not in the Kiwi but almost certainly in the server running kiwiwpr.sh. Have you looked at your /etc/crontab for a task scheduled at that time?
Do these Kiwis happen to have the "daily restart" option on the admin control tab page set to "yes"? If so there might be a bug with that. The restart is supposed to wait for all users (including kiwirecorder) to disconnect.
I've just checked all 5 of those Kiwis and none were set to daily restart. I saw no restarts on my Maui Kiwi in the last 24 hours, only on the KPH Kiwi which was being used by the overloaded Pi. So I feel certain there is no problem with the Kiwi SW. On one of my TV products a unpublished BIOS upgrade by the motherboard manufacturer introduced a CPU hang of several seconds at what seemed like random times. I suspect an analogous source of these events in the server, not the Kiwi
Kiwiwspr0.5d.sh running on a RPi3+ has worked very well for me during the past 24 hours. At one point last night it was decoding 630 WSPR spots per minute.
At 06:27 UTC the watchdog log indicated that six stale capture jobs were restarted with six new PIDs. This eliminated my ~ 06:23 WSPR failure problem.
For a short experiment I ran seven instances of WSPR with 0.5d from 40 10 m and one instance of 20m WSPR on the Kiwis one remaining channel for comparison. During a one hour period 0.5d decoded 46 spots from 18 unique stations. The Kiwi decoded 41 spots from 16 unique stations. I suppose this shows that the wsprd algorithm used with 0.5d is more efficient than the older one used with the Kiwi. I realize that running more instances on the Kiwi would overburden the limited processing power of the Beagle resulting in even fewer spots.
Kiwiwspr0.5d.sh is a great asset. Once again, my thanks to Rob and everyone else who made it possible.
Hurricane Lane found a bug in the startup code of kiwiwspr last night when my Maui Kiwi lost power for about an hour and the kiwiwspr daemon failed to start after power was restored. I have fixed that bug and several others in the attached file, but existing installations will need to be patched. Unfortunately fixing the startup configuration of an existing installation requires that you make a minor edit to the systemctl configuration file:
1) Verify that the kiwiwspr.service is installed on you machine: pi@PiKPH:~/ham/kiwiwspr $ sudo systemctl status kiwiwspr ? kiwiwspr.service - KiwiSDR WSPR daemon Loaded: loaded (/lib/systemd/system/kiwiwspr.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2018-08-25 07:04:47 PDT; 6min ago Process: 22353 ExecStart=/home/pi/ham/kiwiwspr/kiwiwspr.sh -w a (code=exited, status=0/SUCCESS) Main PID: 22358 (kiwiwspr.sh) CGroup: /system.slice/kiwiwspr.service ??22358 /bin/bash /home/pi/ham/kiwiwspr/kiwiwspr.sh -w a ??26156 /bin/bash /home/pi/ham/kiwiwspr/kiwiwspr.sh -j a,all ??26487 /bin/bash /home/pi/ham/kiwiwspr/kiwiwspr.sh -d a,KPH_HF_3,30 ??26510 ps 28485
Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: User lookup succeeded: uid=1000 gid=1000 Aug 25 07:04:47 PiKPH systemd[22353]: kiwiwspr.service: Executing: /home/pi/ham/kiwiwspr/kiwiwspr.sh -w a Aug 25 07:04:47 PiKPH kiwiwspr.sh[22353]: Sat Aug 25 07:04:47 PDT 2018: INFO, this server already has a /lib/systemd/system/kiwiwspr.service file. So leaving it alone. Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Child 22353 belongs to kiwiwspr.service Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Control process exited, code=exited status=0 Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Got final SIGCHLD for state start. Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Main PID guessed: 22358 Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Changed start -> running Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Job kiwiwspr.service/start finished, result=done Aug 25 07:04:47 PiKPH systemd[1]: Started KiwiSDR WSPR daemon. pi@PiKPH:~/ham/kiwiwspr $
If you don't see a printout like that which indicates the service is running, then you need to: 2) Stop your watchdog daemon: ./kiwiwspr.sh -w z 3) Edit your service file: sudo vi /lib/systemd/system/kiwiwspr.service add 'a' to the end of the line: ExecStart=${KIWIWSPR_ROOT_DIR}/kiwiwspr.sh -w to make it: ExecStart=${KIWIWSPR_ROOT_DIR}/kiwiwspr.sh -w a 4) Load the edited file and start the service: sudo systemctl daemon-reload sudo systemctl start kiwiwspr sudo systemctl status kiwiwspr
You can also confirm that the daemon is running with './kiwiwspr.sh -w s'
I have just posted this discovery in another thread, but it is also highly relevant to users of my kiwiwspr.sh. Later today I hope to share V0.6a of that script with what I hope to be the final set of features and bug fixes.
I recently discovered that Raspbian Stretch has a bug which very, very infrequently causes ethernet packets to be dropped. By running 'sudo rpi-update' on my Pi 3b, which upgraded the kernel from 4.7 to 4.14, I have completely eliminated the 2-5 per day kiwirecorder.py restarts from which I was previously suffering. Running 4.14, my WSPR decoding script 'kiwiwspr.sh' can simultaneously record 17 uncompressed audio streams (3.5 Mbps = 750 receive packets per second ) from 3 Kiwis on one Pi without a single restart in 24 hours.
However, even with that Pi OS upgrade I have encountered one restart which occurred at the same time on two 8 channel Kiwis being run from my Pi. The /var/log/messages on both Kiwis record the same set of restart events on both Kiwis. Those syslog entries are strangely not recorded in ascending time order, so it isn't clear what event initiated the LEAVING messages:
......... Sep 5 13:44:19 kiwisdr kiwid: 5d:13:18:01.816 01234567 ca_pause 16560 ca_pause_old 192 ca_shift=28 code_creep=-220 code_period_ms=1 code_period_samples=16368 Sep 5 23:05:44 kiwisdr kiwid: 5d:22:39:26.317 01234567 ca_pause 65527 ca_pause_old 55 ca_shift=4756 code_creep=-4811 code_period_ms=4 code_period_samples=65472 Sep 6 03:21:27 kiwisdr kiwid: 6d:02:55:09.367 01234567 ca_pause 16372 ca_pause_old 4 ca_shift=16 code_creep=-20 code_period_ms=1 code_period_samples=16368 Sep 6 03:28:04 kiwisdr kiwid: 6d:03:01:46.068 01234567 UPDATE: check scheduled Sep 6 06:25:11 kiwisdr rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="430" x-info="http://www.rsyslog.com"] rsyslogd was HUPed Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.800 .1234567 0 21096.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:02) Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.808 .123456. 7 18106.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:03) Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.813 .1234.6. 5 10140.15 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:05) Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.816 ..234.6. 1 3594.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:56:06) Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.821 ..23..6. 4 7040.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:06) Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.830 ..23.... 6 14097.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:04) Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.840 ..2..... 3 3570.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:08) Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.843 ..2..... UPDATE: checking for updates Sep 6 06:25:02 kiwisdr kiwid: 6d:05:58:54.877 ..2..... UPDATE: version 1.219 is current Sep 6 06:27:04 kiwisdr kiwid: 6d:06:00:46.261 ..23.... 3 3570.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED) Sep 6 06:27:05 kiwisdr kiwid: 6d:06:00:47.250 ..234... 4 3594.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED) Sep 6 06:27:06 kiwisdr kiwid: 6d:06:00:48.192 ..2345.. 5 7040.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED) Sep 6 06:27:07 kiwisdr kiwid: 6d:06:00:49.144 ..23456. 6 10140.15 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED) Sep 6 06:27:08 kiwisdr kiwid: 6d:06:00:50.085 ..234567 7 14097.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED) Sep 6 06:27:10 kiwisdr kiwid: 6d:06:00:51.913 0.234567 0 21096.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED) Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.048 0.234567 0 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.052 0.234567 3 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.052 0.234567 4 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.053 0.234567 5 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.053 0.234567 6 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.053 0.234567 7 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here Sep 6 06:27:25 kiwisdr kiwid: 6d:06:01:07.831 0.234567 7 GEOLOC: extreme-ip-lookup.com/json/198.40.45.23 Sep 6 06:27:25 kiwisdr kiwid: 6d:06:01:07.834 0.234567 6 GEOLOC: geoloc_task:P2:T36((50.000 msec) TaskSleep) Sep 6 06:27:25 kiwisdr kiwid: 6d:06:01:07.847 0.234567 5 GEOLOC: geoloc_task:P2:T35((50.000 msec) TaskSleep) Sep 6 06:27:26 kiwisdr kiwid: 6d:06:01:07.854 0.234567 4 GEOLOC: geoloc_task:P2:T34((50.000 msec) TaskSleep) Sep 6 06:27:26 kiwisdr kiwid: 6d:06:01:08.039 0.234567 3 GEOLOC: geoloc_task:P2:T33((50.000 msec) TaskSleep) Sep 6 06:27:26 kiwisdr kiwid: 6d:06:01:08.259 0.234567 0 GEOLOC: geoloc_task:P2:T32((50.000 msec) TaskSleep) Sep 6 06:29:10 kiwisdr kiwid: 6d:06:02:52.047 01234567 1 18106.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED) Sep 6 06:29:24 kiwisdr kiwid: 6d:06:03:06.048 01234567 1 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here Sep 6 06:29:24 kiwisdr kiwid: 6d:06:03:06.821 01234567 1 GEOLOC: extreme-ip-lookup.com/json/198.40.45.23 Sep 6 08:40:26 kiwisdr kiwid: 6d:08:14:07.981 01234567 ca_pause 18040 ca_pause_old 1672 ca_shift=20 code_creep=-1692 code_period_ms=1 code_period_samples=16368 Sep 6 10:40:14 kiwisdr kiwid: 6d:10:13:55.890 01234567 ca_pause 18648 ca_pause_old 2280 ca_shift=1964 code_creep=-4244 code_pe ........
Thinking my restarts were due to some HW limit of the Pi, I bought an odroid but haven't set it up Now that the Pi is fixed, I don't see that I need to learn a whole new environment.
Comments
I.E. average off-site users by default can't fire up a session that will stop it rebooting, local users can (and local addresses do, by default).
If the owner (or trusted user) is off-site via public IP and wants to force a non dying session they append (for example) &no_rbt_[pswd]
That should work with scripts.
I.E. average off-site users by default can't fire up a session that will stop it rebooting, local users can (and local addresses do, by default).
If the owner (or trusted user) is off-site via public IP and wants to force a non dying session they append (for example) &no_rbt_[pswd]
That should work with scripts.
Session and restart enforcement sounds more like a server policy than a user decision. If a trusted user connects via public IP then they can log in with their credentials to get a privileged session status, so an extra parameter isn't really needed for that, is it?
1) Checking if an update is available doesn't "kill" any sessions. You can try it manually in the update tab as proof.
2) Right now if an update is available, and there are user connections, the update will not begin until the last user disconnects.
3) "User connections" doesn't currently count "internal connections" which right now is only WSPR auto-run. Kiwirecorder et al are considered normal user connections. So only WSPR auto-run will be interrupted by an update (and then restarted).
4) Checking for, and if necessary building, an update occurs on each Kiwi server restart.
I just now spent some time re-testing all this. I changed the code slightly to shift the 0200Z check to 5 minutes in the future. Then tried connecting with kiwirecorder and then WSPR auto-run. It all worked fine. Kiwirecorder connections were not closed by an update check or pending update build. So I'm not sure what you guys are talking about.
At AI6VN/KH6 kiwirecorder.py never fails while at KPH it it fails frequently, sometimes several times per hour.
I just noticed that at KPH the Raspberry Pi server running kiwiwspr.sh had 100% cpu activity due to a rogue zombie process I created last week while debugging Internet connection issues.
After killing that process I seem to have reduced or eliminated the dying kiwirecorder.py's, although will take another day to be sure of the level of improvement.
So I would theorize that your server runs a program at 0602 that consumes CPU or other resources on your server. If you don't know of a crontab job at that time, you could schedule 'top -b > top.log' to run 0658 to 0606 and look at what was running then.
In my systems, the first 8-14 lines of top output are COMMAND python until each even 2 minutes when one or more 'wsprd' commands are at the head of the list for about 45 seconds
Being able to run 8 simultaneous instances of WSPR on one Kiwi is a fantastic feature. Thanks so much to all who made it possible.
Gene W3PM
It also incorporates many command line changes which I hope make it easier to use.
However it requires changes to the CAPTURE_JOBS[] array in the kwiconfig.conf file and you must add at least one WSPR_SCHEDULE[] entry.
It is running well at 3 sites, but as always save your current kwiwspr.sh and kwiwspr.conf file in case this new version doesn't work for you.
It is a very lightweight program and you can easily spot all 14 LF/MF/HF WSPR bands on a Raspberry Pi
I welcome comments and bug reports
Attachments:
https://forum.kiwisdr.com/uploads/Uploader/83/1583fff0adb66258650841176e1931.sh
I found and fixed a bug in the auto-created kiwiwspr.conf.
Delete the kiwiwspr.conf and run the attached 0.5e to auto-create a kiwiwspr.conf which doesn't crash kiwiwspr.sh
There are no other changes or fixes in 0.5e
Attachments:
https://forum.kiwisdr.com/uploads/Uploader/d3/ad166df9de32af23c1e4ded068c997.sh
Anything thoughts on the 06:22 phenomena?
I saw no restarts on my Maui Kiwi in the last 24 hours, only on the KPH Kiwi which was being used by the overloaded Pi.
So I feel certain there is no problem with the Kiwi SW.
On one of my TV products a unpublished BIOS upgrade by the motherboard manufacturer introduced a CPU hang of several seconds at what seemed like random times.
I suspect an analogous source of these events in the server, not the Kiwi
At 06:27 UTC the watchdog log indicated that six stale capture jobs were restarted with six new PIDs. This eliminated my ~ 06:23 WSPR failure problem.
For a short experiment I ran seven instances of WSPR with 0.5d from 40 10 m and one instance of 20m WSPR on the Kiwis one remaining channel for comparison. During a one hour period 0.5d decoded 46 spots from 18 unique stations. The Kiwi decoded 41 spots from 16 unique stations. I suppose this shows that the wsprd algorithm used with 0.5d is more efficient than the older one used with the Kiwi. I realize that running more instances on the Kiwi would overburden the limited processing power of the Beagle resulting in even fewer spots.
Kiwiwspr0.5d.sh is a great asset. Once again, my thanks to Rob and everyone else who made it possible.
Thanks for the experiment. You mean per 630 spots per hour I suppose or 10 per minute. 10 per second would be amazing.
W3PM
I have fixed that bug and several others in the attached file, but existing installations will need to be patched.
Unfortunately fixing the startup configuration of an existing installation requires that you make a minor edit to the systemctl configuration file:
1) Verify that the kiwiwspr.service is installed on you machine:
pi@PiKPH:~/ham/kiwiwspr $ sudo systemctl status kiwiwspr
? kiwiwspr.service - KiwiSDR WSPR daemon
Loaded: loaded (/lib/systemd/system/kiwiwspr.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2018-08-25 07:04:47 PDT; 6min ago
Process: 22353 ExecStart=/home/pi/ham/kiwiwspr/kiwiwspr.sh -w a (code=exited, status=0/SUCCESS)
Main PID: 22358 (kiwiwspr.sh)
CGroup: /system.slice/kiwiwspr.service
??22358 /bin/bash /home/pi/ham/kiwiwspr/kiwiwspr.sh -w a
??26156 /bin/bash /home/pi/ham/kiwiwspr/kiwiwspr.sh -j a,all
??26487 /bin/bash /home/pi/ham/kiwiwspr/kiwiwspr.sh -d a,KPH_HF_3,30
??26510 ps 28485
Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: User lookup succeeded: uid=1000 gid=1000
Aug 25 07:04:47 PiKPH systemd[22353]: kiwiwspr.service: Executing: /home/pi/ham/kiwiwspr/kiwiwspr.sh -w a
Aug 25 07:04:47 PiKPH kiwiwspr.sh[22353]: Sat Aug 25 07:04:47 PDT 2018: INFO, this server already has a /lib/systemd/system/kiwiwspr.service file. So leaving it alone.
Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Child 22353 belongs to kiwiwspr.service
Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Control process exited, code=exited status=0
Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Got final SIGCHLD for state start.
Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Main PID guessed: 22358
Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Changed start -> running
Aug 25 07:04:47 PiKPH systemd[1]: kiwiwspr.service: Job kiwiwspr.service/start finished, result=done
Aug 25 07:04:47 PiKPH systemd[1]: Started KiwiSDR WSPR daemon.
pi@PiKPH:~/ham/kiwiwspr $
If you don't see a printout like that which indicates the service is running, then you need to:
2) Stop your watchdog daemon: ./kiwiwspr.sh -w z
3) Edit your service file: sudo vi /lib/systemd/system/kiwiwspr.service
add 'a' to the end of the line:
ExecStart=${KIWIWSPR_ROOT_DIR}/kiwiwspr.sh -w
to make it:
ExecStart=${KIWIWSPR_ROOT_DIR}/kiwiwspr.sh -w a
4) Load the edited file and start the service:
sudo systemctl daemon-reload
sudo systemctl start kiwiwspr
sudo systemctl status kiwiwspr
You can also confirm that the daemon is running with './kiwiwspr.sh -w s'
I recently discovered that Raspbian Stretch has a bug which very, very infrequently causes ethernet packets to be dropped. By running 'sudo rpi-update' on my Pi 3b, which upgraded the kernel from 4.7 to 4.14, I have completely eliminated the 2-5 per day kiwirecorder.py restarts from which I was previously suffering. Running 4.14, my WSPR decoding script 'kiwiwspr.sh' can simultaneously record 17 uncompressed audio streams (3.5 Mbps = 750 receive packets per second ) from 3 Kiwis on one Pi without a single restart in 24 hours.
.........
Sep 5 13:44:19 kiwisdr kiwid: 5d:13:18:01.816 01234567 ca_pause 16560 ca_pause_old 192 ca_shift=28 code_creep=-220 code_period_ms=1 code_period_samples=16368
Sep 5 23:05:44 kiwisdr kiwid: 5d:22:39:26.317 01234567 ca_pause 65527 ca_pause_old 55 ca_shift=4756 code_creep=-4811 code_period_ms=4 code_period_samples=65472
Sep 6 03:21:27 kiwisdr kiwid: 6d:02:55:09.367 01234567 ca_pause 16372 ca_pause_old 4 ca_shift=16 code_creep=-20 code_period_ms=1 code_period_samples=16368
Sep 6 03:28:04 kiwisdr kiwid: 6d:03:01:46.068 01234567 UPDATE: check scheduled
Sep 6 06:25:11 kiwisdr rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="430" x-info="http://www.rsyslog.com"] rsyslogd was HUPed
Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.800 .1234567 0 21096.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:02)
Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.808 .123456. 7 18106.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:03)
Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.813 .1234.6. 5 10140.15 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:05)
Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.816 ..234.6. 1 3594.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:56:06)
Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.821 ..23..6. 4 7040.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:06)
Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.830 ..23.... 6 14097.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:04)
Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.840 ..2..... 3 3570.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 Point Reyes Station, California, USA (LEAVING after 23:58:08)
Sep 6 06:25:11 kiwisdr kiwid: 6d:05:58:53.843 ..2..... UPDATE: checking for updates
Sep 6 06:25:02 kiwisdr kiwid: 6d:05:58:54.877 ..2..... UPDATE: version 1.219 is current
Sep 6 06:27:04 kiwisdr kiwid: 6d:06:00:46.261 ..23.... 3 3570.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED)
Sep 6 06:27:05 kiwisdr kiwid: 6d:06:00:47.250 ..234... 4 3594.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED)
Sep 6 06:27:06 kiwisdr kiwid: 6d:06:00:48.192 ..2345.. 5 7040.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED)
Sep 6 06:27:07 kiwisdr kiwid: 6d:06:00:49.144 ..23456. 6 10140.15 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED)
Sep 6 06:27:08 kiwisdr kiwid: 6d:06:00:50.085 ..234567 7 14097.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED)
Sep 6 06:27:10 kiwisdr kiwid: 6d:06:00:51.913 0.234567 0 21096.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED)
Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.048 0.234567 0 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here
Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.052 0.234567 3 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here
Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.052 0.234567 4 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here
Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.053 0.234567 5 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here
Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.053 0.234567 6 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here
Sep 6 06:27:24 kiwisdr kiwid: 6d:06:01:06.053 0.234567 7 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here
Sep 6 06:27:25 kiwisdr kiwid: 6d:06:01:07.831 0.234567 7 GEOLOC: extreme-ip-lookup.com/json/198.40.45.23
Sep 6 06:27:25 kiwisdr kiwid: 6d:06:01:07.834 0.234567 6 GEOLOC: geoloc_task:P2:T36((50.000 msec) TaskSleep)
Sep 6 06:27:25 kiwisdr kiwid: 6d:06:01:07.847 0.234567 5 GEOLOC: geoloc_task:P2:T35((50.000 msec) TaskSleep)
Sep 6 06:27:26 kiwisdr kiwid: 6d:06:01:07.854 0.234567 4 GEOLOC: geoloc_task:P2:T34((50.000 msec) TaskSleep)
Sep 6 06:27:26 kiwisdr kiwid: 6d:06:01:08.039 0.234567 3 GEOLOC: geoloc_task:P2:T33((50.000 msec) TaskSleep)
Sep 6 06:27:26 kiwisdr kiwid: 6d:06:01:08.259 0.234567 0 GEOLOC: geoloc_task:P2:T32((50.000 msec) TaskSleep)
Sep 6 06:29:10 kiwisdr kiwid: 6d:06:02:52.047 01234567 1 18106.05 kHz usb z0 "kiwirecorder.py" 10.14.70.84 (ARRIVED)
Sep 6 06:29:24 kiwisdr kiwid: 6d:06:03:06.048 01234567 1 GEOLOC: 10.14.70.84 sent no geoloc info, trying from here
Sep 6 06:29:24 kiwisdr kiwid: 6d:06:03:06.821 01234567 1 GEOLOC: extreme-ip-lookup.com/json/198.40.45.23
Sep 6 08:40:26 kiwisdr kiwid: 6d:08:14:07.981 01234567 ca_pause 18040 ca_pause_old 1672 ca_shift=20 code_creep=-1692 code_period_ms=1 code_period_samples=16368
Sep 6 10:40:14 kiwisdr kiwid: 6d:10:13:55.890 01234567 ca_pause 18648 ca_pause_old 2280 ca_shift=1964 code_creep=-4244 code_pe
........
Now that the Pi is fixed, I don't see that I need to learn a whole new environment.