Working beautifully here, on a Pi4b running Buster, SWLIO52RP as the callsign as I've not yet earned my license.. 5 threads running from my Kiwi, not much load to be seen on the Pi. The noise graph is a great addition compared to the kiwiwspr daemon which I was running for the past few weeks.
Question mainly for Rob, On the noise graphs. Is there an easy way to cull unwanted graphs? I set up a few Kiwi definitions so that I could easily identify cape and enclosure, over time and through various tests I have managed to generate lots of graphs that I no longer need. I expect with enough time I could probably work it out myself but other tasks call. Cheers Stu
Your question is not clear to me. You can of course download the RP microSD image and burn it to a microSD card before your RPi arrives. Bur installing and configuring WD must be done on a running RP, although you don't need a Kiwi to install and configure WD.
Asking for a friend... If someone was mucking around with configs, changing a config file before jobs had properly exited and ended up with zombie jobs, where would my friend remove those? I.E. from wsprdeamon -s
"Got pid '25922' from file, but it is not running"
Thanks Rob, the second line kept giving me "-bash: kill: (24950) - No such process" with a different PID each time. I ended up deleting the recognisable-from-error bits of expected.jobs and hhmm.sched (in the wsprdaemon directory)
The way I broke it was to have one .conf file for the BeagleboneAI and another for normal running, I was using "wsprdaemon -z" so had assumed after that I could just swap .conf files (using WinSCP). Once a realised something was not working I assumed I had broken something local fired up a VM image created earlier thinking that would be OK "wsprdaemon -z" and replaced the wsprdaemon.conf file causing the same issue (and facepalm).
I probably need Numpty level instruction for how to cleanly substitute one .conf file for another. Also if I set up a clean VM for store/backup what command should I use for a short test to ensure connection, decode and graphs are working but avoid setting the service to automatically start it to avoid this issue. I will RTFM after work if it's in there.
This is all on a VM so I know I'm creating extra work..
By "I" obviously I mean my friend, I would not do anything so stupid.
Another thing that occured to me as a possible feature for those users with multiple antennas. If there was an antenna "profiles" section where someone could define
"Maglp 220,630,160,80,80eu" "Vert1 30,20,15,10",
The user could define a shedule as "KIIWI_1,Maglp KIWI_2,Vert1" etc. that way it is easy to ensure the correct profile is being applied to the right antenna at the right SDR.
that kill line will generate that error printout as it kills itself, but it always kills all other WD tasks before printing the error. you should have no problem changing conf files as long as you -z with the same conf file as you -a
WD logs to ~/wsprdaemon/watchdog.log. so "tail -f watchdog.log" to see error printouts. For more logging verbosity, start WD with '-v -a', or for even more '-vv -a'
Check the call sign you entered in the RECEIVER_LIST receiver definition lines of your wsprdaemon.conf You can also get a more verbose log in /tmp/wspr-captures/uploads.d/uploads.log (and other log files) by stopping WD with '-z' and then starting it with '-v -a'
wspr_spots.txt contained lists of some stations spotted earlier, but is now empty
curl.log contains
/home/martin/wsprdaemon/wsprdaemon.sh: line 2232: curl: command not found
uploads.log contains
Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() starting curl MEPT transfer attempt #1 Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: curl returned error code => 127, so try again /home/martin/wsprdaemon/wsprdaemon.sh: line 2232: curl: command not found Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() starting curl MEPT transfer attempt #2 Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: curl returned error code => 127, so try again /home/martin/wsprdaemon/wsprdaemon.sh: line 2232: curl: command not found Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: timeout after trying 2 curl xfers Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() starting curl MEPT transfer attempt #1 Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: curl returned error code => 127, so try again /home/martin/wsprdaemon/wsprdaemon.sh: line 2232: curl: command not found Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() starting curl MEPT transfer attempt #2 Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: curl returned error code => 127, so try again /home/martin/wsprdaemon/wsprdaemon.sh: line 2232: curl: command not found Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: timeout after trying 2 curl xfers
I had WD 2.5a running on VMware Workstation 15 player Installed Ubuntu 18.04LTS with all additional software necessary.
WD runs flawless for some days. However suddenly i realized, that some bands do not spot from one of the two kiwis anymore. I found it very strange... but verified it. In the beginning only very few spots are missing - after a while no spots at all on 30m any more from Kiwi_01 - with nothing changed.... so:
Then I stopped WD-service with './wsprdaemon25a.sh -z' after all services stopped, I rebooted both Kiwi Beagles Then I close VMwware Workstation 15 player and restart it.
It don't restart, but throws a lot of allerts in a terminal like window For example: 'Failes do start Snappy daemon'
I do not know anything about Linux so I cannot do anything with the following alert: See 'systemctl status snapd.service' for details I cannot input anything anywhere in the VMware terminal console... where should I look for the details and how ?
I made a copy of the VM-machine, at a time when it still worked So I tried to start up the old VM-machine file - same problem...
It is the second time VM-machine crashed running WD 25a
Has anyone found such problems before?
I also found that a service of WD ensures autostart of WD, when Linux starts up. However - I don't run my Kiwi's remote - so I don't need it - I have the impression that this 'autostart'service might conflict with VMware Workstation...
Somebody an idea for a solution ? So far after 2-3 days everything crashes and all software including Ubuntu itself needs to be rebuild from scratch!
How that can be prevented ? What is wrong with my environment Recent strong hardware Windows10 host with lots of memory Large VMware memory etc.. Hardware limitations cannot be reason for software flaws...
Ulli, ON5KQ
P.S.: unfortunately I cannot copy/paste the VM-ware terminal alerts as I do not get the alert messages out of VM-ware terminal..
To disable the 'autostart' feature of WD run: sudo systemctl disable wsprdaemon. Then run WD -z. Then execute 'rm -rf /tmp/wspr-captures/*' to clean out the temporary file system files used by WD. You can also clean out log files in '~/wsprdaemon/' Then run 'top' to see if there are any lingering wpsrdaemon programs. WD should not autostart after restarting your VM. WD is running on several Ubuntu 18.04 LTE systems, but a VM instance probably has subtle differences that I can't check.
Rob, thanks for info on that... I need to reinstall everything, as I cannot input this now, because the VM doesn't start Linux anymore. May be it is best to buy a rasberry pi mini-computer and run it native.
Which mini-PC, I should buy to run 2 Kiwi's including the Noisegraphs plus an audio stream from a different receiver (ELAD) for a third antenna being in use? I think a standard Rasberry cannot handle the noisegraphs with 16 channels running (and eventually even more) However I do not want to spend the money for a large desktop PC running day and night...
I have just brought up a battery backed up 1GB Pi4b at KPH to replace the 16 core x86 server running on AC power. The Pi 4 has plenty of cpu for 14 channels of WD including noise graphing, and I expect you can add several more CPU intensive apps without disrupting WD. That Pi can be bought for the same $35 as a 1 GB Pi3, although you might want to spend an extra $10-20 for the 2 GB or 4 GB Pi4 to support future applications. You will need a USB-c power supply connection and a micro HDMI cable. So in my case I bought a basic kit for the first Pi4 There is a trick to getting a full 1920x1080 VNC desktop on the Pi 4. Search for my recent instructions on that topic in the Pi forms
I noticed a "problem" that puzzles me: On around the 27th of October, I noticed that the "KA7OEI-1" WSPR reports from the Northern Utah WebSDR system disappeared from both the KB9AMG and WA2ZKD statistics (I only checked those two) - but it was still showing up in the noise graphs and on WSPRNet.org.
I decided to stop - and restart - the WSPRDaemon system completely (even though this happens piecemeal every night, anyway) and the stats - perhaps coincidentally - started showing up on the aforementioned systems once again.
Is it possible (how?) that the WSPRDaemon had something to do with this, or was it some sort of problem - apparently selective - with wsprnet.org?
Hi Clint, wsprnet.org is the source of ZKD and AMG lists, so if your spots were in wsprnet.org but not ZKD/AMG, then the problem was not in WD. There is a seperate flow of data and files to create the WD logs and graphs, so uploads could fail while graphs appear. So I have no explanation for your extended disappearance. There are some WD log files which could potentially grow to fill the /tmp/wspr-captures file system, but without access to your PC while in a failure state I can't really diagnose the cause.
Your GPS was diagnosed and fixed by JKS v1.335. In a busy 8 channel mode the Kiwi was not searching for new satellites. I encounters your GPS problem at KPH during WD v2.6 development and can certify that the problem has been fixed.
This afternoon I got wsprdaemon running on a Raspi.
It was all working OK and produced noise graphs in the in the /var/www/html folder
Edit - the graph in the tmp/wsprdaemon/ folder is being updated.
On running wsprdaemon.sh -s and I occasionally get a Python program exit: close failed in file object destructor; sys.excepthook is missing message.
I believe those are otherwise symptomless events, but I have seen those error messages on both Pis and x86 Linux machines. In working on v2.6 I think I have eliminated them.
Then I guess I haven't fixed that error message ;=( There is one new 'hidden' feature in v2.6 which you might find useful: run WD -va and 'tail -f ~/wsprdaemon/watchdog.log' and you will see 'OV' messages about Kiwi overloads.
I've now been successfully running wsprdaemon on a Raspi for about a month.
However on inspecting my wsprnet stats I noticed that nothing had been reported for a couple of weeks, even though the noise graphs were updating OK on the various websites and the individual band logs were up to date.
I couldn't see anything obvious in the wsprdaemon logs, so I did a reboot and everything now seems to be back to normal.
Any suggestions as to what I should look for if this happens again ?
Comments
5 threads running from my Kiwi, not much load to be seen on the Pi.
The noise graph is a great addition compared to the kiwiwspr daemon which I was running for the past few weeks.
Thank you
On the noise graphs. Is there an easy way to cull unwanted graphs?
I set up a few Kiwi definitions so that I could easily identify cape and enclosure, over time and through various tests I have managed to generate lots of graphs that I no longer need.
I expect with enough time I could probably work it out myself but other tasks call.
Cheers
Stu
If someone was mucking around with configs, changing a config file before jobs had properly exited and ended up with zombie jobs, where would my friend remove those?
I.E. from wsprdeamon -s
"Got pid '25922' from file, but it is not running"
kill $(ps aux | awk '/wsprdaemon/{print $2}')
rm -rf /tmp/wspr-captures/*
wsprdaemon -a
I ended up deleting the recognisable-from-error bits of expected.jobs and hhmm.sched (in the wsprdaemon directory)
The way I broke it was to have one .conf file for the BeagleboneAI and another for normal running, I was using "wsprdaemon -z" so had assumed after that I could just swap .conf files (using WinSCP).
Once a realised something was not working I assumed I had broken something local fired up a VM image created earlier thinking that would be OK "wsprdaemon -z" and replaced the wsprdaemon.conf file causing the same issue (and facepalm).
I probably need Numpty level instruction for how to cleanly substitute one .conf file for another. Also if I set up a clean VM for store/backup what command should I use for a short test to ensure connection, decode and graphs are working but avoid setting the service to automatically start it to avoid this issue.
I will RTFM after work if it's in there.
This is all on a VM so I know I'm creating extra work..
By "I" obviously I mean my friend, I would not do anything so stupid.
If there was an antenna "profiles" section where someone could define
"Maglp 220,630,160,80,80eu"
"Vert1 30,20,15,10",
The user could define a shedule as "KIIWI_1,Maglp KIWI_2,Vert1" etc. that way it is easy to ensure the correct profile is being applied to the right antenna at the right SDR.
you should have no problem changing conf files as long as you -z with the same conf file as you -a
WD logs to ~/wsprdaemon/watchdog.log. so "tail -f watchdog.log" to see error printouts.
For more logging verbosity, start WD with '-v -a', or for even more '-vv -a'
With excellent help from Stu, I now have wsprdaemon running on a VM image on my PC.
It all seems to be functioning correctly, all the processes seem to be fine and I can see noise graphs being updated etc.
The only bit that I can't get to work is the upload of spots to wsprnet.
I can see time stamped text files with reported spots being updated in the /tmp/wspr-captures/uploads.d/wsprspots.d/G8JNJ_IO80pw folder.
But none of these appear on wsprnet.
Any suggestions ?
Regards,
Martin - G8JNJ
Check the call sign you entered in the RECEIVER_LIST receiver definition lines of your wsprdaemon.conf
You can also get a more verbose log in /tmp/wspr-captures/uploads.d/uploads.log (and other log files) by stopping WD with '-z' and then starting it with '-v -a'
Rob
/tmp/wspr-captures/uploads.d/
wspr_spots.txt contained lists of some stations spotted earlier, but is now empty
curl.log contains
/home/martin/wsprdaemon/wsprdaemon.sh: line 2232: curl: command not found
uploads.log contains
Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() starting curl MEPT transfer attempt #1
Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: curl returned error code => 127, so try again
/home/martin/wsprdaemon/wsprdaemon.sh: line 2232: curl: command not found
Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() starting curl MEPT transfer attempt #2
Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: curl returned error code => 127, so try again
/home/martin/wsprdaemon/wsprdaemon.sh: line 2232: curl: command not found
Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: timeout after trying 2 curl xfers
Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() starting curl MEPT transfer attempt #1
Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: curl returned error code => 127, so try again
/home/martin/wsprdaemon/wsprdaemon.sh: line 2232: curl: command not found
Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() starting curl MEPT transfer attempt #2
Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: curl returned error code => 127, so try again
/home/martin/wsprdaemon/wsprdaemon.sh: line 2232: curl: command not found
Wed 30 Oct 17:14:56 GMT 2019: uploading_daemon() ERROR: timeout after trying 2 curl xfers
Regards,
Martin - G8JNJ
Execute 'sudo apt-get install curl' to get it.
Looks like I missed that, should have tested it more, had assumed as it ran and populated noise graphs etc it was good.
Thanks again for the software Rob.
Stu
Ubuntu
Yes that fixed it all now OK - Thanks guy's.
Regards,
Martin - G8JNJ
Installed Ubuntu 18.04LTS with all additional software necessary.
WD runs flawless for some days.
However suddenly i realized, that some bands do not spot from one of the two kiwis anymore. I found it very strange... but verified it. In the beginning only very few spots are missing - after a while no spots at all on 30m any more from Kiwi_01 - with nothing changed.... so:
Then I stopped WD-service with
'./wsprdaemon25a.sh -z'
after all services stopped, I rebooted both Kiwi Beagles
Then I close VMwware Workstation 15 player and restart it.
It don't restart, but throws a lot of allerts in a terminal like window
For example:
'Failes do start Snappy daemon'
I do not know anything about Linux so I cannot do anything with the following alert:
See 'systemctl status snapd.service' for details
I cannot input anything anywhere in the VMware terminal console... where should I look for the details and how ?
I made a copy of the VM-machine, at a time when it still worked
So I tried to start up the old VM-machine file - same problem...
It is the second time VM-machine crashed running WD 25a
Has anyone found such problems before?
I also found that a service of WD ensures autostart of WD, when Linux starts up. However
- I don't run my Kiwi's remote - so I don't need it
- I have the impression that this 'autostart'service might conflict with VMware Workstation...
Somebody an idea for a solution ?
So far after 2-3 days everything crashes and all software including Ubuntu itself needs to be rebuild from scratch!
How that can be prevented ? What is wrong with my environment
Recent strong hardware Windows10 host with lots of memory
Large VMware memory etc..
Hardware limitations cannot be reason for software flaws...
Ulli, ON5KQ
P.S.: unfortunately I cannot copy/paste the VM-ware terminal alerts as I do not get the alert messages out of VM-ware terminal..
Then run WD -z.
Then execute 'rm -rf /tmp/wspr-captures/*' to clean out the temporary file system files used by WD. You can also clean out log files in '~/wsprdaemon/'
Then run 'top' to see if there are any lingering wpsrdaemon programs.
WD should not autostart after restarting your VM.
WD is running on several Ubuntu 18.04 LTE systems, but a VM instance probably has subtle differences that I can't check.
I need to reinstall everything, as I cannot input this now, because the VM doesn't start Linux anymore.
May be it is best to buy a rasberry pi mini-computer and run it native.
Which mini-PC, I should buy to run 2 Kiwi's including the Noisegraphs plus an audio stream from a different receiver (ELAD) for a third antenna being in use?
I think a standard Rasberry cannot handle the noisegraphs with 16 channels running (and eventually even more)
However I do not want to spend the money for a large desktop PC running day and night...
Ulli, ON5KQ
The Pi 4 has plenty of cpu for 14 channels of WD including noise graphing, and I expect you can add several more CPU intensive apps without disrupting WD.
That Pi can be bought for the same $35 as a 1 GB Pi3, although you might want to spend an extra $10-20 for the 2 GB or 4 GB Pi4 to support future applications.
You will need a USB-c power supply connection and a micro HDMI cable. So in my case I bought a basic kit for the first Pi4
There is a trick to getting a full 1920x1080 VNC desktop on the Pi 4. Search for my recent instructions on that topic in the Pi forms
I decided to stop - and restart - the WSPRDaemon system completely (even though this happens piecemeal every night, anyway) and the stats - perhaps coincidentally - started showing up on the aforementioned systems once again.
Is it possible (how?) that the WSPRDaemon had something to do with this, or was it some sort of problem - apparently selective - with wsprnet.org?
Any ideas?
73,
Clint
KA7OEI
wsprnet.org is the source of ZKD and AMG lists, so if your spots were in wsprnet.org but not ZKD/AMG, then the problem was not in WD.
There is a seperate flow of data and files to create the WD logs and graphs, so uploads could fail while graphs appear.
So I have no explanation for your extended disappearance. There are some WD log files which could potentially grow to fill the /tmp/wspr-captures file system, but without access to your PC while in a failure state I can't really diagnose the cause.
Your GPS was diagnosed and fixed by JKS v1.335. In a busy 8 channel mode the Kiwi was not searching for new satellites. I encounters your GPS problem at KPH during WD v2.6 development and can certify that the problem has been fixed.
I have posted a late apha version of WD2.6 in github: https://github.com/rrobinett/wsprdaemon/tree/rrobinett-v26
Logging of overload events is one new feature you might find useful.
This afternoon I got wsprdaemon running on a Raspi.
It was all working OK and produced noise graphs in the in the /var/www/html folder
Edit - the graph in the tmp/wsprdaemon/ folder is being updated.
On running wsprdaemon.sh -s and I occasionally get a Python program exit: close failed in file object destructor; sys.excepthook is missing message.
Is this correct or are there any fixes for this ?
Simple step by step guide please :-)
Regards,
Martin - G8JNJ
I believe those are otherwise symptomless events, but I have seen those error messages on both Pis and x86 Linux machines.
In working on v2.6 I think I have eliminated them.
Rob
I'm using the late alpha version of WD2.6 found in github: https://github.com/rrobinett/wsprdaemon/tree/rrobinett-v26
Regards,
Martin - G8JNJ
There is one new 'hidden' feature in v2.6 which you might find useful: run WD -va and 'tail -f ~/wsprdaemon/watchdog.log' and you will see 'OV' messages about Kiwi overloads.
I've now been successfully running wsprdaemon on a Raspi for about a month.
However on inspecting my wsprnet stats I noticed that nothing had been reported for a couple of weeks, even though the noise graphs were updating OK on the various websites and the individual band logs were up to date.
I couldn't see anything obvious in the wsprdaemon logs, so I did a reboot and everything now seems to be back to normal.
Any suggestions as to what I should look for if this happens again ?
Regards,
Martin - G8JNJ