Second build sold out. Message will appear here when store is ready for third build ordering.

wsprdaemon - A Raspberry Pi WSPR decoding service

1356

Comments

  • Check to see if your /tmp/wspr-captures file system has overflowed.
    Alternatively, reboot your Pi.
    I will add checks for file overflow condition in future versions.
  • Hi Rob,

    I had wondered if it could be some sort of overflow problem as the reboot seems to have fixed it, at least in the short term.

    I'll keep my eye on it.

    Regards,

    Martin - G8JNJ
  • Hi Rob,

    It's done it again, the noise plots are uploading but not the WSPR spots.

    I've tried rebooting but no luck.

    Which logs should I inspect ?

    Regards,

    Martin - G8JNJ
  • I can see a problem when WD caches many spots due to an internet outage or outage at wsprnet.org. But that should be solved by a reboot
    Also, spots may not get reported if the /tmp/wspr-captures file system fills up.
    Run 'watch df /tmp/wspr-captures' for several minutes to see if overflowing is a problem and increase the size of that file system if it is.
  • Hi Rob,

    I tried that command but I'm not sure what I should be seeing.

    I got the response /tmp/wspr-captures: No Such File or Directory

    and then lines and lines of blocks.

    I'm not a software guy, so I don't know the command string to increase the file size, and any information would need to be spelled out for me.

    I'm running the 2.6 version, but I think I'll rebuild the Raspi OS from scratch and then load 2.5a instead.

    Regards,

    Martin - G8JNJ
  • Update - my missing spots have now appeared on wsprnet.

    I think part of the problem may be the response time of wsprnet as it's taking progressively longer to get any response back from the website. I noticed this a while ago but now it's getting to be much worse.

    I think it's suffering from it's own success and all those automated systems now running that are uploading huge volumes of spots :-)

    Regards,

    Martin - G8JNJ
  • I have been watching the WD upload log and see many, many timeouts on upload attempts to wsprnet.org.
    WD caches it spots until a successful upload so when wsprnet.org goes offline for 10 minutes or more WD can be caching 100 or more spots.
    wsprnet.org seems to reject uploads of 100+ spots (I'm not sure what is that limit, so that is a guess). At that point WD gets into a continuous upload failure loop while its cache of spots grows larger and large.
    I am going to add a limit of upload size to WD 2.6b to try to address this problem.
    But the larger problem is the frailty of wsprnet.org.
    I maintain a research grade database of noise data at wsprdeamon.org and I am thinking I will add a spots database as well.
    WD users will be able to make real queries of their spot data and create graphs from queries.
    PowernumptyG0LUJ
  • If you did make a database one thing I'd really like is the ability to run two or more systems and break down which works better on a band.
    I have one KiwiAI on a LZ1AQ loop (14Ch) and a second KiwiBBG on a Wellbrook (running 7 so I have a slot free to check the antenna).
    Currently it takes a long time to really get a feel for adding a band from the more limited BBG.

    As usual a very personal reason for a feature but I do wonder if someone had say three loops (+ nulls) with otherwise identical setups, they could detect an opening or atmospheric anomaly in a specific direction.
  • edited December 2019
    Hi Stu,

    >
    >If you did make a database one thing I'd really like is the ability to run two or more systems and break down which works better on a band.
    >

    I've started a new thread on this subject.

    http://forum.kiwisdr.com/discussion/1817/using-wspr-data-to-compare-antennas

    Regards,

    Martin - G8JNJ
  • I have made antenna comparisons at KPH by configuring the second antenna system to log spots as KPH/N.
    Then Jim's site shows a comparison by band of the two using: "curl www.wa2zkd.net:8088/today.html | html2text | grep KPH"
    More detailed comparisons can be made at Phil's excellent site: wspr.vk7jj.com/"
    Powernumpty
  • edited December 2019
    Hi All,

    Anyone else been contacted by Corrie - M0XDK - WSPRNet Admin, regarding optimising uploads ?

    Edit -Rob I have copied you in on an email regarding this.

    Regards,

    Martin - G8JNJ
  • Hi Martin,
    I have not received an email on this subject. My email address is in 'wsprdaemon.sh -h' output
    thanks
    rob
  • Hi Rob,

    That's the address I included you on as a cc - I've just resent it please check spam etc.

    Regards,

    Martin - G8JNJ
  • I've not seen anything but I'm not on the usual databases like QRZ so it would take a few minutes more to find an email address.
    Cheers
    Stu
  • Rob: isn't html2txt called html2text ?
  • yes, thanks for the correction
  • I have just checked in V2.6b to https://github.com/rrobinett/wsprdaemon where you will find installation and upgrade instructions.
    Among other improvements, this version addresses an upload problem which may be causing WD users to pound the wsprnet.org database.
    So I strongly encourage existing user to upgrade to V2.6b
    PowernumptyG8JNJHB9TMC
  • In the notes it recommends "mv ~/wsprdaemon ~/wsprdaemon.save"
    Should not be something like "mv ~/wsprdaemon/wsprdaemon.conf ~/wsprdaemon.save"

    On my non standard VM I tried a couple of unsucessfull git pulls and obviously got it wrong (nothing changes eh) so for speed just ended just removing the whole directory (graphs and all).
    Figured a days worth of QRM was fine to lose if it meant not hammering wsprnet.org

    Thanks for the project
    Stu
  • Hi Stu,

    That's what I ended up doing too.

    Probably a good idea to have a proper clean out anyway :-)

    My thanks to Rob for the updates.

    Regards,

    Martin - G8JNJ
  • Might want to update the title of this thread?
    (Both kiwis are now providing audio streams to a pair of Pi4s, both now happily appearing to decode at 2.6a version.)
  • edited December 2019
    One Pi4 can support about 30 Kiwi-sourced channels of wspr decoding and noise logging
  • :) Yes, I know that one pi4 can happily take the output of of two BBG Kiwis, but my Kiwis are one in Limerick and one in Zurich, so not to easy to do one for both.
  • edited December 2019
    I have just checked in V2.6b code which further improves the upload operations. The code is also more wsprnet.org server friendly, so hopefully it will help suppress some of the long hangs of the past few weeks.
    Get it with 'git pull'
    HB9TMC
  • Good afternoon,
    after some time, I am again a happy user of WD installed on my VMware Workstation Player 15 (Virtual machine with Ubuntu 18 LTS).

    The updates posted on Github can be easily installed on a regualar basis by "git pull"... I like that easy maintainance...

    Now I have a question:
    I am running 2 kiwi's and have configured them in the .conf file for configuration of WD
    However what do I need to do to always spot the strongest decode of a single particular call at a given moment. Or in other words, do I need to configure the 'merge' parameter ? What is the purpose of the merge parameter and how to use it ?
    Can someone give an example, please.

    Currently I am testing a lot with various antennas - the problem is the heavy interference and broadband noise from local later afternoon to late evening from strong LED-Noise and strong radiation from the neighbors VDSL2 telecom connection. Trying to phase-out the noise sources without succes, unfortunately. Afternoon till late evening is almost useless for the moment.
    Summer is much better here, when LED-lamps are not needed in the evenings (due to late daylight hours..)

    Thanks for info about the :'merge' parameter.

    Ulli, ON5KQ
  • examples for wsprdaemon.conf


    "MERGED_RX_0 K74,K76 WA2ZKD FN13ed XXXXX"

    K74 and K76 I declared as receivers

    declare WSPR_SCHEDULE=(

    "00:00
    K74,2200
    MERGED_RX_0,630
    MERGED_RX_0,160
    MERGED_RX_0,80
    MERGED_RX_0,80eu
    MERGED_RX_0,60
    MERGED_RX_0,60eu
    MERGED_RX_0,40
    MERGED_RX_0,30
    MERGED_RX_0,20
    MERGED_RX_0,17
    MERGED_RX_0,15
    K74,12
    MERGED_RX_0,10

    "
  • I owe to Jim the suggestion to implement this important MERG feature in WD.

    Here is a fuller explanation and example of the use of MERGed receivers:
    1) MERG_K0_K1 merges the spots decoded from KIWI_0 with those decoded from KIWI_1
    2) MERG_K0_A0 merges the decodes from KIWI_0 with decodes from the analog audio output of a 10M receiver
    3) MERG_K0_A1 merges the decodes from KIWI_0 with decodes from the analog audio output of a 20M receiver

    This is *NOT* diversity reception where the RF or audio signals are mixed. The decodes are performed separately on the signals from each of the MERGed receivers, from which WD picks the best set of decodes to be posted to wsprnet.org.

    As a configuration example, here is a section of the conf for my Pi4 with two local Kiwis and two USB audio dongles attached:
    declare RECEIVER_LIST=(
            "KIWI_0                  192.168.1.20:8073     AI6VN         CM88mc    NULL"           ### Kiwi fed by 160M - 10M antenna
            "KIWI_1                  192.168.1.21:8073     AI6VN         CM88mc    NULL"           ### Kiwi fed by 2200M - 160M antenna
            "AUDIO_0                     localhost:1,0     AI6VN         CM88mc  foobar"               ### Audio from a receiver tuned to 10M.  The id AUDIO_xxx is special and defines a local audio input device as the source o 
            "AUDIO_1                     localhost:2,0     AI6VN         CM88mc  foobar"               ### Audio from a receiver tuned to 20M
            "MERG_K0_K1                  KIWI_0,KIWI_1     AI6VN         CM88mc  foobar"          ### The antennas useful ranges overlap on 160M
            "MERG_K0_A0                 KIWI_0,AUDIO_0     AI6VN         CM88mc  foobar"        ### A 'virtual' receiver useful only for 10M WSPR band
            "MERG_K0_A1                 KIWI_0,AUDIO_1     AI6VN         CM88mc  foobar"        ### A 'virtual' receiver useful only for 20M WSPR band
    )
    
    declare WSPR_SCHEDULE=(
       "00:00  MERG_K0_A0,10   KIWI_0,15   KIWI_0,17   MERG_K0_A1, 20   KIWI_0,30   KIWI_0,40   KIWI_0,80   MERG_K0_K1,160   KIWI_1,630   KIWI_1,2200"
     
    
    When so configured, on 20M WD merges the signals received by AUDIO_1 with those from KIWI_0 and posts only one copy (the strongest) of each received signal.
    Similarly on 10M WD merges the signals received by AUDIO_0 with those from KIWI_0 , and on 160M WD merges signals from KIWI_0 with those from KIWI_1.

    Not only will MERGed rxs avoid double posting spots to wsprnet.org, but as importantly when WD is run at verbosity level 1 (start as ./wsprdaemon.sh -va'), WD generates decode reports each two minutes which show all the spots in a MERG rx and which spot was chosen to be posted to wsprnet.org. Here is an example from another WD site of what you can learn in those posting.log files:
    
    odroid@wspr:/tmp/wspr-captures$ ls -lrt /tmp/wspr-captures/*/*/posting.log
    -rw-r--r-- 1 odroid odroid      0 Jan  9 21:10 /tmp/wspr-captures/MERGED_RX_0/2200/posting.log
    -rw-r--r-- 1 odroid odroid  32232 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/17/posting.log
    -rw-r--r-- 1 odroid odroid  87501 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/20/posting.log
    -rw-r--r-- 1 odroid odroid  22376 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/12/posting.log
    -rw-r--r-- 1 odroid odroid  23300 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/60/posting.log
    -rw-r--r-- 1 odroid odroid  26226 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/10/posting.log
    -rw-r--r-- 1 odroid odroid  28614 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/15/posting.log
    -rw-r--r-- 1 odroid odroid  27073 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/80eu/posting.log
    -rw-r--r-- 1 odroid odroid  47324 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/160/posting.log
    -rw-r--r-- 1 odroid odroid  88536 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/30/posting.log
    -rw-r--r-- 1 odroid odroid  24427 Jan 10 04:41 /tmp/wspr-captures/K74/2200/posting.log
    -rw-r--r-- 1 odroid odroid  53565 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/630/posting.log
    -rw-r--r-- 1 odroid odroid  65804 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/80/posting.log
    -rw-r--r-- 1 odroid odroid  24378 Jan 10 04:41 /tmp/wspr-captures/MERGED_RX_0/60eu/posting.log
    -rw-r--r-- 1 odroid odroid 172637 Jan 10 04:42 /tmp/wspr-captures/MERGED_RX_0/40/posting.log
    odroid@wspr:/tmp/wspr-captures$ tail -n 20 /tmp/wspr-captures/MERGED_RX_0/40/posting.log
    Fri Jan 10 04:39:59 UTC 2020:   7.039999    W4ENN        -16    -27     -16p
    Fri Jan 10 04:39:59 UTC 2020:   7.040049    N6KOG        -28            -28p
    Fri Jan 10 04:39:59 UTC 2020:   7.040050    ZS3RF        -13    -13p    -14
    Fri Jan 10 04:39:59 UTC 2020:   7.040069   WA8KNE          1     -0       1p
    Fri Jan 10 04:39:59 UTC 2020:   7.040073   KJ4IHN        -23            -23p
    Fri Jan 10 04:39:59 UTC 2020:   7.040106    K4APC         -5    -14      -5p
    Fri Jan 10 04:39:59 UTC 2020:   7.040156    N9QDS        -20    -21     -20p
    Fri Jan 10 04:39:59 UTC 2020:   7.040160         -12    -17     -12p
    Fri Jan 10 04:42:10 UTC 2020:  FREQUENCY     CALL POSTED_SNR     K74     K76       TOTAL=19, POSTED=11
    Fri Jan 10 04:42:10 UTC 2020:   7.039999   KD9JYV        -19    -22     -19p
    Fri Jan 10 04:42:10 UTC 2020:   7.040000          -28            -28p
    Fri Jan 10 04:42:10 UTC 2020:   7.040014    AF4MP        -26            -26p
    Fri Jan 10 04:42:10 UTC 2020:   7.040028    W4DJW         -9     -9p    -10
    Fri Jan 10 04:42:10 UTC 2020:   7.040039     KK1D        -22    -22p    -22p
    Fri Jan 10 04:42:10 UTC 2020:   7.040058    W5MWI        -23    -24     -23p
    Fri Jan 10 04:42:10 UTC 2020:   7.040068   VE3OWV        -23    -26     -23p
    Fri Jan 10 04:42:10 UTC 2020:   7.040126   VE6DAI        -22    -24     -22p
    Fri Jan 10 04:42:10 UTC 2020:   7.040141   KK4YEL        -24    -26     -24p
    Fri Jan 10 04:42:10 UTC 2020:   7.040158    W1CEW        -21            -21p
    Fri Jan 10 04:42:10 UTC 2020:   7.040162    W9RAN         -1     -3      -1p
    odroid@wspr:/tmp/wspr-captures$
    
    You can see from that table that on 40M at 04:42 UTC rx "K76" is generally 1-3 dB more sensitive than "K74"
    G0LUJ
  • Thanks to Rob for a better post..... when I put that up earlier, I lacked the time to be more thorough. Here's a simple bash script I wrote to make comparing RX easier

    #!/bin/bash
    #
    mrx=MERGED_RX_0
    #

    cat /tmp/wsprdaemon/$mrx/*/posting.log | grep -v FREQ | awk '{ printf ("%5s\t %2s\t %8s\t %.3f\t %9s\t %3s\t %3s\t %3s\n", $2, $3, $4, $7, $8, $9, $10, $11)}' | sort -k4,5 |sort -u

    printf "%5s\t %2s\t %8s\t %4s\t %9s\t %3s\t %3s\t %3s\n" "Month" "Day" "Time" "Freq" "Call" "POST" "RX1" "RX2"
  • It is rewarding for me to have users of this feature.
  • you and I both..... while you did the work, it's been cool to see how well the merge thing does...
  • There's also a tool/method to compare the posts (SNR ) from each of Merged receivers. This is interesting because it gives insight into angle of arrival, noise and other aspects of each receiving system. Unfortunately I've forgotten how to invoke it!
Sign In or Register to comment.