The KiwiSDR 2 online store is open for orders! Please visit kiwisdr.nz

SNR Ranking Algorithm?

vk5qivk5qi
in Installation, Antennas and Interference

Hi all,

Is there any documentation (or even better - source code) available for how the SNR calculations used on http://rx.linkfanel.net/snr.html are performed? The 'contact us' link only goes to a GitHub page for a different project, and nobody has been responding to the Issues for a long time.

I would like to investigate how the SNR performance of our stations (the two VK5ARG stations) varies over time, and with various modifications at the site, and hence would like to run the algorithm myself.

Regards,

Mark VK5QI

Comments

  • la3rkla3rk
    1412BA5F-B1BA-4197-9B4E-9AFC5A3493A8.jpeg

    Something like this? Basically a scan every 5 minutes, calculates average level and p90 level (peak) and defines SNR (red line) as the difference between the two. Can be set up for different frequency spans.

    Collected data is fed through a standard tool called rrdtool and shows snr as it varies over time.

    Software are somewhat rough and unfinished. Is based on software found in the kiwi github collection.

    73 de Olaf - LA3RK

  • vk5qivk5qi

    Yes! That looks pretty close to what I'm after.

    Any source code available?

    73

    Mark

  • la3rkla3rk

    Written in Python, needs some tuning and setup. I have not debugged every line of code, but it does what I wanted at least on this location. Runs on a pi on my local net, ie same as the one the Kiwi is on.

    Basically two scripts, one to contact the kiwi (run via crontab), then a script preparing the graphs. Graphs are copied to directory where a webserver can present them or you can pick the graphs directly via ftp or similar.

    Crontab included below;

    # m h dom mon dow command

    # 0 - 3750 kHz / 3750 - etc

    */5 * * * * /usr/bin/python3 /home/ubuntu/KiwiClient/snrtorrd.py -z 3 > 0_3750s.txt

    */5 * * * * /usr/bin/python3 /home/ubuntu/KiwiClient/rrdtograph.py -z 3 > 0_3750g.txt

    7-59/5 * * * * /usr/bin/python3 /home/ubuntu/KiwiClient/snrtorrd.py -z 2 -o 3750 > 3750_11

    250s.txt

    7-59/5 * * * * /usr/bin/python3 /home/ubuntu/KiwiClient/rrdtograph.py -z 2 -o 3750 > 3750_

    11250g.txt

    2-59/2 * * * * /usr/bin/python3 /home/ubuntu/KiwiClient/snrtorrd.py -t 120 > 0_30m.txt

    3-59/2 * * * * /usr/bin/python3 /home/ubuntu/KiwiClient/rrdtograph.py -t 120 > 0_30g.txt

    9-59/5 * * * * /usr/bin/python3 /home/ubuntu/KiwiClient/snrtorrd.py -z 1 -o 11250 > 11250_

    26250s.txt

    9-59/5 * * * * /usr/bin/python3 /home/ubuntu/KiwiClient/rrdtograph.py -z 1 -o 11250 > 1125

    0_26250g.txt

    #*/5 * * * * /usr/bin/python3 /home/ubuntu/KiwiClient/snrtorrd.py -z 8 -o 1650 > 1m7s.txt

    #*/6 * * * * /usr/bin/python3 /home/ubuntu/KiwiClient/rrdtograph.py -z 8 -o 1650 > 1m7g.tx

    t

    1-59/5 * * * * /usr/bin/cp /home/ubuntu/*.png /var/www/html

    Some libraries are needed and some knowledge of python. Source files are based on files found in the kiwiclient github location. You probably need some modules from the same location.

    Nice to see antenna performance over time, performance of changed antenna, changes to hash reduction measures, see timed interference etc.

    73 de Olaf - LA3RK


  • vk5qivk5qi

    That's brilliant, I'll have a go setting that up. If I make changes, do you mind if I put the code up on somewhere like Github?

    73

    Mark VK5QI

  • la3rkla3rk

    No problem, as you note the code is a rewrite based on some of the code found github/kiwiclient and in particular some code that calculates SNR. I added the bits storing the values into rrd files and a separate part which prepares the graphs. Also some changes to adapt to python3.

    Good luck

    73 de Olaf - LA3RK

  • linkfanellinkfanel

    Is there any documentation (or even better - source code) available for how the SNR calculations used on http://rx.linkfanel.net/snr.html are performed?

    As the page already documents: "These ratios are calculated between the 95th percentile of power levels across the whole spectrum (signal), and the median power level (noise)." The source code I use is not public, but you can look at the microkiwi_waterfall.py utility of the KiwiClient suite, which does basically the same thing. The algorithm is really simple, see the bottom of https://github.com/jks-prv/kiwiclient/blob/master/microkiwi_waterfall.py

    For anyone interested in that kind of optimization work or performance monitoring, I would recommend this tool as a good starting basis, like what LA3RK did.

    The 'contact us' link only goes to a GitHub page for a different project, and nobody has been responding to the Issues for a long time.

    Well I take offense to that. There's no project for the simple score page in itself, and the receiver map project that it links to is the prime user of those SNR scores. There's a single issue open, which is about the SNR scores rather than the map even, and I hadn't replied in two months only.

    I just didn't really have much to reply. What I can offer people interested in improvements to the scoring infrastructure, would be to sponsor me for it, or even better, sponsor jks to make it an internal feature of KiwiSDR receivers - if he'd be interested in this of course. See https://github.com/priyom/dyatlov/issues/1#issuecomment-799128561

    But if you want to contact maintainers of this project other than on GitHub, that's me, and you can always send me an email directly.

    I would like to investigate how the SNR performance of our stations (the two VK5ARG stations) varies over time, and with various modifications at the site, and hence would like to run the algorithm myself.

    I think running microkiwi_waterfall.py should do what you want straight out of the box.

  • vk5qivk5qi
    edited March 2021

    I've got LA3RK's code up and running on a RPi which is sitting out at the site with the KiwiSDRs, and have put the code with the relevant dependencies (e.g. the modified wsclient and mod_pywebsockets, which had me wondering for a while...) up on github here for anyone that wants to get something running with minimal fuss:

    https://github.com/darksidelemm/kiwisnr-rrd

    @linkfanel Maybe some more contact details on the SNR page would be useful... After searching your website a bit more I found your e-mail address on your CV, but it wasn't obvious anywhere else.

    73

    Mark VK5QI

  • la3rkla3rk

    Nice to know that you got the software working. I find it a nice tool to see effect of my rfi reduction efforts.

    i wish you luck in your efforts to improve reception.

    73 de Olaf - LA3RK

  • vk5qivk5qi

    I've also advanced this a little bit further - I found out that the code was clipping the SNR at 40 dB! The SNR at the VK5ARG site exceeds 40 dB quite regularly, so there were many blank spots in the plots.

    The plot for one of the public KiwiSDRs is here: https://rfhead.net/temp/koch/192.168.88.205_0_30000-Daily.png

    This updates every 10 minutes. It's interesting to watch the SNR change over the day. On this receiver site it appears the SNR is more affected by propagation conditions than it is by local noise - as it should be!

    Thanks again for kicking this one off Olaf...

    73

    Mark VK5QI

  • la3rkla3rk

    if I remember the code correctly I made some cutoff in the data whenever the readings were off for my site/antenna. This was to avoid large peaks on the graphs, thought it better to introduce «holes».

    Nice to see that you like the graphs. I find them interesting, I can for instance see when noice sources within the house turns off and on.

    Just another piece of advice, I have experienced pi´s that have crashed running rrd. I do not know if this can be attributed to bad powersupply (glitches) or the rrd software. Would suggest that you keep a backup, lot of work to reinstate a broked pi sd card.

    73

    Olaf - LA3RK

  • ChrisSmolinskiChrisSmolinski
    edited August 2023

    About a year or so ago I played around with the trying to estimate KiwiSDR SNR, using the existing (95th vs 50th percentile) code as a starting point. I played around with various methods, comparing the results to what the overall spectrum "looks like". It's one of those tricky things, you can quite easily glance at a spectrum and decide that's a good high SNR setup, or, well, that other one isn't, but getting software to do it is not so easy. This is what I ended up with. It's a cut and paste from the actual code, which has a lot of dead code in it and would probably be more confusing if I pasted the whole thing, hopefully I did not leave any critical bits out. Also this code evolved over time and is probably far from optimized (just looking at it again now a year later I see some stupid stuff like iterating through the sorted data to find the minimum value 😀

    Basically, I get the spectrum signal data from the KiwiSDR, and sort it. I get the minimum value. I then compute new values with that minimum value subtracted off. I then get what I call the peak value, which contrary to the name is actually the 98th percentile value. I then iterate through the bins and find which one is equal to 40% of that peak value. I divide by ten (just for human scaling purposes), round, and call that the "SDR Quality Factor"

    IIRC the method to my madness was to not just compare two fixed percentile values (95th and 50th) for SNR calculations, but to take into account how the signal levels are distributed over the spectrum. Empirically, I found this gave me a better guesstimate as to the actual reception quality.

    Maybe this will be useful to someone.


    $numKiwiSignalBins =1020;
$numLines=128;
$peakPercentile=0.98;
$percentOf=0.40;

for ($i=0; $i<=$numKiwiSignalBins; $i++) $outputAvg[$i] = $outputSum[$i+13] / $numLines;  
sort($outputAvg);

$minValue=99999;
for ($i=0; $i<=$maxKiwiSignalBin; $i++)
	{
	if ($outputAvg[$i]<$minValue) $minValue=$outputAvg[$i];
	}
		
for ($i=0; $i<=$maxKiwiSignalBin; $i++)
	{
	$dataBins[$i]=$outputAvg[$i]-$minValue;
	}

$index=$peakPercentile*$numKiwiSignalBins;
$bin=intval($index);
$peak=$dataBins[$bin];

$percentBin=0;
for ($i=0; $i<=$numKiwiSignalBins; $i++)
	{
	if ($dataBins[$i] < $peak * $percentOf)
		{
		$percentBin=$i;
		}	
	}

$percentBin=$percentBin/10.0;
$percentBin=round($percentBin,2);
$sum=$percentBin;
$s="SDR Quality Factor " . strval($sum);
  • studentkrastudentkra
    edited August 2023

    Hello everyone! Maybe someone will be interested in this. Can dear @jks add this to the new release))

    I wrote a small script for KiwiSDR. The script collects SNR rating data from the site http://rx.linkfanel.net/snr.html and draws rating charts for a certain time for any KiwiSDR receiver. It is very interesting to evaluate the passage of radio waves during the day, as an example.

    Here is a link to my KiwiSDR ratings. I launched this quite recently, a few hours ago. The charts are clickable and updated every 10 minutes.

    SNR_rate.jpg

    You can easily run this on your KiwiSDR and keep track of your SNR ratings. 

    The script has not been debugged yet and requires improvements.

    http://std.swl.su:8073/config/snr.html

    UPD: Thanks @PSO for the tip. I will make a second table with SNR plots.

Sign In or Register to comment.