About Python KiwiClient [Kiwi API question]

bay · March 2021

Hello, everyone

I want to write a program to control Kiwisdr by C#

Just need to achieve switching frequency, recording, listening to sound

I learned Kiwiclient

But I am not good at python

The program for optionParse can run, but cannot debug, and cannot view logic

Has the engineer documented it, or is there a simple demo?

I connected the websocket in Python

But copy sent information, always disconnected, do not know the transmission of information rules

thank you

jks · March 2021

The Kiwi protocol requires the client to send "keep alive" messages at least once every 60 seconds. Otherwise it is assumed the client (Javascript in the browser usually) isn't really running even if the web socket connection is still open for some reason. This had to be done in order to prevent connections from remaining open for long periods of time even through no one was really there. If no keep alive is received the connection is closed.

I don't know if it helps, but I was able to make web socket connections and send Kiwi commands and receive Kiwi sound and waterfall binary data using a program called websocat (web socket concatenate). See: https://github.com/vi/websocat There are binaries available for Windows, Linux and Mac and I was able to run the Mac one without problems.

So, to connect to a Kiwi and get sound data do something like this from the Mac command line:

cat snd.ws.txt | websocat_mac -n ws://my_kiwi:8073/12345678/SND

Where my_kiwi is the name of the Kiwi and the file snd.ws.txt contains these commands to send to the Kiwi:

SET auth t=kiwi p=

SET AR OK in=12000 out=44100

SET squelch=0 max=0

SET genattn=0

SET gen=0 mix=-1

SET ident_user=kiwirecorder.py

SET mod=am low_cut=-5000 high_cut=5000 freq=7550.000

SET agc=1 hang=0 thresh=-100 slope=6 decay=1000 manGain=50

SET keepalive

12345678 is a number and needs to be unique for each connection (hence a timestamp in msec). The websocat -n argument keeps the connection to the Kiwi open after all the commands in the file are sent. But note the connection will still close because additional SET keepalive commands are not being periodically sent.

What you will receive from the Kiwi will be a combination of messages and data looking something like this:

MSG client_public_ip=[your public ip address as seen by the Kiwi]

MSG rx_chans=4

MSG chan_no_pwd=0

MSG chan_no_pwd_true=0

MSG is_local=0,1

MSG max_camp=4

MSG badp=0

MSG version_maj=1 version_min=447

MSG load_cfg= (large amount of configuration data omitted)

MSG center_freq=15000000 bandwidth=30000000 adc_clk_nom=66666600

MSG audio_init=1 audio_rate=12000 sample_rate=12001.062106

MSG stats_cb={"ct":4633,"ce":0,"cf":1000,"cc":0,"cu":[51],"cs":[49],"ci":[0],"ac":0,"wc":0,"fc":0,"ah":0,"as":0,"sr":12001.062106,"ga":0,"gt":0,"gg":0,"gf":0,"gc":66.665900,"go":0,"gr":"RF82ci","ad":0,"au":0,"ae":0,"ar":0,"an":4,"an2":32,"ap":[0,7439,0,0],"ai":[0,2121,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sa":9,"sh":9,"tu":"04:04","tl":"17:04","ti":"NZDT","tn":"Pacific%5c%2fAuckland"}

SND [binary sound data]

...

You can do something similar for the waterfall with the command line:

cat wf.ws.txt | websocat_mac -n ws://my_kiwi:8073/12345678/W/F

And the wf.ws.txt file:

SET auth t=kiwi p=

SET zoom=0 cf=7550.000000

SET maxdb=-10 mindb=-110

SET wf_comp=0

SET wf_speed=1

SET ident_user=kiwirecorder.py

SET keepalive

And you'll receive the waterfall data:

MSG rx_chans=4

MSG chan_no_pwd=0

MSG chan_no_pwd_true=0

MSG max_camp=4

MSG badp=0

MSG version_maj=1 version_min=447

MSG center_freq=15000000 bandwidth=30000000 adc_clk_nom=66666600

MSG kiwi_up=1 rx_chan=0

MSG extint_list_json=%5b%22colormap%22,%22cw_decoder%22,%22devl%22,%22DRM%22,%22fax%22,%22FFT%22,%22fsk%22,%22IBP_scan%22,%22iframe%22,%22iq_display%22,%22loran_c%22,%22navtex%22,%22noise_blank%22,%22noise_filter%22,%22sig_gen%22,%22S_meter%22,%22SSTV%22,%22TDoA%22,%22timecode%22,%22wspr%22%5d

MSG wf_fft_size=1024 wf_fps=23 wf_fps_max=23 zoom_max=14 rx_chans=4 wf_chans=4 wf_chans_real=4 wf_setup

MSG zoom=0 start=0

MSG wf_fps=1

MSG request_dx_update

W/F [binary waterfall data]

...

This discussion is not complete because we need to talk about the parameters for the various SET commands and also the content of the returned binary data. These details are all handled by the kiwiclient / kiwirecorder Python code and that's why you should really try and make an effort to understand it.

bay · March 2021

Thank you😁

XPloRR · August 13

Is there anywhere a C example of a waterfall client?

Need to understand the Waterfall binary image format and protocol.

I am working on a Java client for mobile.

Has someone got a description of the parameters of:

SET auth t=kiwi p=

SET zoom=0 cf=7550.000000

SET maxdb=-10 mindb=-110

SET wf_comp=0

SET wf_speed=1

SET ident_user=kiwirecorder.py

SET keepalive

Can you specify a min and max frequency to view.

What does zoom 0,1,2,3,... mean?

maxdb? mindb?

Another question:

How can I implement a signal strength meter for the audio?

Is there somewhere in the returned audio data a signal strength value returned?

Is it in: MSG rx_chans=4 ? or in somewhere in the

MSG stats_cb={"ct":4633,"ce":0,"cf":1000,"cc":0,"cu":[51],"cs":[49],"ci":[0],"ac":0,"wc":0,"fc":0,"ah":0,"as":0,"sr":12001.062106,"ga":0,"gt":0,"gg":0,"gf":0,"gc":66.665900,"go":0,"gr":"RF82ci","ad":0,"au":0,"ae":0,"ar":0,"an":4,"an2":32,"ap":[0,7439,0,0],"ai":[0,2121,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"sa":9,"sh":9,"tu":"04:04","tl":"17:04","ti":"NZDT","tn":"Pacific%5c%2fAuckland"}

?

jks · August 13

Look at how Kiwiclient / Kiwirecorder works: https://github.com/jks-prv/kiwiclient

It answers all your questions. But you'll have to make an effort to understand how that code works.

XPloRR · August 14

I studied the kiwirecorder and kiwiclient.

I can now capture the audio and show the waterfall in java in my prototype.

But I still have a problem trying to understand the waterfall bandwidth show as a function of the zoomlevel and centerfrequency using the command f.i.:

SET zoom=0 cf=7550.000000

If I i use zoom level 0, I notice the bandwidth is from 0-30000 kHz.

But what is the upper and lower frequency for f.i.

SET zoom=1 cf=7550.000000

I first tought the fomula of the BW=30000/(2^zoom)

But that seems not correct.

Can someone give me the exact formula for lowest and highest frequency of the waterfall as a fuction of cf and zoom?

smg · August 14

Hopefully this helps

The displayed waterfall span at zoom level z is

BW = 30 MHz / 2^z.

Given a centre frequency cf (in kHz), the visible frequency limits are:

BW_kHz = 30000 / (2^zoom)
f_low  = max(0, cf - BW_kHz/2)
f_high = min(30000, cf + BW_kHz/2)

At zoom=0 the Kiwi always shows the full 0–30000 kHz waterfall, regardless of cf. For zoom >= 1, the span centres on cf (clamped so you never go below 0 or above 30000 kHz). (KiwiSDR)

Example with your numbers

Command: SET zoom=0 cf=7550.000000
Waterfall span: 0–30000 kHz (full band). (KiwiSDR)
Command: SET zoom=1 cf=7550.000000
BW_kHz = 30000 / 2 = 15000
f_low = 7550 − 7500 = 50 kHz
f_high = 7550 + 7500 = 15050 kHz
So you’ll see 50–15050 kHz.

Why this matches what the Kiwi does

Zoom levels are power-of-two subdivisions of 30 MHz (e.g., WF7 ≈ 234 kHz, WF8 ≈ 117 kHz), which is exactly 30 MHz / 2^z. (KiwiSDR)
The waterfall data is 1024 bins wide; bin width is Δf = BW/1024. At zoom=0, that’s about 30 kHz/bin. At the max zoom (z=14), Δf ≈ 1.8 Hz/bin. (KiwiSDR)
In the protocol/URLs the frequency values are in kHz (e.g., ?f=22500 for 22.5 MHz), same unit used by the cf you’re sending. (KiwiSDR)
The Kiwi’s nominal coverage is 0–30 MHz (some installs set 32 MHz), hence the clamping in the formula above. (RTL-SDR, kiwisdr.com)

Mapping bins to frequency (handy for your Java code)

If you index bins i = 0..1023:

Δf = BW_kHz / 1024
bin i spans [ f_low + i*Δf , f_low + (i+1)*Δf )   (in kHz)

That should make your prototype line up perfectly with what the Kiwi sends.

John (Kiwi’s author) explicitly describes the zoom behaviour and bin counts here: https://forum.kiwisdr.com/index.php?p=%2Fdiscussion%2F1472%2Fsetting-dispalyed-bandwidth

smg · August 14

Here is a more detailed reply:

Waterfall span is a power-of-two slice of the Kiwi’s HF range. At zoom level z the span is:

BW = Fmax / 2^z

where Fmax is the Kiwi’s configured top frequency (usually 30000 kHz, sometimes 32000 kHz). With a centre frequency cf (in kHz) the displayed limits are:

f_low = clamp(cf - BW/2, 0, Fmax - BW)

f_high = f_low + BW

(That clamping keeps the full span on-screen when you’re near DC or the top end.)

Concrete examples (cf = 7550 kHz, Fmax = 30000 kHz)

zoom=0: span is the whole band → 0 … 30000 kHz (centre is ignored at z0).
zoom=1: BW = 30000 / 2 = 15000 kHz → 50 … 15050 kHz.
zoom=2: BW = 7500 kHz → 3800 … 11300 kHz.
zoom=3: BW = 3750 kHz → 5675 … 9425 kHz.

A couple of useful implementation notes

The waterfall is 1024 bins wide, so bin spacing is Δf = BW / 1024.
At zoom=0 that’s ≈ 30 kHz/bin; at zoom=14 it’s ~1.8 kHz/bin.
At zoom=0 the Kiwi always shows the full waterfall (0–Fmax) regardless of cf.
Some installations set Fmax = 32000 kHz (admin option), so plug that into the same formula if needed.

That should make your Java prototype line up perfectly with what the Kiwi sends.

Where this comes from

Kiwi’s zoom levels are powers-of-two subdivisions of the full band: z0 = 30 MHz, z1 = 15 MHz, … z7 ≈ 234 kHz, z8 ≈ 117 kHz, z14 ≈ 1.8 kHz.
Waterfall data uses a 1024-point FFT, and at zoom=0 the span is 0–30 MHz (or to Fmax) — exactly what you observed.
Example server messages show zoom_max=14 and the waterfall FFT size, which matches the above.

If you want to sanity-check at zoom=1, set cf to the middle of a 15 MHz segment (e.g., 22.5 MHz) and you’ll get a span starting on the 15 MHz boundary with ~15 kHz/bin resolution — exactly 15–30 MHz over 1024

XPloRR · August 14

Thank you! This is clear and helps a lot. This is what I already expected.👍️

I am already quite far with my prototype.

Now I have to polish the interface.

XPloRR · August 14

XPloRR · August 14

Interface working towards something like this (raw prototype), should run on mobile.

XPloRR · August 14

Have to integrate prototype 1 and 2. Still a lot of work to do and debugging.

smg · August 14

The bandwidth available via the API is quite narrow (only a few narrow IQ streams at once.)

There is an SDR++ fork that will connect to a remote kiwi

https://github.com/sannysanoff/SDRPlusPlusBrown

I could be speaking out of turn here, however this subject has come up before and I believe John could be working on making this more available

jks · August 14

The bandwidth available via the API is quite narrow (only a few narrow IQ streams at once.)

Only for the audio. Waterfall is completely separate. How is this not understood after all I've written about it? E.g. https://forum.kiwisdr.com/index.php?p=/discussion/comment/21295/#Comment_21295

Wide-bandwidth mode (relatively speaking) supporting SDR++ is/was an experiment. It is not being worked on at present. Too much other work of immediate impact to customers going on.

jks · August 14

The waterfall image above look very much like the Kiwi waterfall averaging mode (which I'm sure almost no one uses) is enabled. I.e. third menu from the left on the WFn tab of the user control panel. There is API controlling this. It should be off by default. When I connect to the URL shown it is off. But perhaps this is just the way your early prototype is working.

XPloRR · August 15

It is a very early prototype to test the principles and guide myself towards the interface I want to build. I would like to create the nostalgia of the old Kenwood receivers, simplified and combined with a waterfall interface and translated into a portrait mode interface for mobile phones.

ps. I am very new to KiwiSDR (I just discovered it a week ago or so, re-activating my early interest in shortwave radios). So I am a real rookie in this field. I have a background in Engineering and software development, that's all. So I will definitely make mistakes interpreting/understanding KiwiSDR.

XPloRR · August 15

I extended my interface prototype (waterfall not yet implemented here)

smg · August 15

interesting, I assume this is on Android?

What platform are you using for development?

XPloRR · August 15

It should run on IOS and Android and HTML (pc/linux). I am busy translating my prototype to javascript and HTML. So it should work on all platforms. I wrap it later using Android studio in an apk.

XPloRR · August 15

Prototype with waterfall is now working:

todo: bugs, bugs, bugs

implement band buttons, nr, squelch, memory handling, serverlist selection box and so much more

XPloRR · August 15

Implemented the bands and the correct zoom factor for the waterfall.

Implemented the different modulations and the according lo and hi bw

Implemented the scrolling over the waterfall and displaying the bandwidth

Implemented scrolling outside the waterfall and left right

Implemented swiping up down on individual frequency digits

Implemented setting the frequency using the numerical pad

XPloRR · August 15

I have to implement a signal meter. How and where can I find the signal information? Is there a need for a separate socket to open or can I find it in the audio socket data somewhere?

jks · August 15

The S-meter data is in the header at the beginning of every audio packet. See Kiwirecorder for how to decode. Specifically client.py:

  def _process_aud(self, body):
    ...
    smeter,  = struct.unpack('>H', buffer(body[5:7]))
    ...
    rssi   = 0.1*smeter - 127

rssi will be the S-meter value in dBm. So the unsigned 16-bit, big-endian byte-order value in bytes 5-6 of the packet.

jks · August 15

Why does your waterfall look like "streaks" of data? Like you're plotting the same data across multiple scan lines? Rather than one waterfall data set per scan line?

XPloRR · August 15

Thank you that is helpful information! It is a pity there is no real protocol documentation or wiki.

Another thing I search for is how can I implement NR (Noise reduction). Is there a command ("SET ...) to send that info? And what values can it have?

About your comment about the scan lines. I do not quite understand what you mean by streaks. Do you see them horizontally or vertically?

XPloRR · August 16

How often should I send a keepalive signal on the socket? I now set it to 5 seconds.

jks · August 16

It is a pity there is no real protocol documentation or wiki.

In the ten years since the product was introduced there was very, very little need for it. And the people who did need it were able to figure it out on their own, like the original author of Kiwirecorder. The time was much better spent on user/admin documentation (which is still in a poor state). And adding features users wanted.

Do you see them horizontally or vertically?

Vertically. See the images below. The first is the normal Kiwi waterfall. The second has the IIR WF filter turned on (with default parameters). Note how there is averaging, but much detail is lost. This is how I think your waterfall looks. No detail. I don't know if this is intentional or not. Just curious..

How can I implement NR (Noise reduction).

The API for that is complicated due to the many NR/NF algorithms and their varied parameters. Even Kiwirecorder only has support for the standard noise blanker and its two gate and threshold parameters. Look at kiwirecorder/client.py::set_noise_blanker() and how that is processed by the Kiwi server code in rx/rx_sound_cmd.cpp. In particular how the commands CMD_{NB,NR}_{ALGO,TYPE} are processed.

How often should I send a keepalive signal on the socket?

Doesn't matter too much. The timeout period is fairly large. 30 seconds or one minute I think.

XPloRR · August 16

Thank you for your information!

This is the latest release 12 and the new waterfall. Color settings can now be changed.

Still have difficulties to derive a significant s-meter. Help is welcome.

XPloRR · August 17

Noise reduction command works like:

NR off:

SET nr algo=0

NR on

SET nr algo=3

SET nr type=0 param=0 pval=1

SET nr type=0 param=1 pval=0.95

SET nr type=0 param=2 pval=100

SET nr type=0 param=3 pval=0

SET nr type=0 en=1

SET nr type=1 param=0 pval=1

SET nr type=1 param=1 pval=0.95

SET nr type=1 param=2 pval=100

SET nr type=1 param=3 pval=0

SET nr type=1 en=1

XPloRR · August 18

I need help: I try to implement the SQUELCH function. After sniffing in the browser, I found the format:

SET squelch=0 param=0.00 (off)

SET squelch=40 param=0.00 (max)

However, it does not work. What am I doing wrong?

Do I need to specify more params?

Is there another command needed (an initialization?) or does it only work under certain conditions (compression=1)?

jks · August 18

Noise reduction command works like:

Basically. Now look at rx/rx_noise.h. That will tell you the values for nr_algo (the nr_algo_e enum). So there are the 3 NR algorithms you see on the NR section of the main control panel audio tab: WDSP, LMS(Orig), Spec. Now WDSP and Spec(Spectral) have both a denoiser and an autonotch function. So that's why there's the nr_type (nr_type_e enum) parameter. Type is zero to set the denoiser parameters etc. The Spec algo only has one function, so type is always zero when algo = 3 etc.

Howdy, Stranger!

Categories

About Python KiwiClient [Kiwi API question]

Comments

Howdy, Stranger!

Quick Links

Categories

About Python KiwiClient [Kiwi API question]

Comments