v1.577: NBFM and AM/SSB de-emphasis
From the CHANGE_LOG file:
v1.577 Feb 6, 2023
Audio response de-emphasis for AM/SSB and NBFM:
On the main control panel, audio tab, the "de-emp" menu to the right of the volume control
has been greatly improved. For AM/SSB modes the traditional 75 uS (N.Am) and 50 uS (EU)
filter profiles are available. For NBFM a 20 dB / decade (= 6 dB / octave) is available,
with or without ("+LF") attenuation below 400 Hz (e.g. to suppress PL tone etc).
The de-emp menu settings are stored and persistent across page reloads.
Comments welcome. These filters are now FIR-based and designed with Octave from a
frequency response profile. So they are easy to adjust.
A second spectrum peak hold is available. The "Pk" button on the WF tab has been replaced by
"P1" and "P2". Each click on Pk cycled between off/clear(wht) and hold(grn).
But P1 and P2 cycle between three states: off(wht) - on(grn) - hold(red).
This is so one can be in hold mode while the other is on. P1 is the yellow trace and P2 cyan.
The momentary zero icon of the left of the squelch slider has been changed to a toggle.
The default FAX passband has been lowered from 2200 Hz to 1200 Hz. It was observed that for weak
signals this slightly reduced the background noise in the image without lowering the overall
contrast, indicating that the extra passband width was just increasing the noise floor.
Comments welcome. You can type "/2200" or "/2k" in the frequency box to re-widen the passband.
Comments
Hi John,
Excellent additions :-)
The new de-emphasis values make listening much less fatiguing, the narrow nfm setting sounds slightly muffled to my ears at first listening, but there is not much to lisren to at the moment, so I'll reserve my judgement.
The new squelch toggle is much more useful too. However I still find it a bit difficult to find the optimum squelch setting when I move the slider, as the new setting doesn't seem to apply until the slider is released. Would it be possible to adjust the setting whist the slider is still being moved ?
I also really appreciate the second peak hold feature on the spectrum display, as this allows A/B comparisons of signal levels. I prefer the Cyan Slow Dev colour rather than the rainbow hues when using the spectrum display, and I find the cyan second peak hold a bit difficult to distinguish. My spectrum analyser uses cyan, yellow and purple for the traces, and my suggestion would be to use purple for the second maximum hold trace.
I tend to use the maximum hold trace to spot signals that occasionally pop up over a long period of time. This got me thinking if it would somehow be possible to incorporate some form of 'snap to peak' tuning. The idea being that if you move the cursor near to a signal peak on the spectrum display, and click, it would tune the receiver to the nearest peak level. This could be either the current peak or the stored maximum. The latter being useful if you have spotted that a signal has been present, and you wish to tune to that frequency and wait for it to reappear.
Alternatively maybe it could be possible to have an adjustable amplitude threshold, and any signals that exceeded that threshold would be automatically tuned to. A bit like the peak tracking function you would find on a spectrum analyser. Most signals tend to be on 500Hz multiples of frequency, so maybe this would be a suitable resolution to 'snap' to.
Just some thoughts, but all of the new additions, including the reduced FAX bandwidth (I had to manually adjust that) are GREATLY appreciated.
Regards,
Martin
I'm still doubting if an NFM roll-off of 6dB/oct (20dB/dec) starting at 300 or 400 Hz is correct though. It seems very severe compared to the 75/50 uS AM/SSB de-emp.
Since the Kiwi sig gen doesn't have an FM modulator, and I wasn't able to cobble together an AF sig gen input for my 8657b, I could only measure the NFM response on the Kiwi using the noise floor. Probably not ideal. Maybe you could verify with your setup.
But here is the FIR response output from Octave with the design frequency points and values I gave it [The exact values I used are a bit different: see tools/FIR.m in the Kiwi source code. In any event the values are from 6.02 dB * log2(f Hz / 400 Hz) or, equivalently, 20 dB * log10(f Hz / 400 Hz)].
Now this curve looks wrong compared to the usual RC roll-off. It's concave up. But I think that's because the frequency (x) axis is linear, not log. I haven't figured out how to get Octave to plot things the way I want.
Hi John,
I can't guarantee that these first sweeps are 100% correct. I generated them by feeding an audio sweep into an Icom IC3000 transceiver set to wide filter width.
First the KiWi without any de-emphasis.
Next with de-emphasis and LF on. Reasonably flat with a bit of HF roll off. But this assumes the Icom has the correct amount of pre-emphasis applied and the audio response is correct. The lumps and bumps in the response curve, suggest it may have some anomalies.
Also note the difference in audio output when de-emphasis is applied, as you would expect. But maybe the KiWi needs to compensate for the differences, especially relative to the other modes ?
I was a bit concerned about the lack of low frequencies, but I assumed this was because the Icom was filtering the transmit audio to allow CTCSS tones to pass uncorrupted by speech.
I made some further sweeps using an old Marconi signal generator that has got quite bad phase noise. But the audio is flat with no pre-emphasis.
I have zoomed in on the lower part of the audio spectrum.
De-emphasis with no LF
De-emphasis plus LF
De-emphasis off
Finally some wider frequency sweeps with the signal generator and de-emphasis off. The sweeps should be flat. The audio sweep went up to 10KHz with a modulation of +/- 5kHz.
I tried other modulation settings but the recovered audio never seemed to exceed about 4kHz.
NBFM 12kHz receive bandwidth
Narrow NBFM 6kHz receive bandwidth
I hope this helps.
Let me know if you wish me to perform any more tests.
Regards,
Martin
Hi John,
I just made some further comparisons of the broadband background noise level for each of the modes, using the same arbitrary volume settings. The levels will be different again when demodulating actual signals, but I thought this may be a good starting point.
Measurements were made using the excellent (free) digital audio meters from Darkwood designs.
https://darkwooddesigns.co.uk/pc2/meters.html
Using CW with a 500Hz bandwidth as a reference level of 0dB.
USB 3kHz BW no de-emphasis 0dB
USB 3kHz BW 50 or 75uS de-emphasis -1dB
AM 12KHz BW no de-emphasis -3dB
NBFM 12KHz BW no de-emphasis -3dB
NBFM 12KHz BW with de-emphasis -10dB
Narrow NBFM 6KHz BW no de-emphasis -5dB
Narrow NBFM 6KHz BW no de-emphasis -10dB
I think that there needs to be some audio level boost applied when the NBFM de-emphasis is selected, as I find I'm having to move the volume control significantly when changing modes.
Others may be able to contribute further ideas for tests.
Regards,
Martin
I later realised that there was a flaw in my thinking, as the FM noise floor will produce the maximum demodulated output and the AM noise floor will be at a minimum.
Instead I decided to compare the recovered audio from a signal modulated with a 400Hz tone, which should be below the point at which any de-emphasis has a significant effect.
In addition any amplitude modulated signal will be dependent upon the RF gain, so I used AGC and made sure that the signal level was well above the threshold value.
Narrow NBFM 12kHz BW modulation +/- 2.5kHz peak = audio level 0dB
NBFM 12kHz BW modulation +/- 5kHz peak = audio level -1dB
AM 12kHz BW AM 30% modulation = audio level +2dB
AM 12kHz BW AM 50% modulation = audio level +5dB
AM 12kHz BW AM 80% modulation = audio level +8dB
SAM 12kHz BW AM 30% modulation = audio level +2dB
SAU 6kHz BW AM 30% modulation = audio level +2dB
USB 3KHz BW AM 30% modulation (tuned to modulation in upper sideband) = audio level +13dB
CW 3KHz BW AM 30% modulation (tuned to modulation sideband not carrier) = audio level +14dB
KiWi 12kHz BW signal generator FM deviation +/- 2.5kHz peak no pre-emphasis.
Demodulated audio level referenced at 400Hz
Freq De-emph Off De-emph On
400Hz 0dB 0dB
1.0kHz 0dB -6dB
1. 5kHz 0dB -9dB
2.0kHz 0dB -10dB
3.0kHz 0dB -15dB
4.0kHz -8dB -27dB
Note the level with no De-emphasis drops at 4kHz, maybe due to some other filter ?
Maybe Richard can double check that this is making sense, as I don't see a 6dB slope in the noise floor with no signal present and with de-emphasis off that he suggests should be present.
At the back of my mind there is something about FM vs. PM at low modulation indices, and having to apply a 6dB slope to the audio to make PM seem like FM (or maybe was it the other way round). It was a long time ago since I did the theory, but maybe this also applies to the noise distribution ?
Regards,
Martin
Hi John,
Sorry I just realised that should have tried to measure the de-emphasis at the frequencies you used in your simulation. I'll try and do that in future, if it is required.
However my figures and your predicted curve are a pretty good fit up to about 4kHz, when both the flat and de-emphasised response falls off quite rapidly. So I guess there there must be some other filter in operation somewhere in the chain.
This could possibly be why it sounds a bit muffled, even though the basic curve is correct.
Regards,
Martin
I see what's wrong with 4+ kHz. Give me a little while to fix it.
I pushed some changes as a hot fix rather than a full release. Use the "Build now" button on the admin update tab.
There was an unwanted LPF in the FM squelch code attenuating the high end. Also a high end bug in my FIR code. Should be much better now.
The squelch slider now updates the squelch value continuously while it is moved. But note that there is lag due to the audio buffering. So you need to move slowly to fine tune the squelch point.
Spectrum peak hold colors: I forgot about the "slow dev" cyan color. I have changed the second peak trace to magenta.
There is a general problem of the peak trace color contrast against the black background of the spectrum panel and also against the rainbow or cyan spectrum plots. We need to fine tune the defaults. To assist with that I have made all the peak trace parameters javascript variables you can set from the browser javascript console. So you can interactively tune them. I don't want to make these admin parameters. The admin interface is too complicated already and this these don't need to be adjustable. We just need to settle on good defaults. If there are peak traces shown their color/alpha will change immediately when these variables are changed:
spec.peak1_color = 'yellow'
spec.peak2_color = 'magenta'
spec.peak_alpha = 1.0
spec.slow_dev_color = '#66ffff'
Note the usual javascript console typing accelerators apply, so you don't have to laboriously type the full javascript each time (i.e. ^P ^N ^A ^E tab (auto-complete) etc) The colors can be HTML color names or hex values ('#rrggbb', '#rgb') etc.
The alpha (transparency) value used to be 0.55 so the traces didn't completely obscure the rainbow spectrum behind them (the peak traces are drawn over the spectrum plots). But that low a value makes the magenta trace have too little contrast against the black background.
The slow dev color can also be adjusted. The traces look better against a grey slow dev plot as opposed to the current "light cyan" ('#66ffff').
Open javascript console keyboard shortcuts:
Browser // Windows/Linux // Mac
Firefox // ctrl-shift-K // cmd-alt-K
Chrome // ctrl-shift-J // cmd-alt-J
Safari // ctrl-shift-C // cmd-alt-C
Opera // ctrl-shift-I // cmd-alt-I
Hi John,
To the ear that sounds better when listening to noise (not many nbfm signals around at the moment).
I'll do some more tests tomorrow and report back.
Thanks again,
Martin
Hi John,
Some sweeps made today at a low modulation index using 12kHz RX bandwidth.
Frequency De-emph off De-emph on Predicted
400Hz 0dB 0dB 0dB
600Hz 0dB -2dB -3.5dB
1200Hz 0dB -7dB -9.5dB
1800Hz 0dB -11dB -13.1dB
2400Hz 0dB -14dB -15.6dB
3000Hz 0dB -15dB -17.5dB
3600Hz 0dB -18dB -19.1dB
4000Hz -2dB -22dB -20.0dB
5000Hz -3dB -24dB
6000Hz -10dB -24dB
Overall the response pretty close to predicted, and probably not worth bothering about.
-------------------
However when I listened to the background noise and changed the filter bandwidths something seemed a bit wrong.
I think the issue may be an aliasing of signals that are present above the upper cut-off frequency of the filter, being folded back down into the filter passband. I suspect this may result in some of the high level random noise present above the filter upper cut-off frequency appearing within the wanted passband, the noise floor of which will be at a much lower level due to the FM capture effect.
If I perform a low modulation index, upward (low to high) audio sweep stopping at 12kHz, using a KiWi 12KHz filter bandwidth (which produces a 6kHz audio passband).
I notice that when the sweep reaches approximately 6KHz, an aliased downward sweep appears in the audio passband, which continues at a gradually decreasing amplitude until it reaches the noise floor at around 3000Hz. This means that signals in the region of 6 to 9KHz from the carrier frequency, can still be heard.
I can also hear low level sweep artefacts when the sweep reaches around 3KHz up to around 5kHz, but these are not too significant.
I have posted a low quality video showing this occurring at this weblink
https://g8jnj.net/kiwisweep
I'll comment on the colour schemes separately.
Regards,
Martin
Update
On a hunch I decided to try demodulating a plain carrier signal using NBFM, AM, SAM, I/Q and SSB. Just to check tha twhat I was seeing wasn't due to a problem with the signal generator.
The same issue is occurring with all the modes. As soon as the carrier passes the filter upper cut-off frequency, it is aliased back into the receive passband.
You can observe this using the internal signal generator, although the behaviour is slightly different to using an external one.
Set up a 12kHz wide sweep, with it starting at the tuned carrier frequency. Switch off the AGC and set a reasonable RF gain with a bit of headroom, something like about 50dB.
Select I/Q with 12kHz bandwidth, and either listen, or use soundcard scope, to observe the demodulated audio frequency spectrum.
As the carrier crosses the filter upper cut-off frequency, the alias signal starts to appear, and it can be heard (or seen) moving down in frequency through the receiver passband.
If you select NBFM with a similar filter width, you will hear the noise floor quieten as the carrier is within the passband. However it remains quiet even when the carrier has passed through the upper cut-off frequency. It is only when the carrier exceeds just under twice the upper cut-off frequency that the noise floor begins to rise again. Indicating that there is no longer any signal present.
Regards,
Martin
WRT the colours
Shades of Light Grey look quite good for the spectrum.
#b3b3b3 or maybe #999999
However I can't get spec.peak_color1 or spec.peak_color2 to work for me. I tried a few variations of syntax, and had a very quick look through the code to see if I could spot the string in question, but no luck.
I goofed in my earlier post (now fixed). It's really
spec.peak[12]_colorthat you need, notpeak_color[12]Yes that worked, one of the few combinations I didn't try.
The problem is that in order to work with all of the waterfall options they have to be fairly neutral non rainbow colours.
Cyan and Magenta stand out really well and work with most of the other options as they are complementary colours, but they are a bit garish, to my taste anyway.
Pastel shades are also OK and also seem to provide enough brightness against the background
Grey seems to work quite well and stand out against the title bars when set as the spectrum base
spec.slow_dev_color = '#d3d3d3'
White for P1 spec.peak1_color = '#ffffff'
Khaki for P2 spec.peak2_color = '#f0e68c'
Just my suggestions, others may have different opinions.
Regards,
Martin
My earlier comment about triangular noise isn't borne out by my test on another FM demodulator without de-emphasis, which shows a flat (white) noise spectrum. I don't understand this and need to dig further.
On the latest kiwi (v1.577 just rebuilt) I've measured the HF roll-off using FM, 12 kHz IF BW, no de-emphasis, using a Marconi 2019A RF signal generator, an external DDS audio signal generator and about 1 kHz deviation.
The AF out from the Kiwi via Firefox relative to 1 kHz in dB is:
50 +0.8
100 +0.4
200 +0.3
300 +0.2
400 +0.2
1000 0.0
2000 -0.9
3000 -2.3
4000 -4.4
5000 -7.5
5500 -9.8
This roll-off isn't a big deal but flatter would be preferable IMHO. I suspect there's a similar slight slope on AM and USB. It's possible that there's a problem in the RX chain after the Kiwi, which I haven't investigated.
73, Richard G4DYA
Hi Richard,
That's slightly different to what I measured, but it shows a similar trend apart from between 3 and 5.5kHz. At 6kHz I measured -10dB, which is close to your value.
I used 0.5kHz deviation to try and keep the high level modulation sidebands within the receive passband at the higher frequencies, but I don't know how relevant it is to need to do this.
I'm also using a Marconi 2019A signal generator, when it decides to behave itself.
Regards,
Martin
@G4DYA Are you sure you rebuilt v1.577 using the "build now" button on the admin update page? Because that response looks exactly like the unwanted FIR LPF that was hiding in the NFM squelch code that I just removed.
I'm working on some other stuff that should make it into a v1.578 release soon.
@jks Yes I rebuilt about an hour or so before I posted those results.
@G8JNJ At the higher audio frequencies the modulation index is <1 so the higher-order sidebands will be quite small. You can see how small on the Kiwi spectrum/waterfall. (IIRR this is a distinguishing feature of NBFM as opposed to WBFM - it requires only the first-order sidebands, saving bandwidth at the cost of a little harmonic distortion.) My 2019A behaves itself but its idiot owner struggles to remember how to drive it.
It might be worth mentioning Carson's rule for anyone who is unfamiliar with it, https://en.wikipedia.org/wiki/Carson_bandwidth_rule
73, Richard G4DYA
Hi Richard,
At the time I was trying to avoid distortion and additional aliasing signals in the passband whilst I was investigating the sweeps, so used the same setting when making the frequency response measurements. But I understand the valid point you are making.
I used a KiWi with a much earlier version of nbfm demod yesterday, and I was shocked at just how tinny and scratchy it sounded in comparison to the new version, which is way better. So this has definitely been a worthwhile exercise. To the extent that I'm now considering building a proper set of switched down-convertors for some VHF and UHF bands.
BTW Good luck with the HF Radar IDing, it's not easy is it ? At times I think there is some 'spoofing' or emulation of radars used by other countries, so only by using TDoA can you get better idea of who is using what. The Chinese ones seem to have particularly dirty modulation characteristics.
Regards,
Martin
FWIW I saw you guys were working on FM demphasis so I rebuilt version 1.577 just now and listened to some FM voice activity on 10 meters, 12khz bw. Compared to a TenTec Orion I the voice "highs" are significantly reduced with the On or +LF settings. If it wasn't for the tone feedthrough, and sounding harsh, the audio was most intelligible in the Off position. I ran a "soundcard scope" program and enabled the peak hold function and let it run for about 15 seconds for each filter setting with no signals present where I was receiving. See below:
Wayne WA2N
Hi Wayne,
You are correct, something doesn't look right with your traces.
Let's rule out some things first, whilst I try and repeat your findings.
How are you piping the audio from your browser to the Soundcard scope ?
Are any audio enhancements running on your PC ?
Is there any Audio AGC running on any of your sound card sources ?
Regards,
Martin
Maybe there is a problem with the "hot fix" changes I pushed out not getting picked up by use of the "build now" button. I'm trying to get a v1.578 release together sometime today..
Maybe...
However I just tried copying Waynes plots, but mine look different, and more like I would expect them to.
Waynes plots bother me because there isn't a steep roll-off where the filter bandwidth ends, it carries on way past 6kHz.
However there are a few reasons why Waynes plots may look the way they are, including how he is obtaining the audio, and what signals he is listening too.
Let's try listening to a low deviation signal but with 12KHz bandwidth NBFM selected.
Some similarities to Waynes plots, apart from the upper cut-off being better defined.
Now let's have a look at the same signal but using NNFM with a 6KHz bandwidth.
Hmm, that looks pretty flat in all instances, so what's going on ?
Now this is bit is just my theory, and it could well be wrong, or what we are seeing is being caused by something else.
But I think that when listening to a low deviation signal with a wider than necessary bandwidth, there isn't any spectral energy in the 3-6kHz band from the transmission itself, and it's actually the normal fm background noise that is filling that part of the spectrum.
If you look at fm noise with the squelch off using Spectrum Scope, you will see that the response if flat from a few Hz up to the filter cut-off frequency.
When you switch de-emphasis on, it does what it is supposed to do, and rolls off the high frequency noise. On an actual signal we compensate for this by adding pre-emphasis at the transmitter, so that the recovered signal which is de-emphasised has a flat frequency response, and the unwanted noise is reduced with frequency.
Of course all of this relies on the transmitter having the correct pre-emphasis applied in the first place, and some don't. In which case just turn the de-emphasis off.
Any better suggestions or theories anyone ?
Regards,
Martin
Maybe Wayne is running his Kiwi in 3-channel, 20.25 kHz mode which would produce an audio response above 6 kHz?
Maybe ?
There are lots of variable at play here.
However his main concern is that he thinks the KiWi is rolling off too fast, which maybe in comparison to other radios it is, but that's not to say that it's wrong.
I do find that on some signals the audio level is very low in comparison to having the de-emp off. However a lot depends on the spectral distribution of the operators voice (a problem I have), the microphone in use, radio in use and it's frequency response, the deviation in use, etc.
Some CB's and amateur radios seem better than others in this respect.
From my perspective, mainly listening to Amateur NNFM and some 12.5kHz channel spaced commercial traffic, using +/-2.5kHz deviation, it's a major improvement, but other peoples experiences may differ, and in those cases there is always the option of turning the de-emp off, so that it is more like the original configuration.
But it could indeed be, that having the correct de-emphasis curve, that actually meets the specified characteristic is not that common, and that many radios don't follow it, or have the correct pre-emphasis curve on transmit either.
Regards,
Martin
Sorry Guys, I have too many sound devices on this PC and didn't set the input to the scope properly. With the speaker output loopback selected, these shots look better:
I do have a graphic equalizer set up AFTER the soundcard output and before a soundbar on the desk to compensate for high frequency hearing loss that I have, essentially sloping upward above 1 Khz to flatten what I hear. However, that won't affect the graphs above, which now look more like yours. I have not researched what the demphasis curve should look like, I think it's too steep for my personal taste compared to another rig. I rarely listen to NBFM on the Kiwi, so I'm ok with whatever you guys come up with.
Wayne
Hi Wayne,
Is that background FM noise you are listening to, or an actual signal of reasonable strength ?
To me the most recent graphs look like the background noise floor when using FM, and the slope is what I'd expect to see when de-emphasis is applied.
However I've just tried looking at one of my KiWi's receiving just noise, with the de-emp off and it is not correct, with a big step occurring around 4kHz.
But this is because I've not manually updated John's patched version of v1.577
After being manually rebuilt which looks like your plot.
And the same receiving just noise with de-emph +LF on, which also looks like your plot.
Regards,
Martin
The last plots I posted were from a frequency with no signal / modulation. I did a 15 minute search on pre/deemphasis and it looks like 6db/octave 20db/decade is the norm. I even found a post you wrote Martin, so you're obviously more experienced than me! Here's a slide from a very dry lecture on youtube:
(48) 18 -7- 2020 Pre-emphasis and De-emphasis in FM, NBFM and WBFM - YouTube
Anyway, for yuks I fired up an Airspy HF Discovery+ receiver that's connected to my PC in order to compare the roll-off. They really drop out the low frequencies, which sucks, but it looks like the drop at 3.5 KHz is less than the most recent Kiwi response. A better test would have been using my ICOM IC-7300's output for comparison, which I might be able to do in a bit on another PC. Anyway, here's the two graphs Kiwi vs Airspy, maybe the non-linear portion of the response around 500-1000Hz that appears to drop quicker is a factor?
It's possible that the Airspy is using the broadcast WBFM 50, 75 or 100us time constant de-emphasis curve that a lot of receivers seem to have incorrectly also used for NBFM.
It's getting late here, but tomorrow I'll take a look at my Airspys and report back.
I think the extended LF audio response may also fool our brains into thinking it sounds more muffled than it actually is, and the flat part of the NBFM de-emphasis curve on frequencies below about 400Hz may add to that.
On other factor is that you have equalised the audio you listen to in order to compensate for a loss at the HF end. If you have done this so that your favourite radio sounds correct, then maybe because you are used to that as a reference, the new KiWi filtering may sound worse. Psychoacoustics are a strange thing, and difficult to quantify, as everyone hears differently. Blind test A/B HiFi comparisons being a good example.
Regards,
Martin
Here's what the IC-7300 looks like fed from a dummy load, widest filter BW 15 Khz and no signal:
This rolls off pretty fast too, 20 db/decade by eye.