Unexpected IQ phase shift in GPSDO Kiwi [workaround in v1.590]
It has been a welcome surprise to find how useful the Kiwi can be for radio research projects when the Kiwi is fed by a GPSDO like the Leo Bodnar products. While the GPS referencing in a standard Kiwi proeuces exceptional frequency accuracy, measuring the doppler shift of WWV and other signals requires the even greater frequency accuracy produced when the Kiwi is fed by an external GPSDO.
In exploring the capabilities of the Kiwi when fed a Bodnar's 66.660000 Mhz external clock (exactly 5555 times the 12000 sample per second audio output wav file), I have sought to verify the frequency accuracy of the Kiwi's signal processing by viewing the IQ display of a 16,665 Mhz RF signal generated from the second output of a dual output Bodnar. That RF is exactly 1/4 the frequency of the 66.66 ADC clock and so I expected to find no long term phase difference on the IQ display.
So it surprises me that that IQ display shows a 360 degree rotation in about 120 seconds. My oscilloscope shows no phase shift between those two Bodnar signals during that 120 seconds, so it seems that something in the Kiwi's signal processing to the IQ display runs off of a different clock than the ADC.
My goal is for kwirecorder to create a stream IQ wav files with no lost IQ samples, so if this IQ display drift is introduced only in the IQ display extension it won't be a problem for my application.
So before I start validating IQ wav files, should I expect the same drift in them that I observe on the IQ display?
Comments
Yeah, that's not right. Is the PLL turned off on the IQ extension?
There was a bug spotted by Glenn back in 2019 that was causing unexpected rotation/jitter of VLF stations even with GPS/GPSDO referenced Kiwis. Thread here: https://forum.kiwisdr.com/index.php?p=/discussion/comment/8346#Comment_8346 Maybe something is broken again. I'll have a look.
Well, the VLF stuff looks okay. Attached is a 10 minute run of NLK 24.8 kHz. Setup is all in the URL. Note that the PLL is off. I know the GPS is updating on this Kiwi because the "GPS corrections" counter at the bottom left of the IQ panel increments regularly. Stable path at midnight, northern OR to northern WA. S5 signal.
To replicate your higher-frequency setup I would have to fix my hp Z3801 GPSDO (power supply problems, as so many of them have). Then I could lock my hp 8657b to it to produce a decent 66.66. Then split the 10 MHz ref as signal input to the Kiwi as well.
Would it help you to have remote access to my test setup?
Rob,
To what accuracy do we know the Bodnar to be capable of generating the relevant frequencies?
IIRC, the Bodnar uses a chip in the Si5351 family - which means that like the NCO in the KiwiSDR, unless clock frequencies are precisely chosen and/or the modulus of the NCO registers is tailored appropriately, an "exact" frequency can never be produced - only the reduce the residual to something that is vanishingly small - something that can be done in software/FPGA but is harder to manage when committed to silicon.
Clint
All the NCOs use 48-bit frequency registers and phase accumulators as of 2018. The forum thread I mentioned earlier (Nov 2019) makes reference to an even earlier thread (Sep 2018, https://forum.kiwisdr.com/index.php?p=/discussion/1333/external-gps-clock-frequency-suggestion) where this was discussed. Essentially similar behavior observed by Glenn with 25 MHz WWV ground wave and WWVB.
Given the 48 bit NCOs and the phase lock which one would expect from this Bodnar diagram, it seems we don't have an explanation for the drift reported by the IQ display
Yes, I agree completely. Let me work on my test equipment setup (something I've needed to do for a long, long time) and do some checks.
A second dual-output Bodnar arrived today, so I have been able to make further tests.
Since Glenn's N6GN/K2 Kiwi shows very, very little IQ drift when tuned to the line of sight signal from 25 MHz WWV, I have duplicated that test in my lab using 2 Bodnars and I observe the same "almost no drift" results in my lab as I observe on his Kiwi.
In my test one of my Bodnars drives my Kiwi at the same 66,600000 Hz frequency as N6GN/K2. I then set the second Bodnar to output 25,20,15, and 16.65 (1/4 of 66.6) MHz and observed very little IQ drift at 25 Mhz and less at the lower frequencies.
I get similar "almost no IQ drift" results when I feed the Kiwi and external 66.666600 Mhz clock and tune the Kiwi to a signal from the second Bodnar: there is very little IQ drift at 25 and 20 Mhz.
So IQ drift appears to not be a problem with when the ADC runs at 66.6666 Mhz and 66.6 Mhz, but the decoded audio in the wav file is offset by about 0.1Hz.
There is IQ drift when the Kiwi is the 66.660 Mhz which produces no audio frequency errors in the output wav files.
My results suggest to me that something in the Kiwi's demodulation signal processing chain expects the ADC clock to be 66.6666 Mhz. It would be reassuring to underhand why the ADC clock frequency influences that accuracy of the IQ display.
But for radio research purposes it seems that the IQ accuracy is more important than the audio wav file frequency accuracy, so I will proceed to check the IQ wav files using 66.666600 Mhz from a Bodnar.
Okay, I can confirm unexpected very slow IQ rotation. With a 66.66 MHz external adc clk and phase locked 10 MHz rx signal I'm seeing 2:55 min for one rotation of the IQ display. Interestingly, 66.6666 MHz gives one rotation in 1:40 min.
This effect is super small. If for example you change the adc clk +/- 1 Hz the rotation is once per 6.66 secs (the ratio of 66.66 to 10 MHz). So whatever this is it's way down in significant bits of, say, the 48-bit phase word/accumulator of the DDS NCOs. So now I have some suspicions about what it might be.
Fortunately I just got my old laptop working once more so I can run the Xilinx tools and make FPGA changes again. So let me do some poking around..
Alright. I did some stuff in the user interface and I can stop the rotation completely in all cases. 66.66 or 66.6666 or even 66.6659 -- doesn't matter. So this is just some stupid bug related to first time frequency entry. I just need to find it.. 🙄
I have a workaround now, but I still don't understand the root cause. I believe it only happens on the first channel (rx0) and not the others. For example I could configure 8-channel mode and force my first browser connection to rx2 by using
&no_wfin the URL. You can still use the IQ extension even though there is no waterfall (only the audio FFT). I never saw any rotation when configured this way. A subsequent rx0 connection still showed the problem even though rx2 was still connected.The workaround essentially sets the NCO frequency twice for the very first frequency setup of a new connection. This has to be with a little bit of delay between the two. So this is a pipeline bug or something. But so far I haven't found it.
I'll try and get a release out as soon as I can.
Thank you for working on this. I am enhancing WD to record IQ files and need that level of IQ accuracy in order to calibrate my system.
You are sure right about it being small. Even for FST4W at HF I think the spreading it introduces is fairly insignificant for almost all ionospheric propagation, though maybe for chordal propagation (an interesting mode ! ) it might start to be significant.
I'd say most users need not be very concerned about it.
Thanks for pouncing on it John.
Glenn n6gn
Further testing shows my workaround is not perfect. It only fixes the problem roughly 90% of the time. That is, out of 10 page loads (on channel rx0) roughly one of them will show the problem.
How do we know this isn't just a bug in the IQ extension display code? I there any evidence that shows the audio samples are actually slightly out-of-phase w.r.t. the adc clock?
My next test is of the IQ wav files but as I explained in another thread, I need some insight into the format of those files. My goal is to make an hour long IQ recording of 25 MHz WWV from Glenn's Bodnar fed Kiwi which is line of sight to WWV and count the number of samples in the file. If those are correct, then it would seem that the drift in the IQ display extension is generated in it and need not be fixed.
Please give v1.590 a try and see what happens.
Thanks
After 1.586 with 66.60 MHz clock staying perfectly sitll, LOS to WWV25/NIST, 1.590 drifts 1 Hz/minute, clockwise on IQ extension.
Please try it on a channel other than rx0 (e.g. with rx0 already connected).
Also, for rx0, see if the problem persists after every reload of the page.
With both rx0 and rx1 running the same IQ extension and same 25 MHz it appears that neither drifts significantly.
RX2 -RX8 continue to be listening to WSPR.
Running 1.590 with 66.6600 Mhz external clock, on RX1 I see perfect (no movement) of IQ display of 16.665000 Mhz signal derived from the same clock.
Running to 25.0000 Mhz output from a second Bodnar I see very, very small movements of the point on the IQ displa. Those seem entirely consistent with slight drift between my 2 Bodnars.
I will try RX0 later tonight
So 1.590 is a dramatic improvement (in this context) over early code and I think you have fixed the problem I reported. Thanks for this really excellent work.
Now I can try to record and validate IQ wav files!
----
RX0 is as stable as RX1
After killing Rx1, rx0 continues to show no drift even after multple reloads.