IQ File Recording Format

I've had a request to add support for KiwiSDR IQ recording files to Amalgamated DGPS.

Is there any special header information as some other SDR programs include, such as center frequency, bandwidth, etc. or is it necessary to extract that from the filename?

What is the format of the file, just stereo with alternating I and Q values? (And which is first?)

Comments

  • jksjks
    edited February 2019
    That information is only included in the filename currently.

    There is an option for IQ format files (only) to include GPS timestamp information as metadata at periodic intervals in the data portion of the file. The .wav file spec allows for this. But in practice files with this metadata included in the data stream (as opposed to metadata in the header at the beginning of the file) keeps some playback applications from working which is unfortunate. The spec says applications are supposed to ignore metadata chunks they don't recognize.

    It's an open question whether adding metadata to the header would also cause problems with some playback applications. It would be a real headache if we were to make such a change only to have people start complaining that newly recorded files won't playback for them.

    For IQ files, yes, 2-channel (stereo), data is sent I-first, with I and Q samples interleaved.

    An IQ file recorded with kiwirecorder on my Mac. Filename shows 1400.000 kHz CF but no passband. Generally IQ recordings are maximum passband though. Mode for a non-IQ mode file would show in place of "iq" in the filename, e.g. "am", "usb" etc. Sample rate will either be 12000 or 20250 Hz depending on if Kiwi was in 4/8 or 3 channel mode.
    >>> afinfo 20190226T183722Z_1440000_iq.wav 
    File:           20190226T183722Z_1440000_iq.wav
    File type ID:   WAVE
    Num Tracks:     1
    ----
    Data format:     2 ch,  20250 Hz, 'lpcm' (0x0000000C) 16-bit little-endian signed integer
                    no channel layout.
    estimated duration: 24.803556 sec
    audio bytes: 2009088
    audio packets: 502272
    bit rate: 648000 bits per second
    packet size upper bound: 4
    maximum packet size: 4
    audio data file offset: 44
    optimized
    source bit depth: I16
    ----
    
    ChrisSmolinski
  • edited February 2019
    Thanks for the info, very helpful!

    Yes, I suspect many apps expect there to be only one data chunk, not broken up between many metadata chunks. I know mine do :smile:

    As long as the file name format doesn't change (and users don't rename files), parsing out the starting timestamp and frequency works fine, I've just finished doing that in Amalgamated DGPS, and am running some test files though it now. I have to do the same thing for SDR Sharp files. Hopefully there won't be any cases where two apps use a file naming system similar enough to cause confusion between them.

    I'm recording two 20 kHz wide files for my tests, which together barely cover the DGPS band. Well, almost, since there's some European stations that are just below 285 kHz, and I am not taking into account any attenuation near the passband edges, so you might need 3 to cover the entire band. I suppose someone could run multiple instances and record the 3 at the same time.

    I see kiwirecorder uses a different filename format vs the browser, I'll take a look at supporting that as well. First I guess I need to get kiwirecorder running on my Mac.
  • jksjks
    edited February 2019
    I see kiwirecorder uses a different filename format vs the browser, I'll take a look at supporting that as well. First I guess I need to get kiwirecorder running on my Mac.
    I didn't realize that. But the recording function from the Kiwi UI was (greatly appreciated) contributed code.
  • No big deal as I've got it handling the filename format from kiwirecorder, which didn't take too long. Just parsing the data in a different way.

    Now that I have that working... I decided to do something I always wanted to do with kiwirecorder (once I started to use it), use it as a base for a program to route audio from a KiwiSDR to a virtual audio cable, so it can be used with digital decoding apps. I've got that working, in a crude way. But I have made some assumptions, like the sample rate is 12 kHz, and the data comes in 2048 samples at a time. Are these OK assumptions to make?
  • jksjks
    edited February 2019
    I'm a little confused. Sure, people use VACs all the time to route Kiwi audio coming from a browser to decoder apps (e.g. your apps, fldigi, Dream DRM etc.) But is what you want a modified kiwiclient that has a VAC endpoint as opposed to file recording like what kiwiclient/kiwirecorder does? And doing so while the audio is not necessarily being played out of the speaker? (for which any VAC is already capable of handling I would imagine).

    There is a fairly new addition to kiwiclient called kiwi_nc (i.e. Kiwi netcat). Just like Linux netcat it outputs to stdin/out as the endpoint. It hasn't been extensively tested, but that might be an easier place to start than refactoring the kiwirecorder code (although I see now that it's currently missing the "--resample" option which I really should add).

    About the Kiwi sample rate and packet size: Because the Kiwi you're connecting to could be in 3-channel mode (hence sample rate 20.25 kHz instead of 12) you can't assume the sample rate. Better to use the --resample option and get kiwiclient to upsample the 12 or 20 kHz Kiwi rate to a constant 44.1 or 48 kHz. Perhaps that's what some VACs require anyway I'm guessing? If you have some sort of FIFO buffer in the Python code then the Kiwi packet size shouldn't matter. The packet size sent by the Kiwi *is* different depending on if compression is being used or if the mode is IQ vs non-IQ. So again, you want your code to be independent of that.
  • Yes, that's exactly what I am doing, routing the audio to a specified sound output device. Thanks for the suggestion to use --resample, that solves the potential problem. I noticed the different packet sized depending on compression, and from testing with both options it turns out to not be an issue. Now to continue removing unneeded code from kiwirecorder. Although an other choice would be to leave everything there, and instead have a option to route sound to an output device instead of writing to a file...
Sign In or Register to comment.