Direction Finding and linking existing KiwiSDRs

ZL1GWEZL1GWE
in TDoA topics
Hi All  Interested in linking exiting KiwiSDRs for the location of signals via ToA or TDOA.
Is this possible?   RTL have a similar approach for VHF/UHF signals could we achieve this within the KiwiSDR network or other SDR networks?
«13

Comments

  • WA2ZKDWA2ZKD
    too many time variables I would think
  • G8JNJG8JNJ
    Hi,

    Take a look at this thread.


    Although it's not available in real time, this may be a step towards it. By processing time stamped off-air recordings and then determining the time difference between each, it may be possible to perform a hyperbolic fix (or similar) on the transmitter location.

    However my maths and coding are not up to the task :-(

    Regards,

    Martin - G8JNJ
    Tonda
  • KA7UKA7U
    Recording multiple instances using the Python coded kiwirecorder, in Linux, might require a setup of multiple users. Each user would have Python installed and each user would coordinate the recording frequencies and times with the other users. Of course the users could all be the same person on the same computer. To make the recordings start and stop at the same time, each user would setup a crontab with the start and stop times. 

    The latency of start and stop of these crontab jobs should be quite low, the latency over the Internet of the various KiwiSDR receivers could be quite large and variable on any given Internet feed forking into the user's recordings.  Therefore, I would doubt the feasibility of geographically diverse combined recordings for worldwide direction finding. IF the KiwiSDR being recorded provided frequent time stamps, and if the Kiwirecorder was equipped to start and stop recording according to these time stamps, well then maybe. 

    On the other hand, individual users, using directional antennas, or conventional FM TDoA, could independently provide compass headings to a coordinator, who could then plot those lines on a map and triangulate to approximate the general location of a transmitter.  TDoA might work quite well from a space satellite. Oh well...
    Ron
    KA7U
  • jksjks
    There have been three or four people recently express interest in working on HFDF. Some on here or on Github issues or in private emails to me. I'll try and point everyone to this thread.

    It is important to note that V/UHF DF is significantly different from the HFDF problem due to the issues imposed by the nature of the ionosphere at HF. It is much more complicated than you would expect. But we have this wonderful resource: lots of geographically diverse Kiwi installations and now GPS timestamped IQ data thanks to Christoph's recent work (https://github.com/jks-prv/Beagle_SDR_GPS/issues/130)

    KA7U
  • WA2ZKDWA2ZKD
    John states "MUCH"..... as someone who does HF R&D 8-5 on my job, I agree!

  • jksjks
    edited November 2017
    My 100% speculative guess, with little knowledge about the subject, is that for non-realtime bearings (which would be okay for us but useless for "other" HF users) some sort of statistical methods would make the problem easier. Make lots of measurements over a long period of time and you might be able to identify and minimize the multi-path and doppler problems.

    The lightning detection guys (http://en.blitzortung.org/live_lightning_maps.php) have it working really well at VLF where it's mostly ground wave.

  • WA2ZKDWA2ZKD
    GW is a whole different, and easier,  animal than Skywave. I have been studying both of those for a similar work task for about 18 months. Tricky stuff.
  • jksjks
    Christoph's latest post on https://github.com/jks-prv/Beagle_SDR_GPS/issues/130 shows how his GPS timestamps are within 20 usec of DCF77 (77.5 kHz) as received by a couple of Kiwis a few hundred km distant. Really fantastic work!

    KA7U
  • ZL1GWEZL1GWE
    Thank you all, who have responded, there is actually method in my madness.  As the Intruder watch coordinator for NZART here in New Zealand, I have a number of volunteer observers.  I was looking to create a method of using the KiwiSDR receivers with their GPS capabilities to make it easier for identifying interfering signals amonst groups, and individuals.   Currently DF finding is ad hoc, done manually and you can imagine coordination is best of luck in terms attempting to lock down the offending country.   There is plenty of algoriths about, using in commercial systems.  But many of these commercial systems are at least 1 Million Euro with a 24x7 fully resourced team doing that role.  Within the Amateur Radio fraternity, I am looking for a repeatable method, which would enhance our capbailities to identify the interfering source with say Google Earth.  The rationale is given the number of Internet of Things or everything - our world is likely to incur greater interference, and also I would like to improve our reporting capabilities - especially within New Zealand, as the Government of the day does not have a HF monitoring capability at all - but depends on handheld spectrum analysers.  I believe VHF and UHF type RTL methods have been validated, but as you state HF is another thing altogether.  But I will keep trying, anything to create a better way of identifying and coordinating the use of existing equipment, Yagi's directional antenna and correlating these for better method of locating the source would certianly increase our capabilities with better means of liaisitng with authorities.  Another option was to use the WSPR network but if I can use a set of SDRs with GPS, this would certainly improve our current capabiltiies.
  • WA2ZKDWA2ZKD
    The current state of the art

    http://www.entropixllc.com/

  • TokenToken
    edited November 2017
    How complex the HF and ionospheric issue is depends on your expectations of accuracy.  Also, these accuracy expectations are going to define other variables that might, or might not, be players.  For example clock jitter measured in 10's of usec.  Since each 10 usec is maybe (simplistically defined) 3 km of error, does 100 usec error, and up to 30 km (60 km if both sample sites have errors in opposite directions), really matter?

    If you only want to determine the area of a transmission, i.e. within a few hundred km, then the ionospheric issue is often not huge.

    If both receive locations in a given pair are experiencing single hop propagation then the altitude of the ionopshere is already factored in between them to some extent.  However if one RX location has single hop and the other has double hop, then the issue could, potentially, be significant.

    I suspect for many users the acceptable accuracy is loose enough that something could pretty easily be done.  Knowing the city of the source is great, but often just the state or even country is good enough for many users.

    For several years I and a few other listeners over at HFUnderground have been using, casually and with no big push, a technique based on audio recordings.  We have often achieve accuracies of under 100 km for targets several thousand km away.  Sometimes accuracy is less, on the order of several hundred km, and occasionally it is better, we have done better than 25 km accuracy.  Even when we have 300+ km errors that still puts us closer than just "I wonder where this signal is from"?

    If a technique could be developed that time tags the audio stream of remotes in some way to an accuracy of +/- 300 usec or better that would be killer.  Even something as simple as a short 1 PPS time tick synced to UTC time in one audio channel (say left channel) and detected audio in the other (say right channel).

    T!
  • G8JNJG8JNJ
    Hi All,

    I agree even a 100km accuracy would be a good starting point. The advantage of the KiWi network is that it is farily well distributed so once you have a rough location it should be possible to use other KiWi's closer to the suspected location to obtain a more accurate fix. Alternatively lots of fixes could be combined to produce a good average.

    In the UK the radio regulatory body (OFCOM) has a distributed network of 24 remote receiver sites with 20MHz to 3/6GHz DF capability. I believe this uses Keysight / Aligent N6841A units as demo'd in the video linked below.



    Some more background info can be found here


    Some of the research before OFCOM deployed the current network.


    Token - I could add a PPS marker to my KiWi which could be switched in by means of the antenna switch option if this would help with any project. I think Peter, G3PLX, came up with this idea some time ago and I'm keeping him informed about the KiWi enhancements WRT TOA DF'ing as I know this was (and may still be) of interest to the IARU and other National Societies.

    Regards,

    Martin - G8JNJ











    KA7U
  • jksjks
    Wow -- check Christoph's latest post: https://github.com/jks-prv/Beagle_SDR_GPS/issues/130#issuecomment-347206064

    It's a plot of a ground wave TDoA solution for DCF77 using three Kiwis relatively close by.

    KA7UZL1GWE
  • ZL1GWEZL1GWE
    Thank you Martin - I have initiated a project with IARUMS Manager and to liaise with Region 1 and Region 3 - Region 2 appears to be off-line.  Getting the project engine initiated and will update later.
  • G8JNJG8JNJ
    Hi John (ZL1GWE),

    Great stuff - John and Christoph are doing some really interesting work here - So our thanks are definately due to both of them.

    Regards,

    Martin - G8JNJ
  • ChristophChristoph
    Some recent examples of TDoA measurements can be found here:
    SM2BYC
  • ZL1GWEZL1GWE
    Excellent is DFC77 likely to be detected here?  Or would you advise connect, only connect to KiwiSDR receivers synchronised to this system?  Could be a disadvantage to us in Asia-Pacific? 
  • G8JNJG8JNJ
    Hi John (ZL1GWE),

    I'm not sure if you intended to send your last post ?

    However the use of DF77 (time station) was just for test purposes as it's location was known, it was a stable time reference which was useful for calibration and being such a low frequency provides surface wave coverage over much of Europe.

    As others have already pointed out, trying to locate transmitter sites via skywave is a LOT more difficult.

    What Christoph has achieved so far is absolutely spectacular, but it's still a very long way to go before he has a working system with a good degree of accuracy, that is simple enough for the average listener to be able to use.

    At my request he has performed some further tests to try and identify a particular source of interference that can be heard in Europe. Although he has produced some promising looking results, it's not clear if they are correct as we don't know where the actual transmitter site is located. Hmm, I should have thought of that before I asked :-( 

    So to mix a few metaphors, with this test we have a 'chicken and egg situation' where we don't know if the software is producing the correct results or iwhere the real source is to validate it. I think my prompting may have inadvertently persuaded Christoph to 'run before he could walk'.

    I suspect that any further tests will have to concentrate on known transmitter sites, until Christoph has a good degree of confidence in the result that is produced.

    Even assuming that signals propagating by sky wave can be traced to their source, it is likely that this may initially only be possible with 'simple' signals such as low speed data or those that have a strong coherent component. Signal such as SSB voice present a lot of additional challenges.

    I think that Christophs work so far is outstanding, and really hope that it can be further developed as a resource that is available to the amateur community.

    Exciting times.

    Regards,

    Martin - G8JNJ










  • ZL1GWEZL1GWE
    Hi Martin

    Thanks very much for the explanation and update.  Fully understand the vagaries of HF as a communications officer in a past life.   But yes, indeed a wonderful piece of work, especially in the amateur community.

    Hopefully, some of the issues can be resolved with known good sources, and ironed out.   But all and all this is far more promising than when I first started out.

    73

    de ZL1GWE

    John
  • ZL1GWEZL1GWE
    It has been some time, since I had chance to review your helpful comments here:  Great work, but it was recently suggested to me that Long baseline Inferometry would be easier, that TDOA for direction finding, especially if one is looking at the Angle of Arrival (AOA).   So that you could derive the direction of the interence or offending signal.  It has been suggested that fewer SDRs would be requird and then the source could be derived?  Any thoughts?
  • ChristophChristoph
    There is no progress on the angle of arrival technique. It would be great if anyone could try this approach.

    But recently I made first steps towards using propagation delays based on an approximation of ray-tracing using IRI2016 electron density profiles https://hcab14.blogspot.com/

    Also the GitHub repository now contains a working example for TDoA (DCF77) with complete wav and GNSS position data
    https://github.com/hcab14/TDoA

    I guess one of the the next steps would be to translate the octave code into python since this would allow to run the TDoA analysis in rear real time using online KiwiSDR data streams. This should not be too difficult, given the fact that python numpy supports very similar matrix operations compared to octave.
  • jksjks
    I was thinking of using the existing proxy.kiwisdr.com server, which has plenty of spare x86-64 cpu cycles, as a backend to run the TDoA code (Octave or Python). An extension on the Kiwi would act as the user interface, collecting the list of remote sampling Kiwis to use, frequency/mode etc. The resulting .png files would be displayed in an extension window and/or downloadable to the local host. There would have to be a queueing mechanism to keep order on the proxy server.
  • bensonbenson
    Currently it is a pretty elaborate process to pull in the data and get everything installed and running under Linux and have Octave do the analysis to produce the TDoA maps.
    For folks using mostly MS Windows it is quite a challenge to get thru the installation steps.

    A GUI especially in the form of a Kiwi extension would make Christoph's excellent
    work on HF TDoA much more accesible to many more Kiwi users.

    Best regards, Ben.
  • jksjks
    Nice article on rtl-sdr.com about the topic: https://www.rtl-sdr.com/locating-various-hf-transmitters-and-number-stations-with-kiwisdrs
    WA2ZKDTonda
  • G8JNJG8JNJ
    Hi,

    After reading the article I tried installing the Linux version (I'm not a Linux expert).

    I got so far, but the directTDoA.py installer is trying to use PIL (Python Image Library) which is no longer supported and most folks now seem to be using pillow instead.

    I've tried the various workarounds that can be found on the web, but none seem to have the desired outcome.

    It seems to be trying to use PIL to import an image ImageTK, but obviously can't.

    I can't find a way to contact HFlinkz as I'm not a Twitter user (or wish to be).

    Has anyone else had any luck with an install ?

    Regards,

    Martin - G8JNJ
  • G8JNJG8JNJ
    After a few more hours work I've fixed a lot of the missing parts of the various package installs and sorted out version compatibility issues with various modules, so I can now get the map to display and then capture time stamped I/Q files from the preselected KiWI SDR's. So I guess I have been able to make some progress.

    Unfortunately Octave is now reporting errors regarding directory permissions that I'll have to work my way through, but that wilI have to wait as I've had more than enough for today.

    And folks complain about installing Windows software............

    Regards,

    Martin - G8JNJ
  • jksjks
    edited June 2018
    And these sorts of problems are exactly why I'm working on a TDoA "service" using a Kiwi extension front-end. Not always what you want in all cases, but at least you'll have the option.
    WA2ZKD
  • danieldaniel
    I didn't have problems to install using linkz's own howto http://81.93.247.141/~linkz/directTDoA/
  • G8JNJG8JNJ
    Hi Daniel,

    Those are the notes I have been following and slowly working my way through.

    I've got the basic map selection and recording to work, after about 6 hours of effort. But now I've got problems with the Octave scripts that need sorting out.

    My problem is that I'm not a software guy (especially not Linux) and don't really have enough experience of installing stuff onto an Ubuntu platform to be able to sort out all of the various compatibility issues (like calling PIL which is not supported by recent versions of Ubuntu, so I had to load specific versions to resolve compatibility issues) and file permissions etc. (Octave trying to modify files in newly installed directories where the permissions have not been set appropriately) to be able to do it without a very detailed A,B,C type instruction list.

    Most Linux guy's seem to assume that everyone can follow what they are doing, and tend to be fairly light on the exact detail and miss steps as they assume that everyone knows what to do. Then when you throw other components with different command structures and languages like Python and Octave into the mix, it all gets very messy.

    I'm sure I'll work my way through it all eventually, but it's certainly very time consuming and (for me) quite frustrating in comparison to something like a typical Windows application with an installer.

    Regards,

    Martin - G8JNJ
  • linkzlinkz
    Hi Martin

    I'm here & I can help you, give me access to your linux box, it will be solved quickly... ;)

    The directTDoA.py GUI is running fine on Windows, but not the Octave process, everything is ready to work when some Octave.m scripts will be adapted to Windows OS environment.. I'll have a look

    73
    &linkz
This discussion has been closed.