Auto reload dx.json ?

ChrisSmolinskiChrisSmolinski
in General Chat
Would it be possible for the KiwiSDR software to auto reload and process dx.json when it has changed?

I'd like my KiwiSDR to display DX markers for SWBC stations, which of course change continuously throughout the day. Rather than mucking about with the KiwiSDR code, something external that updates dx.json as needed might be easier to get going. When if I get this working I'd be more than happy to release this of course.

Comments

  • CommSigmaCommSigma
    I can see this as a valuable feature, too. It would definitely add value to all the SWBC listeners on my receivers. Could have utility beyond SWBC as well.
  • ChrisSmolinskiChrisSmolinski
    edited September 2020
    While I have a few decades of experience programming, I've never done anything with the KiwiSDR code other than look at it :)

    I'm more than willing to give this a try myself, if someone can point me to even some brief instructions for how to do a build on the KiwiSDR itself.

    Well, I have that part figured out. Now running into other issues... onward I go...
  • jksjks
    edited September 2020
    Chris, if you can figure it out, fine. But you have to remember that it is not a straightforward problem. You have to be very careful otherwise your solution will fail in the case of: a write to the file (externally) at the exact same time as a write to the file as a result of using the Kiwi user interface to update the dx label information. This is a two writer database transaction commit problem, whereas the existing code has a single writer which is comparatively simple.

    At the very least you'd have to add a locking coordination mechanism to both writers. Like a lock file or something. And then the Kiwi side writer has to be run within a Linux child task so the whole Kiwi code doesn't come to a screeching halt while waiting for the lock (the Kiwi code has support routines to assist with this). In fact it's worse than simply waiting for the lock. You have to distinguish that a wait was required due to the locking condition and then re-read the file to pickup whatever potential change was made by the other writer. Then you have to resolve the collision if you both changed the same entry. Finally you can write the file and release your lock. Oh, and you have to guarantee that the other (external) writer is doing the exact same thing. No wonder transactional database packages exist.

    If you don't handle this properly then you've added (another) ticking time bomb to the Kiwi code just waiting to cause mysterious loss of dx.json updates or, worse, corruption of the file preventing any entries from being loaded at all. Can you imagine the support burden this causes me?
  • jksjks
    Now that morning coffee has taken hold: A better way to do this is to have the external writer create a completely different file (say dx.2.json ) containing only the entries it wants to update.

    And then the Kiwi code only does a read of dx.2.json periodically (better: detects a change to file update time). Then does a merge with its own dx.json file, resolves any conflicts, re-writes dx.json and continues on. So the Kiwi side is only a reader of dx.2.json and you've avoided the two-writer problem. Then you only have to guarantee that dx.2.json is in a consistent state when read by the Kiwi side. You can use lock files for that or rely on the json.parse() not generating an error, using sequence numbers, etc. etc.

    This scheme also has the advantage of the writer of dx.2.json not needing any special code. It just does whatever it normally does to make changes to dx.2.json.
  • ChrisSmolinskiChrisSmolinski
    Hi John,

    Yes, the second solution is what I was thinking of long term, directly modifying dx.json was a first step of "can I get something to work, having never touched this code before".
Sign In or Register to comment.