wsprnet log jam
Those of you that run wspr likely have seen issues as of late, wsprnet "slowing down", etc. wspr spots have increased by 250% in the last year. Spots get uploaded from the various reporters using wsjtx-x, wsprdaemon, other computer based applications, Red Pitaya. and the kiwiSDR extension.
Looking at upload logs for December 2020, I see there were more than 72 million spots uploaded, about 30 per second, more at peak times. Ther are 2 methods to do that: 1) a batch method, which minimizes the server loading 2) a low quantity if not single spot upload, which is server intensive and causes the upload congestion. I believe only wsprdaemon uses the more efficient batch method. Changes to the rest requires authors to rewrite code, they would have to buy in. Another look at the spots for Dec suggests than about 20% of the 72M came from kiwiSDR extension uploads from 415 kiwiSDR. The kiwiSDR changed the wspr world!
If the code in the kiwi extension was changed to use the batch method, thr server loading would be greatly reduced. Rob, AI6VN, who wrote the wsprdaemon code know how the batching works, but John might have to tweak the extension code.
Happy New Year!
-Jim
Comments
It was my experience that the single spot upload interface to wsprnet.org takes 200-300 msecs per spot and of course generates lots of TCP setup/terminate traffic. In contrast, WD can upload 100- spots to wsprnet.org in 1000 msecs using the batch mode.
It is my understanding that the wsprnet.org the multi cpu servers are running at 100% CPU capacity, so I fear we are in danger of soon forcing those servers to drop uploaded spots.