Second build sold out. Message will appear here when store is ready for third build ordering.

Local Access Issue [the bot / IP blacklist thread] [fixed in v1.660]

245

Comments

  • edited March 2022

    @Powernumpty Makes sense, but I honestly don't even go in there unless I'm having an issue or checking for updates. I rarely have the Admin tab open long enough for it to time out which is why it's never been an issue. I'll just remember to refresh it next time I see it isn't functioning as expected.

  • jksjks
    edited March 2022

    @KU4BY So as a test of the changes in v1.499 I'd be very interested to know, if you remove the block, if the intruder comes back and appears on your Kiwi's connection listings. And most importantly if the correct ip address shows up on the admin page status tab listing.

  • @jks removed the block. Re-enabled allowing more than one connection per IP address, and kicked off the update.

  • edited March 2022

    I'm not sure if this is related or not but after I ran the update earlier, I've had a ton of "kiwirecorder.py" bots show up. They come in close to each other, stay for just over a minute, then leave together. I've not noticed this many before and I've never seen them show up on 0.00 kHz.

    Could this have been what was happening before only they weren't showing up in the Users area or Status tabs?

  • jksjks
    edited March 2022

    0 kHz / z0 are typically from waterfall-only connections. So these might be from SNR measurement sites. Although they shouldn't appear often, and not a bunch at a time. Yes, one of the consequences of v1.499 is that absolutely everything appears now and is logged.

    We should probably ask the legit SNR measurement sites to change the kiwirecorder user connection name from the default "kiwirecorder.py" to something more specific. For example, any kiwirecorder connection on behalf of the TDoA extension will appear as "TDoA_service".

  • jksjks
    edited March 2022

    That first ip is from the Vultr cloud and the second from the constant.com cloud. So their CIDRs should probably go on the blacklist (I've now done that). Although that begs the question of whether the legit SNR measurement sites are running on a cloud/VPS provider or not.

    If you Google around a bit it turns out there are other projects that use data from the Kiwi public sites for various research purposes, e.g. https://ieeexplore.ieee.org/document/9618011

  • Is there a way to pull a list of all of the connections I received over night? I do have SSH setup on the kiwi if necessary.

    Thanks!

  • edited March 2022

    OK, so I might have figured it out. Here is a list of all of the kiwirecorder.py connections from user.log last night up until about 15 minutes ago. They do appear to have started showing up last night after updating to 1.499. They've probably always been in there but haven't been getting logged. Too many to paste in here so I'm attaching as a text file.

    They come in for a minute and a half each time. There are a lot of duplicates in the list but my question is, what is their purpose? A DOS type thing?

    I'm about to sort through this list and make one hella blacklist! 😂

    It turns out there were only 7 addresses repeatedly connecting.





  • Nice lot of Vultr's you have there. I'm sure there could be a legitimate reason for them constantly connecting, I just haven't heard about it yet. As I was constantly port scanned by them I decided to call that illegal activity and ban the whole ranges.

    I do wonder if they are internet path mapping using the waterfall (or GPS) data as timestamps? Seems strange if they are just looking to scrape frequency data to do it from so many geographically separate locations. I assume you are not close to military bases, no need to answer that, just something I've considered.

  • I am in an area with lots of military bases but most of them are at least an hour away.

    I think I might just break out my old trusty "Linux for Dummies" book and setup a cron job to create a daily report of all of these generic kiwirecorder.py connections to add to the blacklist until folks do as @jks suggested and give them more meaningful names.

    On a side note, I haven't seen the 182.150.24.15 address pop back in since I unblocked it and no more "All Channels are in use." messages either. I'm going to leave it unblocked in hopes that they find me again...😂

  • @KU4BY You could be a nice distance for NVIS military..

    As the other thread on this forum mentions there are many legitimate reasons to use the Kiwi networks, kiwirecorder etc. it only falls apart if the client is not operating in a visible and well controlled manner so the radio owner has no idea what, or who, is doing recording data.

    On the logging I have used remote syslog for a while, simply to look at what happened without needing the Kiwi to be contactable. Obviously you can still extract logging of connections without risking filling the BeagleBone file system with logs or missing events at boot.

  • jksjks
    edited March 2022

    So I've just discovered that the log messages for WF-only connections, like these, is not showing the true center frequency and zoom settings requested by kiwirecorder. It's always displaying cf=0 z=0. The true values in use at the time of the "leaving" log message would give us a better clue what these connections are up to. So I need to fix this. And while I'm at it show a different message distinguishing the connection as WF-only.

  • Well I can report that since I blocked those 7 IP addresses above about 8 hours ago, I have had 0 kiwirecorder.py connections and no issues with the receiver.

  • ... I'm sorry if it's a repetitive question, but I can't always follow the forum information, which is sometimes too technical for me, but these kiwirecorder.py users should be blocked?

    I state that I have been back online for just 2 hours

    Thu Mar 17 07:54:12 00:53:54.242 01..  1     PWD kiwi W/F ALLOWED: no config pwd set, allow any (216.238.73.79)
    Thu Mar 17 07:54:12 00:53:54.650 01..  1   L    58.59 kHz  WF z8  "kiwirecorder.py" 216.238.73.79 (ARRIVED)
    Thu Mar 17 07:54:18 00:54:00.569 01..  1       175.78 kHz  WF z8  "kiwirecorder.py" 216.238.73.79 0:00:06
    Thu Mar 17 07:54:28 00:54:10.568 01..  1       292.97 kHz  WF z8  "kiwirecorder.py" 216.238.73.79 0:00:16
    Thu Mar 17 07:54:38 00:54:20.572 01..  1       703.13 kHz  WF z6  "kiwirecorder.py" 216.238.73.79 Mexico City, Mexico 0:00:26
    Thu Mar 17 07:54:48 00:54:30.568 01..  1      2343.75 kHz  WF z5  "kiwirecorder.py" 216.238.73.79 Mexico City, Mexico 0:00:36
    Thu Mar 17 07:54:58 00:54:40.568 01..  1      3281.25 kHz  WF z5  "kiwirecorder.py" 216.238.73.79 Mexico City, Mexico 0:00:46
    Thu Mar 17 07:55:08 00:54:50.568 01..  1      9375.00 kHz  WF z3  "kiwirecorder.py" 216.238.73.79 Mexico City, Mexico 0:00:56
    Thu Mar 17 07:55:18 00:55:00.568 01..  1     16875.00 kHz  WF z3  "kiwirecorder.py" 216.238.73.79 Mexico City, Mexico 0:01:06
    Thu Mar 17 07:55:28 00:55:10.568 01..  1     20625.00 kHz  WF z3  "kiwirecorder.py" 216.238.73.79 Mexico City, Mexico 0:01:16
    Thu Mar 17 07:55:38 00:55:20.570 01..  1     28125.00 kHz  WF z3  "kiwirecorder.py" 216.238.73.79 Mexico City, Mexico 0:01:26
    Thu Mar 17 07:55:45 00:55:27.298 0...  1   L 28125.00 kHz  WF z3  "kiwirecorder.py" 216.238.73.79 Mexico City, Mexico (LEAVING after 0:01:33)
    Thu Mar 17 08:01:12 01:00:54.745 01..  1     PWD kiwi W/F ALLOWED: config pwd set, but is_local and auto-login set
    Thu Mar 17 08:17:24 01:17:06.082 01..  1     PWD kiwi W/F ALLOWED: no config pwd set, allow any (144.202.84.81)
    Thu Mar 17 08:17:24 01:17:06.393 01..  1   L    58.59 kHz  WF z8  "kiwirecorder.py" 144.202.84.81 (ARRIVED)
    Thu Mar 17 08:17:28 01:17:10.570 01..  1        58.59 kHz  WF z8  "kiwirecorder.py" 144.202.84.81 0:00:04
    Thu Mar 17 08:17:38 01:17:20.568 01..  1       292.97 kHz  WF z8  "kiwirecorder.py" 144.202.84.81 0:00:14
    Thu Mar 17 08:17:40 01:17:22.576 01..  1   L GEOLOC: 144.202.84.81 sent no geoloc info, we got "Seattle, Washington, USA" from geo host #0
    Thu Mar 17 08:17:40 01:17:22.580 01..        task geoloc_task:P2:T003((1000.000 msec) TaskSleep) exited by returning
    Thu Mar 17 08:17:48 01:17:30.568 01..  1       703.13 kHz  WF z6  "kiwirecorder.py" 144.202.84.81 Seattle, Washington, USA 0:00:24
    Thu Mar 17 08:17:58 01:17:40.568 01..  1      1406.25 kHz  WF z5  "kiwirecorder.py" 144.202.84.81 Seattle, Washington, USA 0:00:34
    Thu Mar 17 08:18:08 01:17:50.571 01..  1      3281.25 kHz  WF z5  "kiwirecorder.py" 144.202.84.81 Seattle, Washington, USA 0:00:44
    Thu Mar 17 08:18:18 01:18:00.568 01..  1      9375.00 kHz  WF z3  "kiwirecorder.py" 144.202.84.81 Seattle, Washington, USA 0:00:54
    Thu Mar 17 08:18:28 01:18:10.570 01..  1     13125.00 kHz  WF z3  "kiwirecorder.py" 144.202.84.81 Seattle, Washington, USA 0:01:04
    Thu Mar 17 08:18:38 01:18:20.569 01..  1     20625.00 kHz  WF z3  "kiwirecorder.py" 144.202.84.81 Seattle, Washington, USA 0:01:14
    Thu Mar 17 08:18:48 01:18:30.568 01..  1     28125.00 kHz  WF z3  "kiwirecorder.py" 144.202.84.81 Seattle, Washington, USA 0:01:24
    Thu Mar 17 08:18:56 01:18:38.152 0...  1   L 28125.00 kHz  WF z3  "kiwirecorder.py" 144.202.84.81 Seattle, Washington, USA (LEAVING after 0:01:32)
    Thu Mar 17 08:37:44 01:37:25.862 01..  1     PWD kiwi SND ALLOWED: no config pwd set, allow any (216.73.161.167)
    Thu Mar 17 08:38:17 01:37:59.472 01..  1     PWD kiwi W/F ALLOWED: no config pwd set, allow any (149.28.166.127)
    Thu Mar 17 08:38:18 01:38:00.088 01..  1   L    58.59 kHz  WF z8  "kiwirecorder.py" 149.28.166.127 (ARRIVED)
    Thu Mar 17 08:38:18 01:38:00.568 01..  1        58.59 kHz  WF z8  "kiwirecorder.py" 149.28.166.127 0:00:01
    Thu Mar 17 08:38:28 01:38:10.568 01..  1       175.78 kHz  WF z8  "kiwirecorder.py" 149.28.166.127 0:00:11
    Thu Mar 17 08:38:38 01:38:20.568 01..  1       410.16 kHz  WF z8  "kiwirecorder.py" 149.28.166.127 Alexandria, Australia 0:00:21
    Thu Mar 17 08:38:48 01:38:30.568 01..  1      1406.25 kHz  WF z5  "kiwirecorder.py" 149.28.166.127 Alexandria, Australia 0:00:31
    Thu Mar 17 08:38:58 01:38:40.568 01..  1      2343.75 kHz  WF z5  "kiwirecorder.py" 149.28.166.127 Alexandria, Australia 0:00:41
    Thu Mar 17 08:39:08 01:38:50.569 01..  1      5625.00 kHz  WF z3  "kiwirecorder.py" 149.28.166.127 Alexandria, Australia 0:00:51
    Thu Mar 17 08:39:18 01:39:00.568 01..  1     13125.00 kHz  WF z3  "kiwirecorder.py" 149.28.166.127 Alexandria, Australia 0:01:01
    Thu Mar 17 08:39:28 01:39:10.568 01..  1     16875.00 kHz  WF z3  "kiwirecorder.py" 149.28.166.127 Alexandria, Australia 0:01:11
    Thu Mar 17 08:39:38 01:39:20.571 01..  1     24375.00 kHz  WF z3  "kiwirecorder.py" 149.28.166.127 Alexandria, Australia 0:01:21
    Thu Mar 17 08:39:48 01:39:30.568 01..  1     28125.00 kHz  WF z3  "kiwirecorder.py" 149.28.166.127 Alexandria, Australia 0:01:31
    Thu Mar 17 08:39:49 01:39:31.534 0...  1   L 28125.00 kHz  WF z3  "kiwirecorder.py" 149.28.166.127 Alexandria, Australia (LEAVING after 0:01:32)
    


  • edited March 2022

    I think it is worth blocking Vultr IP addresses until someone provides a valid reason otherwise. My router is constantly port scanned (not legal) from Vultr IP addresses, for me if someone is persistently attacking my network I need no further prompting to block access by default.

    The fact there are other source addresses from the same company but in other geographical locations makes me think this is a well funded (profitable) operation, as they are not paying you, you RF data must be the product being sold?

    You can check IP addresses online at a great number of sites, one I use is abuseipdb.com, so taking an example from your log https://www.abuseipdb.com/check/149.28.166.127 - Vultr

  • ... yes I had checked on another IP Reverse site and anyway they are all 3 IP "Vultr".

    I don't know whether to report to the ABUSE email [OrgAbuseEmail: abuse@constant.com], it would have some effect mainly because I am on proxy reverse and I don't think I can do much about port control.

    The doubt arises, putting the events in line and taking into account that the problems have begun or rather we have noticed them since the beginning of the year, which could all be traced back to the current conflict for reasons unknown to me, because previously a part of some sporadic sly, I do not remember problems of abuse.

    I would add that I may also have blundered about the reasons, but I don't know to what extent it is.

    Just so as not to dismiss the subject, does anyone have an idea of the meaning of these lines?

    Thu Mar 17 09:56:26 02:56:08.608 0...        rx-1 SET extint_setup_c2s: ext_client_init=0(is_locked)
    Thu Mar 17 09:56:55 02:56:36.945 0...        rx0 SET ext_is_locked_status: ext_client_init=0(is_locked)
    Thu Mar 17 09:56:57 02:56:39.204 0...        rx0 SET ext_is_locked_status: ext_client_init=0(is_locked)
    Thu Mar 17 09:57:48 02:57:30.064 0...        rx0 SET ext_is_locked_status: ext_client_init=0(is_locked)
    Thu Mar 17 09:57:59 02:57:40.845 0...        rx0 SET ext_is_locked_status: ext_client_init=0(is_locked)
    


  • It's good that you are reporting the new information in your logs because it will help us develop new entries for the kiwisdr.com downloadable blacklist. This will benefit everyone.

    My guess is that it's a waste of time reporting directly to the cloud providers.

  • I got a new hit today and I'm loving the new log settings. I haven't re-enabled the setting restricting to one connection per IP yet.

    I am curious though, is there a way to disable WF only connections? Or even just block kiwirecorder.py connections? They only used 2 channels this time but if they start hitting all 4 again nobody will be able to login while they're scanning the band.


    Thu Mar 17 17:41:35 03:46:32.087 012.  1      1406.25 kHz  WF z5  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:00:34
    Thu Mar 17 17:41:35 03:46:32.087 012.   2      703.13 kHz  WF z6  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:00:28
    Thu Mar 17 17:41:45 03:46:42.078 012.  1      3281.25 kHz  WF z5  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:00:44
    Thu Mar 17 17:41:45 03:46:42.078 012.   2     2343.75 kHz  WF z5  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:00:38
    Thu Mar 17 17:41:55 03:46:52.078 012.  1      5625.00 kHz  WF z3  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:00:54
    Thu Mar 17 17:41:55 03:46:52.078 012.   2     3281.25 kHz  WF z5  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:00:48
    Thu Mar 17 17:42:05 03:47:02.079 012.  1     13125.00 kHz  WF z3  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:01:04
    Thu Mar 17 17:42:05 03:47:02.079 012.   2     9375.00 kHz  WF z3  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:00:58
    Thu Mar 17 17:42:15 03:47:12.078 012.  1     20625.00 kHz  WF z3  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:01:14
    Thu Mar 17 17:42:15 03:47:12.078 012.   2    16875.00 kHz  WF z3  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:01:08
    Thu Mar 17 17:42:25 03:47:22.084 012.  1     24375.00 kHz  WF z3  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:01:24
    Thu Mar 17 17:42:25 03:47:22.084 012.   2    20625.00 kHz  WF z3  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:01:18
    Thu Mar 17 17:42:34 03:47:30.797 0.2.  1   L 28125.00 kHz  WF z3  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA (LEAVING after 0:01:32)
    Thu Mar 17 17:42:35 03:47:32.078 0.2.   2    28125.00 kHz  WF z3  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA 0:01:28
    Thu Mar 17 17:42:40 03:47:36.427 0...   2  L 28125.00 kHz  WF z3  "kiwirecorder.py" 149.28.38.97 Piscataway, New Jersey, USA (LEAVING after 0:01:32)
    


  • @fabrys Just so as not to dismiss the subject, does anyone have an idea of the meaning of these lines?

    That's debugging information leftover from excluding certain extension when DRM is running. I forget the details (it's been a long time). I should probably remove it.

  • I am curious though, is there a way to disable WF only connections? Or even just block kiwirecorder.py connections?

    On the admin page, control tab, there is the setting: Number of simultaneous channels available for connection by non-Kiwi apps which you could set to "none" or "1". Remember that setting to "none" will not allow your Kiwi to be used for TDoA.

  • @fabrys No point reporting, it's a commercial issue. The cloud providers make money from selling people their infrastructure, if money 'lost due to customers activity' was greater than 'money made' they would review. The issue has been going on for years. It may be that data is being fed to some AI that has since discovered some new ways of extracting content or monetising it so the number of virtual hosts and clients has increased.

  • Re: Number of simultaneous channels available for connection by non-Kiwi apps Looks like that code needs some work. Let me see if I can fix it today.

  • jksjks
    edited March 2022

    So now we have some samples of WF center frequencies and zoom values from the bots. And they appear to change during the lifetime of the connections. Anyone want to do some analysis and guess what they might be up to?

    I wonder if their purpose is semi-legit (research?) and they just don't realize they have a bug where their script makes multiple, simultaneous connections to the same Kiwi?

  • The downloadable IP blacklist has been updated based on the reports above. Next time you've got the admin page opened go to the network tab and see if it says a new download is available.

  • ... I do not understand if these IPs must be reported [VULTR] to update the list,

    however I have included it in my local black list.

    Furthermore, what exactly do lines 3 and 4 indicate?

    Mon Mar 21 20:19:38 19:31:20.966 0... 0      PWD kiwi W/F ALLOWED: no config pwd set, allow any (45.32.124.96)
    Mon Mar 21 20:19:39 19:31:21.581 0... 0    L    58.59 kHz  WF z8  "kiwirecorder.py" 45.32.124.96 (ARRIVED)
    Mon Mar 21 20:19:48 19:31:30.996 0... 0      API: decided connection is non-Kiwi app (0)
    Mon Mar 21 20:19:48 19:31:30.996 0... 0      API: ext_api_users=1 >? ext_api_ch=1 F(OKAY)
    Mon Mar 21 20:19:49 19:31:31.996 0... 0        175.78 kHz  WF z8  "kiwirecorder.py" 45.32.124.96 0:00:11
    Mon Mar 21 20:19:50 19:31:33.004 0... 0    L GEOLOC: 45.32.124.96 sent no geoloc info, we got "Queenstown Estate, Singapore" from geo host #2
    Mon Mar 21 20:19:50 19:31:33.007 0...        task geoloc_task:P2:T002((1000.000 msec) TaskSleep) exited by returning
    Mon Mar 21 20:19:59 19:31:41.996 0... 0        410.16 kHz  WF z8  "kiwirecorder.py" 45.32.124.96 Queenstown Estate, Singapore 0:00:21
    Mon Mar 21 20:20:09 19:31:51.996 0... 0       1406.25 kHz  WF z5  "kiwirecorder.py" 45.32.124.96 Queenstown Estate, Singapore 0:00:31
    Mon Mar 21 20:20:19 19:32:01.996 0... 0       2343.75 kHz  WF z5  "kiwirecorder.py" 45.32.124.96 Queenstown Estate, Singapore 0:00:41
    Mon Mar 21 20:20:29 19:32:11.996 0... 0       5625.00 kHz  WF z3  "kiwirecorder.py" 45.32.124.96 Queenstown Estate, Singapore 0:00:51
    Mon Mar 21 20:20:39 19:32:21.996 0... 0      13125.00 kHz  WF z3  "kiwirecorder.py" 45.32.124.96 Queenstown Estate, Singapore 0:01:01
    Mon Mar 21 20:20:49 19:32:31.999 0... 0      16875.00 kHz  WF z3  "kiwirecorder.py" 45.32.124.96 Queenstown Estate, Singapore 0:01:11
    Mon Mar 21 20:20:59 19:32:41.997 0... 0      24375.00 kHz  WF z3  "kiwirecorder.py" 45.32.124.96 Queenstown Estate, Singapore 0:01:21
    Mon Mar 21 20:21:09 19:32:51.996 0... 0      28125.00 kHz  WF z3  "kiwirecorder.py" 45.32.124.96 Queenstown Estate, Singapore 0:01:31
    Mon Mar 21 20:21:09 19:32:51.999 .... 0    L 28125.00 kHz  WF z3  "kiwirecorder.py" 45.32.124.96 Queenstown Estate, Singapore (LEAVING after 0:01:31)
    


  • jksjks
    edited March 2022

    There is no reporting to Vultr or any other cloud service. Waste of time. The global IP blacklist, on the admin page, network tab, "Downloaded blacklist (read-only)", is kept on kiwisdr.com and maintained by me. That's why I ask for suggestions to add to the list.

    The messages on lines 3 and 4 are related to the new detection scheme of WF-only connections. So if someone thinks the scheme is broken I will have some idea what is going wrong.

    You guys have to STOP being so concerned about every little message you see in the log that is unfamiliar to you. It's a distraction for me to be explaining them to you when I am so busy doing other more important work. I will not be answering such questions anymore.

  • The report was referred to you in the forum and not to the provider, which I understand is a waste of time.

  • I'm much more of a "set it and forget it" kind of guy. I don't bother with the logs unless something isn't right like all 4 of my channels are taken up by ghost connections...😂

    Since this happened though, I have been checking them quite often and did also see the IP from Singapore pop in the other day, I blocked it and moved on. Nothing new since. It's nice to see that this isn't as big of an issue as I originally thought and it is at least manageable now that we can see it in the logs.

    Thanks for the help in getting this straightened out @jks!

  • a single Sinagpore connection? There are legit users there.

  • @WA2ZKD This thread has nothing to do with blocking legit users. It's about blocking bots that are making waterfall only connections every hour taking up as many channels as they want while denying service to legit users.

Sign In or Register to comment.