the accesses by [34] are continuous and often change part of the IP address, at the moment I solved with / 8 even if I forbid access to some [normal] users patience.
Update:
When I say continue I mean that in 10 seconds I have 56 blocked access attempts by [34], I don't know if the Kiwi can handle these rhythms.
Yes, the /8 block on 34 and 35 addresses seems prudent right now if you want to eliminate IP changed connections. We need to keep an eye out to see if connections from outside those ranges become a problem. Hopefully, we'll have some information soon whether this is an attack or just an errant process somewhere out there causing trouble.
My un attended kiwis were hit yesterday from 10h utc... Unfortunately I didn't find our until this morning
Needed to reset router as well and also the setting of network switch to establish the general Internet connection. I guess, because I realized the attack late....
All software was crashed - kiwis didn't start up again, etc...Router not functional...etc...
Fortunately I have backup's so the show is over...
Blocking the 34 and 35 ip ranges did prevent the webserver from restarting. Mine has been up for 3 hours now.
I do believe that it is still knocking the people who are monitoring off during the attacks even though the IPs are blocked. I am going to try moving the IP block to my router instead.
Yesterday, 2 out of 3 of my Kiwis that I share with the proxy flew out of the network and the logs showed exactly the same activity with 34.0.xx 35.0.xx Although the domain registration was ok, when checking the port opening I got the message- .proxy.kiwisdr.com : 8073: NO
At the moment, one of the receivers fell out again, but there was no restart and no problems with logging into the local network. These actions started when the Chinese IP from alibaba notoriously fired long sessions around 10.3MHz and got a ban from me about a month back, but I did not check the logs then because I was convinced that the power supply was starting to go out after 3 years of continuous operation and hence these restart one receiver.
Would probably make sense to packet sniff the connection.
Recorded separately to the Kiwi by port mirroring on a router or switch and making the data available to John off-line if a reset event was captured.
I know that is stating the obvious but there are many capable users out there who don't post on the forum, some may have the kit and the time. I did have a dedicated VM and switch set up for sniffing but I block so much anyway it would be unlikely to produce the goods.
for now, the current host name is after the last attack, so Winnie the Pooh came instead and maybe censorship will disable access by itself, let's check 😀
I too have been seeing coordinated, sustained attacks, but on only one of my Kiwis, and the addresses were NOT in the 34/8 or 35/8 ranges. They appeared to be a poorly programmed bot that tried to keep all channels busy. The user claimed to be in Piscataway NJ, but the IP addresses were all over the world -- probably on VPNs.
When I gave up playing whack-a-mole, I tried an experiment. On a hunch that this was a poorly written bot that wasn't being monitored by its creator, I cleared my IP block list, changed the TCP port number and allowed the change to propagate through to rx.kiwisdr.com. So far I've seen only legit usage on the new port. No attack attempts there, but they persisted on the old port number. TCP SYNs (connection requests) reached several thousand per minute.
So I tried another idea: I redirected the old port to the TCP "discard" service. This service accepts an incoming TCP connection....and then says absolutely nothing. It's like answering a phone and setting it down without even saying hello. Because the client uses a much longer timeout to wait for an application-level response than the timeout they use for a rejected or failed TCP connection, this substantially reduced the total unwanted packet rate.
For my next trick, if I get sufficiently bored, I'll replace the "discard" service with the "echo" service that, true to its name, simply repeats back whatever the connecting party says. That should confuse 'em. Or I just might pipe back the contents of /dev/urandom as fast as the attacker can take it and see what that does...
Like the ideas, I did do something similar for a while (the mainly Vultr based ones) at the router, ties up the connections reducing their resources for other users but I went back to just plain drop. My logs seem to show Vultr still active for the occasional port scan but not specifically for OpenWebRX + KiwiSDR ports. Someone must have some funds for this and not worried about admins kicking them off paid VM hosts. I did wonder if this recent Google was done by seeding some URL's somewhere and waiting for Google to scan but then the source would appear to be a spider and I don't think these are.
After two days of peace following the last update, the attack began again.
Fri Dec 24 19:21:20 18:33:05.809 0... L IP BLACKLISTED: 35.202.193.127
Fri Dec 24 19:21:20 18:33:05.862 0... CONN: 127.0.0.1 X-Real-IP 35.202.193.127
Fri Dec 24 19:21:20 18:33:05.862 0... CONN: 127.0.0.1 X-Forwarded-For 35.202.193.127
Fri Dec 24 19:21:20 18:33:05.863 0... L IP BLACKLISTED: 35.202.193.127
Fri Dec 24 19:21:20 18:33:05.868 0... CONN: 127.0.0.1 X-Real-IP 35.202.193.127
Fri Dec 24 19:21:20 18:33:05.868 0... CONN: 127.0.0.1 X-Forwarded-For 35.202.193.127
Fri Dec 24 19:21:20 18:33:05.868 0... L IP BLACKLISTED: 35.202.193.127
*********************************************
Sat Dec 25 14:35:27 13:47:06.798 0... L IP BLACKLISTED: 34.82.71.133
Sat Dec 25 14:35:27 13:47:06.898 0... CONN: 127.0.0.1 X-Real-IP 34.82.71.133
Sat Dec 25 14:35:27 13:47:06.898 0... CONN: 127.0.0.1 X-Forwarded-For 34.82.71.133
Sat Dec 25 14:35:27 13:47:06.898 0... L IP BLACKLISTED: 34.82.71.133
Sat Dec 25 14:35:27 13:47:06.928 0... CONN: 127.0.0.1 X-Real-IP 34.82.71.133
Sat Dec 25 14:35:27 13:47:06.928 0... CONN: 127.0.0.1 X-Forwarded-For 34.82.71.133
Sat Dec 25 14:35:27 13:47:06.928 0... L IP BLACKLISTED: 34.82.71.133
Sat Dec 25 14:35:27 13:47:07.070 0... CONN: 127.0.0.1 X-Real-IP 34.82.71.133
Sat Dec 25 14:35:27 13:47:07.071 0... CONN: 127.0.0.1 X-Forwarded-For 34.82.71.133
********************************************
Sat Dec 25 19:20:47 18:32:27.274 0... L IP BLACKLISTED: 34.127.116.195
Sat Dec 25 19:20:47 18:32:27.304 0... CONN: 127.0.0.1 X-Real-IP 34.127.116.195
Sat Dec 25 19:20:47 18:32:27.304 0... CONN: 127.0.0.1 X-Forwarded-For 34.127.116.195
Sat Dec 25 19:20:47 18:32:27.304 0... L IP BLACKLISTED: 34.127.116.195
Sat Dec 25 19:20:47 18:32:27.333 0... CONN: 127.0.0.1 X-Real-IP 34.127.116.195
Sat Dec 25 19:20:47 18:32:27.334 0... CONN: 127.0.0.1 X-Forwarded-For 34.127.116.195
Sat Dec 25 19:20:47 18:32:27.334 0... L IP BLACKLISTED: 34.127.116.195
Sat Dec 25 19:20:47 18:32:27.482 0... CONN: 127.0.0.1 X-Real-IP 34.127.116.195
Sat Dec 25 19:20:47 18:32:27.482 0... CONN: 127.0.0.1 X-Forwarded-For 34.127.116.195
I keep banging on three receivers, until port 8073 is blocked on proxy.kiwisdr.com and falling out of the network, reported and nothing
Sat Dec 25 13:24:19 14:20:18.770 .... CONN: 127.0.0.1 X-Forwarded-For 34.70.254.107
Sat Dec 25 13:24:19 14:20:18.770 .... L IP BLACKLISTED: 34.70.254.107
Sat Dec 25 13:24:19 14:20:18.775 .... CONN: 127.0.0.1 X-Real-IP 34.70.254.107
Sat Dec 25 13:24:19 14:20:18.776 .... CONN: 127.0.0.1 X-Forwarded-For 34.70.254.107
Sat Dec 25 13:24:19 14:20:18.776 .... L IP BLACKLISTED: 34.70.254.107
Sat Dec 25 13:24:19 14:20:18.826 .... CONN: 127.0.0.1 X-Real-IP 34.70.254.107
Sat Dec 25 13:24:19 14:20:18.826 .... CONN: 127.0.0.1 X-Forwarded-For 34.70.254.107
Sat Dec 25 13:24:19 14:20:18.826 .... L IP BLACKLISTED: 34.70.254.107
Sat Dec 25 13:24:19 14:20:18.854 .... CONN: 127.0.0.1 X-Real-IP 34.70.254.107
Sat Dec 25 13:24:19 14:20:18.854 .... CONN: 127.0.0.1 X-Forwarded-For 34.70.254.107
Sat Dec 25 13:24:19 14:20:18.854 .... L IP BLACKLISTED: 34.70.254.107
Sat Dec 25 13:24:20 14:20:18.881 .... CONN: 127.0.0.1 X-Real-IP 34.70.254.107
Sat Dec 25 13:24:20 14:20:18.882 .... CONN: 127.0.0.1 X-Forwarded-For 34.70.254.107
Sat Dec 25 13:24:20 14:20:18.882 .... L IP BLACKLISTED: 34.70.254.107
Sat Dec 25 13:24:20 14:20:18.887 .... CONN: 127.0.0.1 X-Real-IP 34.70.254.107
Sat Dec 25 13:24:20 14:20:18.887 .... CONN: 127.0.0.1 X-Forwarded-For 34.70.254.107
Sat Dec 25 13:24:20 14:20:18.887 .... L IP BLACKLISTED: 34.70.254.107
Sat Dec 25 13:24:20 14:20:19.023 .... CONN: 127.0.0.1 X-Real-IP 34.70.254.107
Sat Dec 25 13:24:20 14:20:19.023 .... CONN: 127.0.0.1 X-Forwarded-For 34.70.254.107
Sat Dec 25 13:24:20 14:20:19.023 .... L IP BLACKLISTED: 34.70.254.107
Sat Dec 25 13:24:20 14:20:19.028 .... CONN: 127.0.0.1 X-Real-IP 34.70.254.107
L IP BLACKLISTED: 34.70.254.107
CONN: 127.0.0.1 X-Real-IP 34.70.254.107
CONN: 127.0.0.1 X-Forwarded-For 34.70.254.107
should only appear when your Kiwi is using the proxy service.
Why is this? Well, normally when you add an ip address to the blacklist it is added to the Linux iptables service. This causes the data from 34.70.254.107 in this case to get dropped before even reaching the Kiwi server.
But consider what happens when the proxy is in use. All data is flowing through the proxy and the Kiwi only sees data from proxy.kiwisdr.com, a single ip address. This is the very definition of how a proxy works. So all of a sudden iptables is useless because there are never any packets from 34.70.254.107 to filter. But all the bad traffic from 34.70.254.107 is there, encapsulated by the proxy, having a negative impact on your Kiwi.
So what happens is this: The proxy adds special headers to each packet "X-Real-IP" and "X-Forwarded-For" identifying the actual source ip address (34.70.254.107). The kiwi server also maintains a copy of the blacklist it sent to iptables and filters any packets containing X-* against that list. So this process acts as a substitute iptables when the proxy is in use.
Those messages were leftover from debugging. So next release I'm going to remove them and just add a summary table you can dump out and look at so the log doesn't fill up with all those messages.
The IP blacklist is now being applied to the input of the proxy server, which serves several hundred+ Kiwis. The results are kind of interesting. These are accumulated packet counts and byte counts over ~12 hours. The worst offenders seem to be 39.96/12 34/8 and 35/8
I have some blacklist entries on my home router, learnt from attacks to a work server, it is surprising (or maybe not) to see some known for attacking services like FTP over years, here also.
That PDF is great to identify active IP's.
Would be interesting to see the totals over a month.
Proxy network traffic for the past 30 days. Note that in the last 24 hours we're back to the usual ~10 Mb/s rate. And how the trouble (for the proxy server st least) started around Dec 20/21.
I'm working on adding a blacklist auto-update mode so you don't have to manually click the download button to stay current. It sounds simple, but there are complications that make implementation difficult.
Comments
... as you can see from the lines below, which are from a minute ago:
Tue Dec 21 18:26:48 00: 28: 52.493 0 ... L IP BLACKLISTED: 34.138.64.141
Tue Dec 21 18:26:48 00: 28: 52.498 0 ... CONN: 127.0.0.1 X-Real-IP 34.138.64.141
Tue Dec 21 18:26:48 00: 28: 52.498 0 ... CONN: 127.0.0.1 X-Forwarded-For 34.138.64.141
the accesses by [34] are continuous and often change part of the IP address, at the moment I solved with / 8 even if I forbid access to some [normal] users patience.
Update:
When I say continue I mean that in 10 seconds I have 56 blocked access attempts by [34], I don't know if the Kiwi can handle these rhythms.
My bad @jks , i think i push download and after i restarted my beagle and check if blacklisted ip's are still there...
but at least we have a workaround with blacklisted ip's until you will find a solution.
Yes, the /8 block on 34 and 35 addresses seems prudent right now if you want to eliminate IP changed connections. We need to keep an eye out to see if connections from outside those ranges become a problem. Hopefully, we'll have some information soon whether this is an attack or just an errant process somewhere out there causing trouble.
My un attended kiwis were hit yesterday from 10h utc... Unfortunately I didn't find our until this morning
Needed to reset router as well and also the setting of network switch to establish the general Internet connection. I guess, because I realized the attack late....
All software was crashed - kiwis didn't start up again, etc...Router not functional...etc...
Fortunately I have backup's so the show is over...
Ulli, ON5KQ
You would think if a router went down as well, the intent was malicious.
I am running okay. I did catch this in the log:
Dec 21 19:45:35 kiwisdr kiwid: 04:25:43.079 01.. 1 "(no identity)" 3.35.217.73 incomplete connection kicked
Dec 21 19:45:35 kiwisdr rsyslogd-2007: action 'action 17' suspended, next retry is Tue Dec 21 19:47:05 2021 [try http://www.rsyslog.com/e/2007 ]
A lot of "action 17" in the log and lots of kicked incomplete connections.
Can anyone translate?
I rebooted the whole thing, and the added IPs do persist after a boot.
Blocking the 34 and 35 ip ranges did prevent the webserver from restarting. Mine has been up for 3 hours now.
I do believe that it is still knocking the people who are monitoring off during the attacks even though the IPs are blocked. I am going to try moving the IP block to my router instead.
Yesterday, 2 out of 3 of my Kiwis that I share with the proxy flew out of the network and the logs showed exactly the same activity with 34.0.xx 35.0.xx Although the domain registration was ok, when checking the port opening I got the message- .proxy.kiwisdr.com : 8073: NO
At the moment, one of the receivers fell out again, but there was no restart and no problems with logging into the local network. These actions started when the Chinese IP from alibaba notoriously fired long sessions around 10.3MHz and got a ban from me about a month back, but I did not check the logs then because I was convinced that the power supply was starting to go out after 3 years of continuous operation and hence these restart one receiver.
Would probably make sense to packet sniff the connection.
Recorded separately to the Kiwi by port mirroring on a router or switch and making the data available to John off-line if a reset event was captured.
I know that is stating the obvious but there are many capable users out there who don't post on the forum, some may have the kit and the time. I did have a dedicated VM and switch set up for sniffing but I block so much anyway it would be unlikely to produce the goods.
for now, the current host name is after the last attack, so Winnie the Pooh came instead and maybe censorship will disable access by itself, let's check 😀
here is shiff file from my router, extract from .zip and open with Wireshark
Has anyone reported the IP's?
Not saying they will do anything but interesting to see the response.
Perhaps they gave up using Vultr.
Thanks for the quick update, John
73s and merry x-mas!
Ulli, ON5KQ
I too have been seeing coordinated, sustained attacks, but on only one of my Kiwis, and the addresses were NOT in the 34/8 or 35/8 ranges. They appeared to be a poorly programmed bot that tried to keep all channels busy. The user claimed to be in Piscataway NJ, but the IP addresses were all over the world -- probably on VPNs.
When I gave up playing whack-a-mole, I tried an experiment. On a hunch that this was a poorly written bot that wasn't being monitored by its creator, I cleared my IP block list, changed the TCP port number and allowed the change to propagate through to rx.kiwisdr.com. So far I've seen only legit usage on the new port. No attack attempts there, but they persisted on the old port number. TCP SYNs (connection requests) reached several thousand per minute.
So I tried another idea: I redirected the old port to the TCP "discard" service. This service accepts an incoming TCP connection....and then says absolutely nothing. It's like answering a phone and setting it down without even saying hello. Because the client uses a much longer timeout to wait for an application-level response than the timeout they use for a rejected or failed TCP connection, this substantially reduced the total unwanted packet rate.
For my next trick, if I get sufficiently bored, I'll replace the "discard" service with the "echo" service that, true to its name, simply repeats back whatever the connecting party says. That should confuse 'em. Or I just might pipe back the contents of /dev/urandom as fast as the attacker can take it and see what that does...
Like the ideas, I did do something similar for a while (the mainly Vultr based ones) at the router, ties up the connections reducing their resources for other users but I went back to just plain drop. My logs seem to show Vultr still active for the occasional port scan but not specifically for OpenWebRX + KiwiSDR ports. Someone must have some funds for this and not worried about admins kicking them off paid VM hosts. I did wonder if this recent Google was done by seeding some URL's somewhere and waiting for Google to scan but then the source would appear to be a spider and I don't think these are.
If anyone want to report to google, you can do it here: https://support.google.com/code/contact/cloud_platform_report?hl=en
I made the report.
After two days of peace following the last update, the attack began again.
*********************************************
********************************************
I keep banging on three receivers, until port 8073 is blocked on proxy.kiwisdr.com and falling out of the network, reported and nothing
Note that the log messages:
should only appear when your Kiwi is using the proxy service.
Why is this? Well, normally when you add an ip address to the blacklist it is added to the Linux iptables service. This causes the data from
34.70.254.107in this case to get dropped before even reaching the Kiwi server.But consider what happens when the proxy is in use. All data is flowing through the proxy and the Kiwi only sees data from proxy.kiwisdr.com, a single ip address. This is the very definition of how a proxy works. So all of a sudden iptables is useless because there are never any packets from
34.70.254.107to filter. But all the bad traffic from34.70.254.107is there, encapsulated by the proxy, having a negative impact on your Kiwi.So what happens is this: The proxy adds special headers to each packet "X-Real-IP" and "X-Forwarded-For" identifying the actual source ip address (
34.70.254.107). The kiwi server also maintains a copy of the blacklist it sent to iptables and filters any packets containing X-* against that list. So this process acts as a substitute iptables when the proxy is in use.Those messages were leftover from debugging. So next release I'm going to remove them and just add a summary table you can dump out and look at so the log doesn't fill up with all those messages.
The IP blacklist is now being applied to the input of the proxy server, which serves several hundred+ Kiwis. The results are kind of interesting. These are accumulated packet counts and byte counts over ~12 hours. The worst offenders seem to be 39.96/12 34/8 and 35/8
I have some blacklist entries on my home router, learnt from attacks to a work server, it is surprising (or maybe not) to see some known for attacking services like FTP over years, here also.
That PDF is great to identify active IP's.
Would be interesting to see the totals over a month.
Proxy network traffic for the past 30 days. Note that in the last 24 hours we're back to the usual ~10 Mb/s rate. And how the trouble (for the proxy server st least) started around Dec 20/21.
Some new last evening /today. Alicloud Hong Kong
47.57.141.16
47.57.13.80
47.57.8.3
47.57.244.18
47.57.70.123
Haven't looked up the range yet guessing 47.57.0.0/16
Will save other syslogs look for patterns.
It's
47.56.0.0/15which I've now added.I'm working on adding a blacklist auto-update mode so you don't have to manually click the download button to stay current. It sounds simple, but there are complications that make implementation difficult.
Thanks for the /15 clarification.
I can imagine the auto download would take some safety checking.
Hi John,
In your auto-update please ensure that a malicious user can't blacklist 192.168.0.0/16, 10.0.0.0/8 or 172.16.0.0/12.
Thanks,
Rob