v1.462: update problems
This discussion was created from comments split from: v1.462: MANUAL INTERVENTION REQUIRED to receive future updates.
It looks like you're new here. If you want to get involved, click one of these buttons!
Comments
John,
Similiar than others, I find the update process (update_fix in console) doesn't work. My first attempt with just one kiwiSDR for initial try, results in
I then made a new SD-Flashcard and start from there with the recent file from dropbox
I have done it 4 times now.... until after about 90min of manual update from the dropbox vers. 1.39 I am back to
root@kiwisdr:~/Beagle_SDR_GPS#WAfter reboot, the browser cannot connect the kiwi, when I try to connect from inside the network - all I get is:
"This site can't be reached - 192.168.10.151 refused to connect"
The kiwi is for sure on this internal IP-adress - I can ping it...
Then I tried log-in via SSH (which always worked fine before, until the version 61
SSH log-in from the raspberry pi to the kiwi attempt results in:
-----------------------------------
pi@ON5KQ-Pi4:~ $ ssh pi@192.168.10.151
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:wiE82gs8F++OZ8fFdLEgjVm8+IemkCg4reDBkK80+FA.
Please contact your system administrator.
Add correct host key in /home/pi/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /home/pi/.ssh/known_hosts:4
remove with:
ssh-keygen -f "/home/pi/.ssh/known_hosts" -R "192.168.10.151"
ECDSA host key for 192.168.10.151 has changed and you have requested strict checking.
Host key verification failed.
pi@ON5KQ-Pi4:~ $
----------------------
What does this all mean ?
How can I reach the kiwi / beagle directly so I can take any action...
USB ? serial cable ?
It is difficult to keep loving the many kiwis, I run for the community...
Is there a quick and effective cure ?
Ulli, ON5KQ
My Point is, if I input, what the message says and try to ssh again - no way to log in. Similar message - just the 'offender' randomly changes...
Obviously the ssh password is not 'understood'....
I will start rebuilding the kiwi OS once more (5th try)...
But why I never can see the admin page after waiting 90min of OS-rebuild with update_fix.... ?!?
@Ulli
Offending ECDSA key in /home/pi/.ssh/known_hosts:4
remove with:
ssh-keygen -f "/home/pi/.ssh/known_hosts" -R "192.168.10.151"
Don't panic. This is part of the ssh security concept and means that the kiwi's ssh host key has changed so that raspi's ssh client can't consider your kiwi an already "known" host anymore. In order to prevent being cheated by a dangerous wrong host which only pretends to be what you expect, the connection is refused.
To "reset" the situation and make connections possible again, the stored key must be removed from the raspi's list of known hosts. As the messages tells, this is done with the following command:
ssh-keygen -f "/home/pi/.ssh/known_hosts" -R "192.168.10.151"After that, you can connect again. But as the host is still unknown from the raspi's point of view and this is the "first" attempt, you will be explicitly asked to allow it. If you do, the new key will be added to the list of known hosts, which means that this host (your kiwi) is subsequently considered trustworthy by your raspi, so that future connection attempts will work without the need to ask for permission again.
"...so that future connection attempts will work without the need to ask for permission again."
My point is, that they don't work after all what you described (and what is already mentioned in the SSH alert)
Here is what I did before:
So as in the original post of John (first post in this thread:
"....After the restart delay you should be running the updated version and automatic updates should work again. Verify by clicking the "check now" button on the admin page, "update" tab and see that you get a message saying "running the most current version"
This is what doesn't work, as there is no admin page and no kiwi operation at all. Only ping response....
Something is missing here...
Ulli, ON5KQ
John, good morning!
Don't worry - it is more valuable for me to understand what is the reason for the problem - so the effort I have to put into that MYSELF, is not for nothing.
I am sure you would be able to help me within minutes in reality - however, other than a working product, I would have really learned nothing!
I am convinced, that the threads of the battleground called "Internet" will further grow and today unsocial people will just get more criminal... so, as a consequence, anybody using IoT devices or computers or any other smart home stuff MUST fully understand what he is doing with this devices.
As I now find strange behaviour of the kiwis (a kind of IoT device), I must find out the reason and learn from it.
As soon as I find the reason for the strange behaviour, I will post it here so others can learn from it as well, if they ever find some similar problem.
Fortunately the kiwi's are hobby stuff and there is not real urgency involved... but the unfriendly and criminal, completely uncontrolled (some people call it free) Internet forces drastic security measures to survive - it is just the reality. We cannot dream of an unrealistic world, so sorry!
just doing a new 1.390 -->1.464 upgrade via console-update_fix command. It is my intention to copy and paste the total console response to a file for further documentation.
Ulli, ON5KQ
Ulli, Agreed. I've learnt more from Kiwi that most other devices, it is interesting enough when working to want to keep it that way.
I messed up a Kiwi recently and did a complete Original SD card !.2 restore then upgrade to recent without an issue, might be good to try that if you have one, that would test if the issue comes from the Dropbox image.
I also had a self-made backup ~1.432 but I hit issues with that, not least of which was forgetting the root password, I considered resetting it via grub then just went 1.2 and didn't look back.
I'd also be tempted to close/reopen browser and even reset in-line network switches when wiping to an earlier image, just takes those out of the equation.
Stu
Would the second part of that not be an error? the update will probably be ongoing as it is already at 1.390 which does not have an upgrade problem so is probably doing the upgrade anyway, perhaps you can't ssh simply as it is too busy?
"...perhaps you can't ssh simply as it is too busy?"
hmm, seems unlikely to me, as I so far tried SSH only after upgrade script has finished and after reboot. That was the moment, when after reboot the kiwi-webpages disappeared and the browser says... "not reachable'...
Need to check other network issues as well...
Some time ago I found my level 2 HP Procurve-switch blocks NTP by default - so needed to take 'special measures to get NTP passed... - I realised that this is a bug, HP has never solved in their product.... so I would be not surprised to find more stuff like this...
Nowadays it is easier to just introduce a new product, rather than support the old one...
Ulli
I attach the text file, which contains the complete upgrade response from the Kiwi-SDR console tab when doing 'update_fix' from the latest dropbox flashed SD card file (kiwi OS Vers 1.390)
It is late evening here in Belgium - tomorrow I am busy and don't have time to look into that further... it will be weekend, when I can examine the problem further...
Ulli, ON5KQ
I just finished re-flashing from sd cards. Both the v1.2 factory image and the current v1.390 recovery image mentioned here: http://kiwisdr.com/quickstart/index.html#id-dload
As expected, they both automatically updated to the latest version (v1.464) without problems. So in principle things are not fundamentally broken.
@Ulli Did you notice in your log the following two errors?
/bin/sh: 1: cannot open ../build/obj_O3/sstv.d.tmp: No such file
/bin/sh: 1: cannot open ../build/obj_O3/sstv_common.d.tmp: No such file
This is not good. Probably filesystem damage of some sort. Or perhaps a full filesystem, although I would have expected a different error message in that case. It might even be a damaged eMMC chip since multiple builds have failed across multiple re-flashes.
Also, it finished the "make" part of the update_fix script, but not the "make install". If you use the "dmesg" command I wonder if you would see lots of eMMC access errors.
John, I have seen that, but didn't focus on that issue (yet)...
I can try to take a different SD-card to flash... (weekend)
John, whole day busy with the kiwis...
I found the following:
There seem to be a connection problem with the DUC in the kiwi-Software for the noip.com Dynamic DNS Service! Remember that I tried to update directly from 1.461 to 1.464 (which includes the patch for UTF-8
As I have very strong noip.com password with many characters incl special characters, this password seems not to be translated correctly from the DUC
It is just a guess, but at least when I input my account data for the DUC and then try it, there is no "successful' response from noip.com at least.
Instead, if I try to reload the kiwi web-admin page I loose connection to the kiwi completely and need to re-install everything (from flashcard) to reset it and start from a blank Kiwi-OS
The same happens (without any noip.com account data filled in), but with my very strong admin passwords of many characters incl. special characters.
I do NOT want to relax the required passwords.
So with such a admin-password set and reloading the Kiwi's web-admin page, I immediately loose connection.
Could it be the patch in Vers 1.464, which doesn't translate the special characters correctly ?!
Those two fundamental issues are the remaining problems as I can oversee it at the moment.
There was no problem at all with Vers 1.461
regards,
Ulli, ON5KQ
And because I have strong passwords on all my kiwis - updating from Vers 1.461 to Vers 1.464 breaks the connection of all kiwis after reboot immediately.
It has nothing to do whether the update has been done in the console with 'update_fix' nor automatically.
If my guess it correct we would need a version 1.465 unfortunately, I am sorry
this would also explain, why the upgrade process works for many users, but not for me, as most people seem to use relatively simple passwords...
Ulli, ON5KQ
John,
can't you offer Vers 1.461 somewhere for download ?
For the moment new features are less important, as with 1.464 any reboot keeps breaking all connections to my kiwi's anyway.
But Vers 1.461 worked....
As a diagnostic step to verify your assumption, you could temporarily remove any special characters from your passwords and increase their length instead so that security is not reduced.
If this confirms your suspicion, you could keep it as an interim solution until a fix is available.
(At least that's the way I would do.)
73, Manfred
I'm sorry - back to where I have been yesterday:
1;) upgrade from clean kiwi-OS from flashcard 1.390 (dropbox file) to 1.464
at the end from the process, I can see vers 1.464 in the web-admin page but for example still with original QTH grids, etc.....
So I did not change anything and keep all fields original
2.) reboot the kiwi server = all connection lost
So: even without any passwords - nothing
Ulli, set up remote logging, pipe the syslog to a remote syslog server.
I can't help feeling someone else should have hit the exact same issue if it is password fields.
Stu
the lingering original QTH etc is a clue. If you can connect to the console from the admin page, I'd delete (rm) all the kiwi stuff that lives in root's file area to clean out old stuff. Then do another new OS from the SD card. There are other ways to accomplish this but IMO, the issue is the lingering old stuff.
Something else is going on here. I have no problems using Unicode in the admin password field with v1.464. How are you entering UTF-8 into the password field? Just pasting UTF-8 characters in directly? (that's what I did). Or using some sort of typed escape sequence?
When you have a Kiwi with admin page problems please do a couple of things. First, do you get just a white/blank screen or a browser "can't connect" error? Second, bring up the browser's Javascript console and refresh the admin page. Now look for errors in the console messages. Or copy/paste the whole thing here as an attachment. Key sequences to bring up the Javascript console:
Browser // Windows/Linux // Mac
Firefox // ctrl-shift-K // cmd-alt-K
Chrome // ctrl-shift-J // cmd-alt-J
Safari // ctrl-shift-C // cmd-alt-C
Opera // ctrl-shift-I // cmd-alt-I
Third, if you can, ssh/PuTTY into the Beagle and look at the config file directly. Do the commands:
cdk
head admin.json
For my admin password that has the single UTF-8 character 📶 ("ANTENNA WITH BARS", see below) I get the following where F0 9F 93 B6 is the URI encoded 4-byte UTF-8 sequence.
"user_password": "",
"user_auto_login": true,
"admin_password": "%F0%9F%93%B6",
"admin_auto_login": true,
A very handy website for decoding UTF-8 is here:
www.ltg.ed.ac.uk/~richard/utf-8.cgi?input=📶&mode=char
Also, please email, as attachments, the contents of your ~/kiwi.config/admin.json and ~/kiwi.config/kiwi.json files to support@kiwisdr.com
Don't scrub anything from the files (i.e. leave the password fields intact so I can see the exact encoding).
@Ulli We did not resolve from a couple days ago what those strange, possible filesystem errors were in the build log. If you use the
dmesgcommand do you see any eMMC/sd write errors logged?/bin/sh: 1: cannot open ../build/obj_O3/sstv.d.tmp: No such file
/bin/sh: 1: cannot open ../build/obj_O3/sstv_common.d.tmp: No such file
John,
"...First, do you get just a white/blank screen or a browser "can't connect" error?"
Yes, exactly - connectivity breaks - so I need to rebuild everything to input anything you suggest.
But it is already almost midnight - it will be for tomorrow...
Ulli
@Ulli You didn't answer my question. Blank screen or "can't connect" from browser are two hugely different problems. I need to know exactly which it is.
"this site can't be reached'.... white background... nothing else
I am rebuilding the kiwi-OS again from flash-card. I use the console with update_fix command now - need to wait almost 90min to see, if the strange file-system errors appear again...
Will let you know...
I see now different faults:
for example
That did not happen before!
upgrade is not ready yet... just saw this fault after the first minutes of upgrade 1.390 --> 1.464
I'll ask again, why are people building from the flash card then using update_fix?
Why not just build from flash card and LET IT UPDATE normally.
I thought the flash card build replaced version back to one where updates worked (not the specific version mentioned in this thread) so there was no need to do anything except let it upgrade.
The reason is, that I want to see the response of the update_fix in the console, to see possible errors - I can't see that when doing automatic updates...