Debian 8.x (Jessie) to 10.11 (Buster) upgrade for Kiwis using BeagleBone Green/Black
These instructions are now OBSOLETE as there is a procedure for upgrading to Debian 11.
It looks like you're new here. If you want to get involved, click one of these buttons!
These instructions are now OBSOLETE as there is a procedure for upgrading to Debian 11.
Comments
Debian 8.x (Jessie) to 10.11 (Buster) upgrade for Kiwis using BeagleBone Green/Black
[updated 11-Jul-22]
[updated 27-Jan-22]
Important: For now don't attempt this upgrade if you configure your Kiwi to use a static Ethernet ip address (as opposed to DHCP assigned). A change to the Kiwi code needs to be made first.Should be okay now as this is fixed in v1.487 which is built as part of the procedure below.Part 1 of 3
None of the following applies to BeagleBone AI systems which use Debian 9. Similarly for AI-64 systems. These instructions are for Kiwis using BeagleBone Green or Black ONLY.
Debian 8.x, used by the current Kiwi software on BB Green or Black, is no longer receiving security updates. See: www.debian.org/News/2020/20200709 For this reason some of you will be interested in upgrading to a later Debian release.
The Kiwi software has been ported to Debian 10.11 (Buster). But there are some strict requirements that must be met before the upgrade can be performed:
The upgrade process unfortunately cannot be automated. You must have some familiarity working with computers, the Kiwi hardware, Linux and micro sd cards.
Requirements:
Procedure:
Read all of these instructions and make sure you understand them before proceeding. If any of the steps gives you an error message or an unexpected result STOP and ask for help on the Kiwi forum or support email.
Login to the Beagle using ssh or PuTTY
Find a way to login to a Debian shell running on the Beagle. This is not the same as the web-based connection to the Kiwi admin page. It is a text-based connection to Debian/Linux running on the Beagle so you can perform certain operations while the Kiwi software is not running.
From an external PC/laptop you typically use the program
ssh
from Linux or Mac. From Windows usessh
from the command prompt or thePuTTY
program. Use the same name of your Kiwi to connect, but not the Kiwi port number. For example, if you usemy_kiwi:8073
to connect from a browser just use themy_kiwi
part with ssh/PuTTY. Specifyroot
as the user name, e.g.ssh root@my_kiwi
. You will be prompted for the root account password. This will either be the Kiwi's serial number, available on the admin page, "network" tab (second line, right) or written in the white silkscreen box on the top of the Kiwi circuit board. Alternatively the password might be the same as the Kiwi admin password. If your Kiwi happens to be running a software version earlier than v1.354 then the password might be blank, i.e. don't enter anything when prompted for the password (just type the enter/return key for ssh). If you've ever set a custom root password then that password will probably still be in effect.Make an ssh/PuTTY connection to the Kiwi now. Once connected type the command
dog
. You should see output similar to the following:Debian 8.11 [any value from 8.5 to 8.11 is okay]
BeagleBoard.org Debian Image 2016-05-13
Linux www 4.4.9-ti-r25 #1 SMP Thu May 5 23:08:13 UTC 2016 armv7l GNU/Linux
Leave the connection open. We'll return to it in a later step.
Backup
Make a full backup of your current Debian 8 system. Go to the admin page, "backup" tab. Insert an sd card (8 GB minimum) into the Beagle's sd card slot. The exposed contacts of the sd card should face away from the bottom of the Beagle circuit board as you insert the card into the slot (this seems counterintuitive). Push the card until it clicks as it locks in place.
Despite the warning shown in the red box about removing the card immediately after the backup completes, leave the card installed for the next step. Click the button "Click to write micro-SD card". The backup should take less than 10 minutes and say "WORKED" in green on the second line, right, if it completes successfully. If it says "ERROR" note the error message in the last few lines of the scrolling text box at the bottom. Try another sd card as they have high failure rates.
Continue with Part 2 in the next post.
Part 2 of 3
Verify the Kiwi configuration files were correctly saved
The Kiwi configuration files saved as part of the backup will be used to restore your Kiwi configuration to the new Debian 10.11 version. It is very important to check now for their integrity on the backup. Return to the ssh/PuTTY connection and type:
mmc
The output should be similar (not exact) to the following. mmcblk1 must be present. It says "29.8G" because a 32 GB SD card was being used.
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
mmcblk0boot0 179:8 0 2M 1 disk
mmcblk0boot1 179:16 0 2M 1 disk
mmcblk0 179:0 0 3.7G 0 disk
└─mmcblk0p1 179:1 0 3.7G 0 part /
mmcblk1 179:24 0 29.8G 0 disk
└─mmcblk1p1 179:25 0 2.3G 0 part
mosd1
cd /media/sd_boot/root/kiwi.config
The diff commands should produce no output showing that the files have no differences.
/usr/bin/diff admin.json ~/kiwi.config
/usr/bin/diff kiwi.json ~/kiwi.config
/usr/bin/diff dx.json ~/kiwi.config
cd
umsd
Remove the sd card. Push it into the socket until it clicks and releases. Then you can remove it from the card slot.
Use the
halt
command to shutdown Debian and power off the Beagle.Prepare the Debian 10.11 sd card image and re-flash the Beagle
Use a second sd card for this step (i.e. do NOT use the card you just used for backup above). Use an 8 or 16 GB card if you can. 32 GB and larger cards may or may not work due to booting issues. For 32+ GB cards you will likely have to hold down the Beagle boot button (switch) before power up (release a few seconds after the 4 LEDs first light up).
Follow these instructions kiwisdr.com/quickstart/index.html#id-dload to place the Kiwi/Beagle-specific Debian 10.11 image on the card. Use the card to re-flash the Beagle as described in the instructions. Be certain to remove the card after the re-flashing completes and before you power up the Kiwi.
[updated] As mentioned in other posts below you'll probably have to hold down the Beagle
user/boot
button for the Beagle to properly boot off the sd card.Establish communication with the Kiwi admin interface
Determine the (possibly) new local ip address of the Kiwi. Remember that the new Debian 10 image defaults to using DHCP to assign an ip address to the Ethernet port and also defaults to using port 8073. The documentation at kiwisdr.com gives numerous ways of determining the ip address. The easiest are described here: kiwisdr.com/quickstart/index.html#id-just-tell-me
So if you have been using a static ip address then later on you will have to repeat whatever procedure you used when the Kiwi was first installed to transition from using DHCP to a static ip. Note that if you've setup (in your router) DHCP constant ip assignment via association with the Ethernet MAC address that this will still work. This is because the MAC address is unchanged across Debian upgrades.
Wait for any possible network software update to complete
The Debian 10 image contains version v1.486 of the Kiwi software. It is likely the current Kiwi software version is newer than this. After you re-flash and power up the Beagle the v1.486 software will begin an automatic software update if it can reach kiwisdr.com over the Internet.
[updated] This is now true. Version v1.487 (or greater) now exists.
Once connected to the admin interface, go to the "update" tab and click the "check now" button. If the response indicates "update in progress" then you must wait for it to complete. This may take 10 to 30 minutes. Reload the page and click the button until the response is "running the most current version". There will be a short period of time when the page reload will not respond as software is restarting.
It is also possible you will get an error message if the Kiwi is unable to reach kiwisdr.com to check the update status. It's not essential to get the update process working here. Only that you wait for it to complete if it's running. This situation might occur if you had a complicated network setup with Debian 8. It may be better to restore the Kiwi configuration in the next step and fix the network situation as a final step.
Continue with part 3 in the next post.
Part 3 of 3
Restore the configuration to the Kiwi
At this point the Kiwi server will be running on Debian 10. It is important to stop it before restoring the configuration files. Otherwise a restored file will be immediately overwritten when the Kiwi software decides it has to save a configuration value. Proceed as follows:
ssh/PuTTY into the Kiwi which is now running Debian 10. Because of tighter security you can no longer login to the root account on a new Debian setup. Use the "debian" user account, e.g. "ssh debian@my_kiwi", "ssh debian@kiwisdr.local", "ssh debian@192.168.1.xxx" etc. The password will always be the Kiwi's serial number since a clean copy of Debian is being used. See the earlier instructions about where to find the serial number.
Now use the command
sudo su
to obtain a root shell. Use the serial number as password again. A post on the 2nd page of this thread shows how to modify/etc/ssh/sshd_config
to enable root logins.Stop the Kiwi server.
cdp
mst
Insert backup sd card you made earlier on.
mmc
Should show output similar to the following. mmcblk0 must be present. Note that the mmc0/mmc1 number is swapped compared to Debian 8!
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
mmcblk1boot0 179:8 0 2M 1 disk
mmcblk1boot1 179:16 0 2M 1 disk
mmcblk1 179:0 0 3.7G 0 disk
└─mmcblk1p1 179:1 0 3.7G 0 part /
mmcblk0 179:24 0 29.8G 0 disk
└─mmcblk0p1 179:25 0 2.3G 0 part
Mount the sd card, copy the configuration files, verify the files, unmount the card.
mosd0
cd /media/sd_boot/root/kiwi.config
Copy the required configuration files.
cp admin.json kiwi.json dx.json ~/kiwi.config
diff admin.json ~/kiwi.config
diff kiwi.json ~/kiwi.config
diff dx.json ~/kiwi.config
[New] Copy the optional files. Ignore any error about a particular file being missing.
cp config.js photo.upload noip2.conf frcp.ini ~/kiwi.config
cp opt.admin_ip opt.no_console opt.git_no_https ~/kiwi.config
Unmount the sd card.
cd
umsd
Start the Kiwi server.
ku
Your Kiwi should now be running again using your specific configuration.
Remaining issues
ssh
connection you have open.End of procedure
Can it be that the new installer scripts still will install Deb 8.5 and Version 390?
Okay, bug in the script on kiwisdr.com
@jks Thanks! Does it possible full remote upgrade Debian for BBG like for AI with change /etc/apt/sources.list?
Does it possible full remote upgrade Debian for BBG like for AI with change /etc/apt/sources.list?
It would not be any less work. And much more error prone for most users.
From what I've read you cannot skip Debian releases. So you would have to upgrade D8 => D9 => D10. And any changes to sources.list have to correctly include all the BeagleBone repos (i.e. the ones from rcn-ee.com). Then you have to apply all the Kiwi-specific changes to D10 that are different from D8 (e.g. differences with /boot/uEnv.txt).
Rather than try and talk people through all that it was just safer to develop and test a fully working Kiwi-specific D10 image.
Yes, I understand, - re-flashing from microSD is faster and more stable way to upgrade, but if KiwiSDR is at a remote position, then this will be difficult. If you can give more info about how-to "apply all the Kiwi-specific changes to D10" it was perfect... Thanks.
Observations so far:
@G4DYA You need disconnect power from BBG, insert micro-SD card, press the "USER" button and fix it, then connect power, after 30-40 sec your BBG start flashing from micro-SD card (see LED status) and you can release the button. After flash compleate BBG is shutdown. Remove micro-SD and continue setup as instructed. You can flash new image to BBG without KiwiSDR cape and power it via micro USB cable from power bank. Hope it's helpful.
It only boots from the micro-SD card if you hold down the user button on the BBG while re-powering. (It's worth re-reading the quick-start guide that came with the hardware for info on this.)
I think this might be true only the very first time you ever use an sd card as a flasher. I only had it happen once using a fresh card during D10 development. And not in years using the pile of other flasher cards I reuse.
It's also necessary to backup and restore config.js and photo.upload if you've altered these.
Thanks. You don't to do anything to backup these files as all files are included in the backup function. But I have added these files, and a few more, to the restore instructions.
The Kiwi admin's network config doesn't. Debian 10 uses Connection Manager (connman) and this seems to override settings in Kiwi network config. Setting a static IP with connmanctl works OK over reboots.
Okay, this is quite possible. I note now that the
/etc/network/interfaces
file is still there in D10. But in the default DHCP mode all the entries foren0
are commented out. And just now I've found D10-specific documentation about connman and how it interacts with/etc/network/interfaces
Looks like I'll have to change the kiwi code to invoke the necessary connman commands to setup static ip addressing when D10 is being used.The directory /var/lib/connman/ethernet_<mac_address>_cable/ should presumably not have been included as this is specific to one of John's RXs.
Probably. But this isn't preventing your eth0 interface (with a different MAC) from working, right? It didn't cause me any problems. So this is just a stale cache entry. I wonder if connman is smart enough to delete it after a while.
I think this might be true only the very first time you ever use an sd card as a flasher. I only had it happen once using a fresh card during D10 development. And not in years using the pile of other flasher cards I reuse.
I had same problem like @G4DYA and procedure with pressed button 100% worked for all my BBG.
I tried to boot from two SD cards without success before holding down the user button while powering up.
The stale entry in /var/lib/connman didn't cause a problem.
The code to invoke connman will need to execute two commands, one to set the ip address, netmask and gateway and the other to set the ip addresses of the DNS servers. (These changes are implemented immediately, so it's worth doing the DNS one first because the IP change will stop your ssh session.)
In both cases the ethernet MAC address needs to be in the command,
connmanctl config ethernet_<macaddress>_cable --nameservers X.X.X.X Y.Y.Y.Y
connmanctl config ethernet_<macaddress>_cable --ipv4 manual IP_address netmask gateway_IP
If you can give more info about how-to "apply all the Kiwi-specific changes to D10"
After the D8 => D9 => D10 upgrade at some point you'll do a
gclone Beagle_SDR_GPS
followed by amake; make install
. The Makefile should do most of the required customizations (e.g. install needed packages). But there are a few things I did by hand that would of been present in the image and which you must now do yourself.1) In /boot/uEnv.txt after the line: (note how it's commented out)
#uboot_overlay_addr4=/lib/firmware/<file4>.dtbo
add one that says:
uboot_overlay_addr4=/lib/firmware/cape-bone-kiwi-00A0.dtbo
Reboot after making this change and having done the
make install
above.2) You'll have to change the default debian account password ("
temppwd
") manually. This because the Kiwi automatic password change mechanism doesn't work without some other changes that are more complicated than just changing it manually.I think that might be all you have to do. I hope doing this doesn't brick your Kiwi at a remote location.
Okay, v1.487 uses connmanctl on Debian 10 to change the Ethernet ip and dns.
One of the important things for complete remote upgrade is upgrade U-Boot version:
and after that edit uEnv.txt file...
For any users who want to do remote upgrade D8>D9>D10, - be careful and think about possible problems (like kiwisdr brick on the ~2000 km from your home), test full upgrade at the home site first.
Take care of your health, 73!
PS Remote upgrade BBG was complete successfully on my http://msk.swl.su:8073:
For some reason I was unable to recover the kiwi config files from the backup (two different BBG, both reported SD card write success on backup). The step 3
.config
had only two files in both times. Highly likely to be my fault but couldn't spot my error.Out of laziness I used WinSCP to take a copy of kiwi.config directory (as root on Debian 8) then copied it back to the new 10 the same way but as user 'debian' on D10 (to /home/debian/ directory) then copied files from there as per restore / diff instructions via ssh. (had to "apt install colordiff")
Only strange thing I could not quite figure was that on the "public" part of the admin page the Maidenhead locator was back to default although the lat/log was correct.
I wrote simple bash script for backup /root/kiwi.config/ directory to tar.gz file with IP address and date on his name (look like kiwi.config_192.168.1.5_2022-01-12.tar.gz about 8MB). If I do any changes on Admin page I run this script under "debian" user with "sudo", then copy creating file to my NAS via SCP protocol. It's not for this upgrade, but having backup is always a good idea. :)
@jks I have read all the above and also the updated:
"Optionally downloading the software (if you lost the SD card etc.)" on the website.
I follow the instructions on making a new image..
But then you go straight on to talk about downloading from wget....
.
.
My question is:
It this a second method of upgrading to Debian 10.. Assuming the BBG/BBB is already running Dedian 10.
If this is the case can you please make it a little clearer.
Many thanks,
Jimo
The instructions in the first post of this topic at one point say:
Follow these instructions kiwisdr.com/quickstart/index.html#id-dload to place the Kiwi/Beagle-specific Debian 10.11 image on the card. Use the card to re-flash the Beagle as described in the instructions. Be certain to remove the card after the re-flashing completes and before you power up the Kiwi.
And those QuickStart instructions on kiwisdr.com offer two options for putting the image on the sd card: one if using Mac or PC (Windows), and another if using Linux (including a Beagle). The Mac/PC method involves various third-party tools for writing sd cards. But the Linux/Beagle method does it all from a small script you download using wget (the script uses standard Linux tools). The script itself does another download of the much larger image file.
Once you have re-flashed to the Kiwi-customized Debian 10 image you continue with the instructions in the first post of this topic.
I don't know what you mean "Is this a second method of upgrading to Debian 10". It is the only way of upgrading to Debian 10 that will get you the proper Kiwi customizations*.
*There is @rz3dvp's method, but we do not support it because it requires significant Linux/Debian knowledge and it is too easy for you to make a mistake and get stuck. If you get stuck we would only refer you back to these instructions anyway, so you might as well start there.
@jks Thank you.
By "second method" I was thinking of the @rz3dvp method. Was hoping to use it. But I do understand now you are only referring to your method which is safer to carry out.
73's
Jimo.
@rz3dvp Can you please give me some help with your method - the rz3dvp method of remote upgrading to D10 - Buster.
Was running v1.486 - all good.
I upgraded from D8-> D9 -> D10 as per instructions and Buster was working fine.
Now to get the Kiwi running I ran:
gclone Beagle_SDR_GPS
make
make install
The 'make install' ran and right at the very end gave a error but run 'sh -x ./update_fix' and it worked fine and then 'make install' all ran fine.
But now the bootloader...
I ran:
debian@beaglebone:~$ cd /opt/scripts/tools/
debian@beaglebone: $ git reset --hard
debian@beaglebone:$ git pull
debian@beaglebone:~$ cd /opt/scripts/tools/developers/
debian@beaglebone:$ sudo ./update_bootloader.sh
I did get:
WARNING, the u-boot builds behind this repo is not updated anymore...
Please migrate to the debian package: [bb-u-boot-am335x-evm or bb-u-boot-am57xx-evm]
-----------------------------
Legacy script will resume in 5 seconds...
bootloader:[/dev/mmcblk0]:[U-Boot 2016.03-00001-gd12d09f]
Bootloader Recovery .....
Then I did debian@beaglebone:$ sudo reboot
And lost it ... )-:
So could not do John's last step ...
nano /boot/uEnv.txt and then after the line:
(note how it's commented out)
#uboot_overlay_addr4=/lib/firmware/<file4>.dtbo
add one that says:
uboot_overlay_addr4=/lib/firmware/cape-bone-kiwi-00A0.dtbo
Reboot after making this change
and having done the make install above.
Can you please let me know where I am missing a step or going wrong?
I seem to be messing up on the U-Boot Bootloader somehow.
73's
Jimo.
@Jimo write me directly to email rz3dvp@gmail.com and I will send to you my short "how-to", - how I was success upgrade my 5 KiwiSDR remotely and without problems. It's need more time, but works with all my KiwiSDR's.
@rz3dvp Thank you. Got it.
Struggling with update. Can do the shell script kiwiSDR-download-KiwiSDR-Debian-10-create-micro-SD-flasher.sh. Seem to work fine when run on the beagle. However I seem to get some sort of failure in the last step, ie writing the image to the beagle. It goes on forever flashing back and forth on the 4 leds. After a long time (hour) the 4 leds started to flash simultaneously. Beagle never turned off.
I can do the same with the original sd card, leds flashes back and forth and the beagle eventually turns off after approx 5 minutes.
I use some 32Gb sd cards, smallest available at my local shop.
Any advice?
73 de Olaf - LA3RK
I managed to upgrade to buster without using a micro SD card, simply by editing /etc/apt/sources.list and using apt. There were a few nits along the way but it seems to have upgraded and is running kiwid again. Haven't rebooted yet though.
Could you post the current contents of /etc/apt/sources.list to make sure I have them right?
I ended up bricking my unit. Oh well.
Updates really need to be easier. It's linux; why can't it just be "apt update; apt upgrade"?
With the amount of people out there who have little Linux experience its probably no bad thing that it is not a "One command and three days of support" event.
Technically I doubt your have "bricked" it, just done enough outside of the recommended method to earn your "self support" badge.
Personally I could see a method where the configs are uploaded to a central location, a new install in done via a newer card image then the configs are pulled back down via http. E.G. there was admin option to "combine and encrypt with password XXXX, then upload to kiwi.com" then recover from my.kiwi.com, the encryption being local only and no chance of config recovery if the user forgets their password, so that server never has clear text files. I'm perhaps a bit biased by having stuck my Kiwi in a case that makes getting to the SD card a pain (I obviously earnt the "Self support - Hardware" badge).
distribution of binaries rather than local builds has been mentioned and would solve a lot