FPGA not responding [fixed]
Hi all,
I was working on upgrading my kiwi to Debian 10 according to the forum posts and I was getting issues after loading the 10 image.
I first thought it was the image, so used other SD card and even tried to transfer the image from the beagle to the SD card.
At the end i restored the back, which allowed me to login again (with 10 image I get ping reply but I cannot login to the beagle, even when using the serial number as password).
The kiwi however is not starting. In the logs I FPGA not respondig. Any idea what is causing that?
I checked all the voltages and they are good (no oscilloscope at my disposal though).
From the log:
Feb 15 02:37:48 kiwisdr kiwid: 00:00:00.446 KiwiSDR v1.489 --------------------------------------------------------------------
Feb 15 02:37:48 kiwisdr kiwid: 00:00:00.450 compiled: Feb 1 2022 19:40:36 on kiwisdr
Feb 15 02:37:48 kiwisdr kiwid: 00:00:00.451 -debian 8
Feb 15 02:37:48 kiwisdr kiwid: 00:00:00.451 /etc/debian_version 8.5
Feb 15 02:37:48 kiwisdr kiwid: 00:00:00.451 background mode: delaying start 30 secs...
Feb 15 02:38:18 kiwisdr kiwid: 00:00:30.501 reading configuration from file /root/kiwi.config/kiwi.json: 387 tokens
Feb 15 02:38:18 kiwisdr kiwid: 00:00:30.506 reading configuration from file /root/kiwi.config/admin.json: 121 tokens
Feb 15 02:38:19 kiwisdr kiwid: 00:00:30.893 serial number from EEPROM: 4059
Feb 15 02:38:19 kiwisdr kiwid: 00:00:30.956 reading configuration from file /root/kiwi.config/dx.json: 9004 tokens
Feb 15 02:38:19 kiwisdr kiwid: 00:00:30.960 908 dx entries
Feb 15 02:38:19 kiwisdr kiwid: 00:00:30.974 .... firmware: SDR_RX4_WF4
Feb 15 02:38:19 kiwisdr kiwid: 00:00:30.978 .... firmware: rx_chans=4 wf_chans=4
Feb 15 02:38:19 kiwisdr kiwid: 00:00:30.981 .... firmware: RX bufs=4 samps=170 loop=85 rem=0 intr_usec=14166
Feb 15 02:38:19 kiwisdr kiwid: 00:00:30.981 .... firmware: WF xfer=9 samps=911 rpt=50 loop=18 rem=11
Feb 15 02:38:19 kiwisdr kiwid: 00:00:30.981 .... listening on default port 8073/8073 for "openwebrx"
Feb 15 02:38:19 kiwisdr kiwid: 00:00:30.984 .... webserver for "openwebrx" on port [::]:8073
Feb 15 02:38:19 kiwisdr kiwid: 00:00:31.215 .... ### using SPI_DEV /dev/spidev1.0
Feb 15 02:38:20 kiwisdr kiwid: 00:00:31.994 .... FPGA not responding: 0xffff
Feb 15 02:38:32 kiwisdr kiwid: 00:00:00.692 KiwiSDR v1.489 --------------------------------------------------------------------
8073 doesnt seem to be listening:
root@kiwisdr:~# netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 *:ssh *:* LISTEN
tcp6 0 0 [::]:ssh [::]:* LISTEN
udp 0 0 *:bootps *:*
udp 0 0 *:1900 *:*
udp 0 0 192.168.7.2:ntp *:*
udp 0 0 kiwisdr.acme.local:ntp *:*
udp 0 0 localhost:ntp *:*
udp 0 0 *:ntp *:*
udp 0 0 *:39318 *:*
udp 0 0 *:59306 *:*
udp 0 0 *:mdns *:*
udp6 0 0 [::]:60273 [::]:*
udp6 0 0 fe80::7a04:73ff:fe5:ntp [::]:*
udp6 0 0 ::1:ntp [::]:*
udp6 0 0 [::]:ntp [::]:*
udp6 0 0 [::]:mdns [::]:*
Active UNIX domain sockets (only servers)
Proto RefCnt Flags Type State I-Node Path
unix 2 [ ACC ] STREAM LISTENING 15189 /var/run/minissdpd.sock
unix 2 [ ACC ] STREAM LISTENING 14438 /var/run/avahi-daemon/socket
unix 2 [ ACC ] STREAM LISTENING 14440 /var/run/dbus/system_bus_socket
unix 2 [ ACC ] STREAM LISTENING 12727 /run/systemd/private
unix 2 [ ACC ] SEQPACKET LISTENING 12749 /run/udev/control
unix 2 [ ACC ] STREAM LISTENING 12752 /run/systemd/journal/stdout
Comments
The web server won't start if it can't talk to the FPGA (doesn't know a cape is connected).
If you say more specifically, the 'issues' you had on Debian10, that might help diagnose.
Was it that you could not get 10 to start installing at all, even with the user button pressed?
thanks for the reply.
Yes i had to press the boot button to get it to load 10.
After that the lights will go off, i remove the power and SD card and boot. DHCP will provide the IP (i have a reservation for the mac) and I can SSH to it, but it wont accept my credentials.
From another post I understand that it should be my serial number, but that doesnt work.
I also downloaded a clean 8 image and I have exactly the same SSH issue with that one.
I suspect i have the same problem on those images as with my backup image, that the FPGA isnt responding, but I have no way of checking ...
According to the messages log it can read the serial number, so I would assume there is communication with the board ... Im just afraid this is a hardware issue now instead of software (I had to get it out of the case, not sure how sensitive it is)...
The username under 10 is debian (root ssh not allowed by default) so it probably required debian and the serial number.
On 8 I beleive it depends if you have already a set a password and what version the install image starts from, or is currently updated to.
I'd be tempted to push for 10 but try debian / serial
OK on the comms, I missed that, (looking while I should be working..) the no FPGA is probably an overlay issue which I don't understand at this stage but I'd go for Debian 10 anyway and fault find from there.
Yes, I would go back to trying to get the Debian 10 re-flash to work. Start with the instructions section "Establish communication with the Kiwi admin interface".
Rather than wait for any network update to complete try logging in immediately as user
debianwith passwordtemppwd(note two letter "p"s). You can't login as userrootin Debian 10. And if the software update fails for some reason thedebianaccount password change to the serial number will not occur. Use the commandsudo suto get a root shell.With a root shell you can poke around and see if the update is running, why it might have failed, and if it worked, continue with the instruction section "Restore the configuration to the Kiwi".
Getting the "FPGA not responding" message is really bad. Like Stu implied it could be an issue with the kernel device tree (overlay) not being installed properly so the I/O pins between the Beagle and FPGA are not initialized correctly. The procedure for doing this changed between Debian 8 and 10. So if a Debian 10 installation was interrupted before it completed, and the Debian 8 way of setting up the device tree was left behind, this might explain the issue. But that's just a guess. You really need to get logged in to Debian 10 to check.
Ok thank you both ... I missed the root thing with debian 10, so let me try that again tonight when I have some time to sit down for that ...
edit:
@jks ok im in debian 10 now. Im seeing the errors as when I restore my original config.
i did an up and let is completely update. But that didnt change the FPGA not responding msg. My debian password has also not been changed to my serial number. Its temppwd ...
Any idea what is going on?
Is it the same after a power cycle?
Yes, same after a power cycle ...
Not sure if this output helps:
The Debian password won't be changed until the server code makes it far past where is stops due to the FPGA communication problem.
There are more debugging messages printed if you run the server manually. To do that type:
sudo su (if your shell is not already a root shell)
cdp (change to server directory)
mst (stops the background server)
./d (runs server in debug mode)
Paste the full set of messages here please.
same as in my previous screenshot:
Well, not completely the same. All the extra lines not starting with "L", when run as ./d, have extra debugging info. It all looks good.
Do you have access to another Kiwi? It would be interesting to swap boards and see if the problem travels with the Kiwi board (hardware issue) or stays with the Beagle (software issue).
It was there, edited the post... might not have updated for you ...
I wish i had, that would have been my first action ... sadly i dont have...
Anyway to get more debug out of a manual connect string maybe? is there something like lsusb for spi?
hmm, took everything apart and put it back (kiwi and beagle), which i have done before ...
And now its running ... I need to play around a bit more (with the risk of breaking it again), maybe a bad contact on one of the pins?
You mean it's fully working now?
Yeah, you have to be 100% certain you align the pins from the Kiwi board correctly with the header socket pins when you put them back together. Because if you are off even one row incorrect voltages will be applied to the Kiwi board and it will be instantly destroyed.
The printed instructions that come with the Kiwi warn about this (also here: http://kiwisdr.com/quickstart/quickstart.pdf)
ive got it fully running with debian 10 and my config restored ...
Yeah im sure the pins were correctly alligned (and it shows, its not destroyed :) )....
Let take it all apart again and see what it does. I need to put it back in the housing as well ... At least we know the chip isnt broken or something ...
took it apart again, put it in the housing and reconnected all wires ...
All fine still, must have been a bad contact or something ... or gremlins
Thanks for all the help, im back in the air!
Cheers,
Rick
Just for posterity: I had the exact same issue describe in this thread, solved the same way. I hope pins aren't getting loose!
<Cont'd>
Currently my kiwi1 is encountering a similar issue (FPGA not responding 0xffff).
But unfortunately after separating kiwi/BB (then w/ air gun spray)& carefully putting them back together again, no good progress could be seen.
I'm not really willing to disassemble the kiwi2 board and conduct a swap test to find the cause of failure. Planning to get a new BB as replacement first.