Programming Comments - How to revive a BeagleBone with a bad eMMC

Summary

I had a BeagleBone Green Wireless that worked fine when I first received it. Several months later, I couldn't get it to work anymore. I tried to re-install numerous versions of boot images, but even that wouldn't work. It would get partway through the installation, then the 4 LEDs would start flashing in unison. Every once in a while it would finish installing the image, but then wouldn't boot correctly.

This is how I discovered what was wrong, and how I fixed it.

Serial Console

I initially did not own a serial debug cable for the BeagleBone computers. Seeed no longer sells them, and Adafruit wanted $42 USD to ship a debug cable. Considering a new BBGW is $35, this did not make sense to me.

Eventually found a Prolific PL2303 debug cable on Amazon for $4 CAD, including shipping. Worked perfectly. Connect the serial debug cable as follows to the BeagleBone:

black wire on pin #1 (GND)
green wire on pin #4 (RX)
white wire on pin #5 (TX)
red wire is not connected (+5V, will burn out the BB if connected)

What I finally observed was a sequence of serious I/O error messages logged to the serial console during the installation of the image:

[  116.634355] edma 49000000.edma: edma_prep_slave_sg: Failed to allocate a descriptor
[  116.642061] omap_hsmmc 481d8000.mmc: prep_slave_sg() failed
[  116.647671] omap_hsmmc 481d8000.mmc: MMC start dma failure
[  117.305788] mmcblk1: unknown error -1 sending read/write command, card status 0x900
[  117.313590] blk_update_request: I/O error, dev mmcblk1, sector 4720640
[  117.320212] blk_update_request: I/O error, dev mmcblk1, sector 4720648
[  117.326810] blk_update_request: I/O error, dev mmcblk1, sector 4720656
[  117.333405] blk_update_request: I/O error, dev mmcblk1, sector 4720664
...

Error Block Detection

I did a bit of reading, and determined what I wanted to use is badblocks(8). Turns out this can be called directly by mkfs:

man 8 mkfs.ext4
...
-c	Check the device for bad blocks before creating the file system.
	If this option is specified twice, then a slower read-write test
	is used instead of a fast read-only test.

On the micro SD card, prior to inserting it into the BeagleBone, edit the file /opt/scripts/tools/eMMC/init-eMMC-flasher-v3-bbgw.sh (or the appropriate file if it isn't a BBGW device). Look for the 3 places where mkfs is called, and add the parameters "-c -c". For example:

	message="mkfs.ext4 -c -c ${ext4_options} ${destination}p2 -L ${rootfs_label}" ; broadcast
	echo "-----------------------------"
	LC_ALL=C mkfs.ext4 -c -c ${ext4_options} ${destination}p2 -L ${rootfs_label}
	echo "-----------------------------"

When the BB boots with these changes, the installation will take much longer to run. But the entire eMMC will be checked, and bad blocks will be identified and skipped.

With these changes in place, this is what you should see on the serial console during the installation of the image:

mkfs.ext4 -c -c -O ^metadata_csum,^64bit /dev/mmcblk1p1 -L rootfs
-----------------------------
mke2fs 1.43 (17-May-2016)
Discarding device blocks: done
Creating filesystem with 954112 4k blocks and 238560 inodes
Filesystem UUID: 400838d6-1e13-444e-95d4-b684016ce029
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736
Testing with pattern 0xaa: done
Reading and comparing: done
Testing with pattern 0x55: done
Reading and comparing: done
Testing with pattern 0xff: done
Reading and comparing: done
Testing with pattern 0x00: done
Reading and comparing: done
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done
-----------------------------
Formatting: /dev/mmcblk1 complete

In my case, this corrected the I/O errors and helped revive a device that otherwise had completely stopped working.