Linux: Setting up a development environment - Revision history

Grinberg: /* Setting up tftp */

2017-09-18T07:27:46Z

‎Setting up tftp

Grinberg: /* Cross-Compilation */

2016-12-19T14:45:48Z

‎Cross-Compilation

Grinberg at 15:32, 20 September 2016

2016-09-20T15:32:47Z

Grinberg at 15:29, 8 February 2016

2016-02-08T15:29:16Z

New page

= Overview =
Development for an embedded device presents some unique challenges:
* Software development is almost always done on a system other than the one on which the software will eventually run. It is common nowadays to use x86 desktop workstations to develop software for ARM-based embedded products. Such an approach requires cross-compilation tools and environment.
* Communication with the embedded device is almost always done remotely. This requires the use of communication tools such as ssh or serial communication programs.
* Software deployment entails a lot of overhead: each change to the kernel version requires that all the kernel modules be replaced on the device as well. Filesystem updates can take a long time.

This article will guide you through setting up a usable development environment to tackle the unique challenges of embedded software development.

= Cross-Compilation =
A cross compiler makes it possible to compile software for a different system than the one on which development is done. In cross-compiler terminology, the host system is the system on which development and compilation is performed (usually an x86 workstation), while the target system is the architecture on which the software will run (such as ARM).

There are several options for cross-compilation toolchain setup. The cross-compiler should support the ARM embedded-application binary interface ("[http://en.wikipedia.org/wiki/Application_binary_interface#EABI EABI]"). Some of the recommended tools are:
* [http://www.mentor.com/embedded-software/sourcery-tools/sourcery-codebench/editions/lite-edition/ Sourcery CodeBench Lite]
* [http://www.denx.de/wiki/DULG/ELDK ELDK]

= Communicating with the embedded device =
During advanced stages of development it is often possible to connect a keyboard (and mouse) to the embedded device and control it directly. However, this is usually not an option during the early stages of development, and it is not the most convenient option since it forces the developer to get away from their development workstation to engage the device. A more convenient option is connecting to the device using ssh, or via a serial port.

== Serial port communication ==
The two most popular serial communication programs are [https://en.wikipedia.org/wiki/Minicom minicom] and [http://www.columbia.edu/kermit/ kermit]. Both programs also support scripting for the communication session, which can be used to automate software tests and deployment tasks.

=== Working with kermit ===
The following kermit settings should work with the vast majority of CompuLab SoM products.

* Create a kermit.conf file with the following content:
<pre>
workstation-pc # cat << eof > /path/to/kermit.conf
set carrier-watch off
set handshake none
set flow-control none
set key \127 \008
robust
eof
</pre>

* Run kermit (set TTY to the appropriate device file, e.g. ttyS0 or ttyUSB0):
<pre>
workstation-pc # sudo kermit -l /dev/<TTY> -b 115200 -y /path/to/kermit.conf
</pre>

* You should now see the kermit prompt. Type "connect" to start communicating with the device:
<pre>
C-Kermit 9.0.302 OPEN SOURCE:, 20 Aug 2011, for Linux+SSL+KRB5 (64-bit)
Copyright (C) 1985, 2011,
Trustees of Columbia University in the City of New York.
Type ? or HELP for help.
(/home/) C-Kermit> connect
</pre>

== ssh ==
ssh stands for "Secure SHell". It is an encrypted network protocol that facilitates remote login over an unsecured network. ssh can be used in advanced stages of development when the device OS has support for network communication and a running user space. Using ssh requires knowing the IP address of the embedded device, so if it is not known in advance you will have to access the device in some other way and query it using the {{cmd | ifconfig}} command. For this reason ssh is best used as an alternative means of communication with the device.

* Make sure your workstation is in the same network as the device, and initiate the ssh connection:
<pre>
workstation-pc # ssh root@<device IP address>
</pre>

= Software deployment =
Two tools that significantly simplify software deployment are TFTP (Trivial File Transfer Protocol) and NFS (Network File System).

TFTP is a simplified version of FTP. Its main advantage is that it is basic enough to be easily supported by bootloaders like U-Boot. TFTP can be used to quickly load a new bootloader or Linux kernel binaries via network once your bootloader has network support.

With NFS you can store the embedded device's filesystem on your development machine instead of storing it on the embedded device itself (the embedded device will connect to your workstation as part of Linux boot and will look for the filesystem in your NFS folder). Working this way has multiple adventages:
* Managing kernel modules is easier: instead of manually copying new kernel modules to the embedded device, the Linux build script can be setup to install new modules into the filesystem in your NFS folder.
* Sending files to the remote device is easier (just copy them to the filesystem in your NFS folder).
* It avoids filesystem divergence between embedded devices by using the exact same filesystem for all the devices.

== Setting up tftp ==
* Install the following packages:
<pre>
workstation-pc # sudo apt-get install xinetd tftpd tftp
</pre>

* Create /etc/xinetd.d/tftp:
<pre>
workstation-pc # cat << eof > /etc/xinetd.d/tftp
service tftp
{
protocol = udp
port = 69
socket_type = dgram
wait = yes
user = nobody
server = /usr/sbin/in.tftpd
server_args = /tftpboot
disable = no
}
eof
</pre>

* Create the folder described in server_args and provide it with the appropriate permissions:
<pre>
workstation-pc # sudo mkdir /tftpboot
workstation-pc # sudo chmod -R 777 /tftpboot
</pre>

* Restart the xinetd service
<pre>
workstation-pc # sudo /etc/init.d/xinetd restart
</pre>

* You can test the tftp setup by dropping a file into your /tftpboot folder, booting the embedded device into U-Boot bootloader, and attempt to download the file to memory:
<pre>
U-Boot# dhcp
link up on port 0, speed 1000, full duplex
BOOTP broadcast 1
BOOTP broadcast 2
DHCP client bound to address 192.168.50.219 (834 ms)
U-Boot# setenv serverip <your ip address>
U-Boot# tftp 80a00000 test.img
link up on port 0, speed 1000, full duplex
Using cpsw device
TFTP from server <your ip address>; our IP address is 192.168.50.219
Filename 'test.img'.
Load address: 0x80a00000
Loading: ##
2.4 MiB/s
done
Bytes transferred = 10091 (276b hex)
</pre>

== Setting up nfs ==
* Install the nfs server
<pre>
workstation-pc # sudo apt-get install nfs-kernel-server
</pre>

* Edit /etc/exports file and add the following entry:
<pre>
/path/to/nfs/export *(rw,no_subtree_check,sync,no_root_squash)
</pre>

* Start the nfs server
<pre>
workstation-pc # sudo service nfs-kernel-server start
</pre>

* You can now use kernel bootargs to point the kernel to the filesystem located in your nfs exported file. For example:
<pre>
U-Boot# setenv bootargs 'console=ttyO0,115200n8 root=/dev/nfs nfsroot=<serverip>:/path/to/nfsroot'
</pre>

= Chroot with QEMU =
QEMU is a hardware emulator and hypervisor. One of its features is providing a translation layer that dynamically translates computer instructions from one architecture to another. This makes it possible to host and chroot into a filesystem that was compiled for a different architecture than the workstation architecture. You can use this feature to interact with a filesystem that is intended to be installed on the embedded target as if you were actually running an ARM device, but your session will be backed by the x86 hardware.

== Binfmt_misc ==
The {{cmd|binfmt_misc}} Linux kernel feature allows running of different binary formats provided the existence of an appropriate interpreter. In conjunction with statically built {{cmd|qemu-arm}} or {{cmd|qemu-armeb}} the {{cmd|binfmt_misc}} provides ability to run ARM binaries on x86 systems.
{{Note|Most modern Linux distributions enable the {{cmd|binfmt_misc}} support by default}}
The {{cmd|binfmt_misc}} should be enabled in the Linux kernel configuration in {{parameter|"Userspace binary formats"}} menu:
<pre>
.config - Linux/arm 4.2.0-cm-t43-4 Kernel Configuration
> Userspace binary formats ───────────────────────────────────────────────────
┌─────────────────────── Userspace binary formats ────────────────────────┐
│ Arrow keys navigate the menu. <Enter> selects submenus ---> (or empty │
│ submenus ----). Highlighted letters are hotkeys. Pressing <Y> │
│ includes, <N> excludes, <M> modularizes features. Press <Esc><Esc> to │
│ exit, <?> for Help, </> for Search. Legend: [*] built-in [ ] │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ [*] Kernel support for ELF binaries │ │
│ │ [*] Write ELF core dumps with partial segments │ │
│ │ <*> Kernel support for scripts starting with #! │ │
│ │ <*> Kernel support for MISC binaries │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
├─────────────────────────────────────────────────────────────────────────┤
│ <Select> < Exit > < Help > < Save > < Load > │
└─────────────────────────────────────────────────────────────────────────┘
</pre>

== QEMU ==
Statically build QEMU ARM emulator should reside in the target (ARM) filesystem to allow invocation of ARM binaries on x86 system using {{cmd|binfmt_misc}}.
{{Note|Ubuntu Karmic Koala, Debian Sid and Squeeze and probably other Linux distributions provide pre-built packages for static QEMU}}
If the binary package is not available in your distribution, building static version of QEMU is straightforward:

wget http://download.savannah.gnu.org/releases/qemu/qemu-0.11.1.tar.gz
tar xzf qemu-0.11.1.tar.gz
cd qemu-0.11.1
./configure --static --target-list="armeb-linux-user arm-linux-user"
make
cp arm-linux-user/qemu-arm armeb-linux-user/qemu-armeb /path/to/target/rootfs/user/local/bin

{{cmd|qemu-arm}} and {{cmd|qemu-armeb}} have to be registered with {{cmd|binfmt_misc}} as ARM binary interpreters:
echo ":qemu-arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:\
/usr/local/bin/qemu-arm:" > /proc/sys/fs/binfmt_misc/register
echo ":qemu-armeb:M::\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28:\
\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:\
/usr/local/bin/qemu-armeb:" > /proc/sys/fs/binfmt_misc/register

After the above setup is complete it is possible to execute
<pre>
workstation-pc # chroot /path/to/target/rootfs /bin/sh
</pre>
and use the chroot'ed filesystem for application development

[[Category:Linux]]
[[Category:U-Boot]]
[[Category:Android]]

@@ Line 14: / Line 14: @@
 The cross-compiler should support the ARM embedded-application binary interface ("[http://en.wikipedia.org/wiki/Application_binary_interface#EABI EABI]").<br>
 Recommended tools:
 * [https://releases.linaro.org/components/toolchain/binaries/latest-5/arm-linux-gnueabihf/ Linaro GNU hard floating point]
 * [http://www.denx.de/wiki/DULG/ELDK ELDK]

@@ Line 122: / Line 122: @@
 DHCP client bound to address 192.168.50.219 (834 ms)
 U-Boot# setenv serverip <your ip address>
-U-Boot# tftp 80a00000 test.img
+U-Boot# tftpboot 80a00000 test.img
 link up on port 0, speed 1000, full duplex
 Using cpsw device
@@ Line 218: / Line 218: @@
 [[Category:U-Boot]]
 [[Category:Android]]

@@ Line 1: / Line 1: @@
-= Overview =
+== Overview ==
 Development for an embedded device presents some unique challenges:
 * Software development is almost always done on a system other than the one on which the software will eventually run. It is common nowadays to use x86 desktop workstations to develop software for ARM-based embedded products. Such an approach requires cross-compilation tools and environment.
@@ Line 7: / Line 7: @@
 This article will guide you through setting up a usable development environment to tackle the unique challenges of embedded software development.
-= Cross-Compilation =
+== Cross-Compilation ==
-A cross compiler makes it possible to compile software for a different system than the one on which development is done. In cross-compiler terminology, the host system is the system on which development and compilation is performed (usually an x86 workstation), while the target system is the architecture on which the software will run (such as ARM).
+A cross compiler makes it possible to compile software for a different system than the one on which development is done.
-There are several options for cross-compilation toolchain setup. The cross-compiler should support the ARM embedded-application binary interface ("[http://en.wikipedia.org/wiki/Application_binary_interface#EABI EABI]"). Some of the recommended tools are:
+There are several options for cross-compilation toolchain setup.
-* [http://www.mentor.com/embedded-software/sourcery-tools/sourcery-codebench/editions/lite-edition/ Sourcery CodeBench Lite]
+The cross-compiler should support the ARM embedded-application binary interface ("[http://en.wikipedia.org/wiki/Application_binary_interface#EABI EABI]").<br>
 * [http://www.denx.de/wiki/DULG/ELDK ELDK]
-= Communicating with the embedded device =
+== Communicating with the embedded device ==
-During advanced stages of development it is often possible to connect a keyboard (and mouse) to the embedded device and control it directly. However, this is usually not an option during the early stages of development, and it is not the most convenient option since it forces the developer to get away from their development workstation to engage the device. A more convenient option is connecting to the device using ssh, or via a serial port.
+During advanced stages of development it is often possible to connect a keyboard (and mouse) to the embedded device and control it directly.
-== Serial port communication ==
+=== Serial port communication ===
-The two most popular serial communication programs are [https://en.wikipedia.org/wiki/Minicom minicom] and [http://www.columbia.edu/kermit/ kermit]. Both programs also support scripting for the communication session, which can be used to automate software tests and deployment tasks.
+The two most popular serial communication programs are [https://en.wikipedia.org/wiki/Minicom minicom] and [http://www.columbia.edu/kermit/ kermit].
-=== Working with kermit ===
+==== Working with kermit ====
 The following kermit settings should work with the vast majority of CompuLab SoM products.
@@ Line 48: / Line 54: @@
 </pre>
-== ssh ==
+=== ssh ===
-ssh stands for "Secure SHell". It is an encrypted network protocol that facilitates remote login over an unsecured network. ssh can be used in advanced stages of development when the device OS has support for network communication and a running user space. Using ssh requires knowing the IP address of the embedded device, so if it is not known in advance you will have to access the device in some other way and query it using the {{cmd | ifconfig}} command. For this reason ssh is best used as an alternative means of communication with the device.
+ssh stands for "Secure SHell".
 * Make sure your workstation is in the same network as the device, and initiate the ssh connection:
@@ Line 56: / Line 66: @@
 </pre>
-= Software deployment =
+== Software deployment ==
 Two tools that significantly simplify software deployment are TFTP (Trivial File Transfer Protocol) and NFS (Network File System).
-TFTP is a simplified version of FTP. Its main advantage is that it is basic enough to be easily supported by bootloaders like U-Boot. TFTP can be used to quickly load a new bootloader or Linux kernel binaries via network once your bootloader has network support.
+TFTP is a simplified version of FTP.
-With NFS you can store the embedded device's filesystem on your development machine instead of storing it on the embedded device itself (the embedded device will connect to your workstation as part of Linux boot and will look for the filesystem in your NFS folder). Working this way has multiple adventages:
+With NFS you can store the embedded device's filesystem on your development machine instead of storing it on the embedded device itself (the embedded device will connect to your workstation as part of Linux boot and will look for the filesystem in your NFS folder).
 * Managing kernel modules is easier: instead of manually copying new kernel modules to the embedded device, the Linux build script can be setup to install new modules into the filesystem in your NFS folder.
 * Sending files to the remote device is easier (just copy them to the filesystem in your NFS folder).
 * It avoids filesystem divergence between embedded devices by using the exact same filesystem for all the devices.
-== Setting up tftp ==
+=== Setting up tftp ===
 * Install the following packages:
 <pre>
@@ Line 120: / Line 133: @@
 </pre>
-== Setting up nfs ==
+=== Setting up nfs ===
 * Install the nfs server
 <pre>
@@ Line 141: / Line 154: @@
 </pre>
-= Chroot with QEMU =
+== Chroot with QEMU ==
-QEMU is a hardware emulator and hypervisor. One of its features is providing a translation layer that dynamically translates computer instructions from one architecture to another. This makes it possible to host and chroot into a filesystem that was compiled for a different architecture than the workstation architecture. You can use this feature to interact with a filesystem that is intended to be installed on the embedded target as if you were actually running an ARM device, but your session will be backed by the x86 hardware.
+QEMU is a hardware emulator and hypervisor.
-== Binfmt_misc ==
+=== Binfmt_misc ===
-The {{cmd|binfmt_misc}} Linux kernel feature allows running of different binary formats provided the existence of an appropriate interpreter. In conjunction with statically built {{cmd|qemu-arm}} or {{cmd|qemu-armeb}} the {{cmd|binfmt_misc}} provides ability to run ARM binaries on x86 systems.
+The {{cmd|binfmt_misc}} Linux kernel feature allows running of different binary formats provided the existence of an appropriate interpreter.
 {{Note|Most modern Linux distributions enable the {{cmd|binfmt_misc}} support by default}}
 The {{cmd|binfmt_misc}} should be enabled in the Linux kernel configuration in {{parameter|"Userspace binary formats"}} menu:
@@ Line 168: / Line 185: @@
 </pre>
-== QEMU ==
+=== QEMU ===
 Statically build QEMU ARM emulator should reside in the target (ARM) filesystem to allow invocation of ARM binaries on x86 system using {{cmd|binfmt_misc}}.
 {{Note|Ubuntu Karmic Koala, Debian Sid and Squeeze and probably other Linux distributions provide pre-built packages for static QEMU}}
 If the binary package is not available in your distribution, building static version of QEMU is straightforward:
+<pre>
- wget http://download.savannah.gnu.org/releases/qemu/qemu-0.11.1.tar.gz
+wget http://download.savannah.gnu.org/releases/qemu/qemu-0.11.1.tar.gz
- tar xzf qemu-0.11.1.tar.gz
+tar xzf qemu-0.11.1.tar.gz
- cd qemu-0.11.1
+cd qemu-0.11.1
- ./configure --static --target-list="armeb-linux-user arm-linux-user"
+./configure --static --target-list="armeb-linux-user arm-linux-user"
- make
+make
- cp arm-linux-user/qemu-arm armeb-linux-user/qemu-armeb /path/to/target/rootfs/user/local/bin
+cp arm-linux-user/qemu-arm armeb-linux-user/qemu-armeb /path/to/target/rootfs/user/local/bin
 {{cmd|qemu-arm}} and {{cmd|qemu-armeb}} have to be registered with {{cmd|binfmt_misc}} as ARM binary interpreters:
- echo ":qemu-arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\
+<pre>
- \xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:\
+echo ":qemu-arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\
- /usr/local/bin/qemu-arm:" > /proc/sys/fs/binfmt_misc/register
+\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:\
- echo ":qemu-armeb:M::\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28:\
+/usr/local/bin/qemu-arm:" > /proc/sys/fs/binfmt_misc/register
- \xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:\
+echo ":qemu-armeb:M::\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28:\
- /usr/local/bin/qemu-armeb:" > /proc/sys/fs/binfmt_misc/register
+\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:\
 After the above setup is complete it is possible to execute