diff --git a/docs/.gitignore b/docs/.gitignore deleted file mode 100644 index 421e6d3e3f..0000000000 --- a/docs/.gitignore +++ /dev/null @@ -1,15 +0,0 @@ -*.log -*.aux -*.toc -*.out -*.lg -*.dvi -*.idv -*.4ct -*.4tc -*.xref -*.tmp -*.dvi -*.html -*.css -*.pdf diff --git a/docs/Makefile b/docs/Makefile deleted file mode 100644 index c113d62571..0000000000 --- a/docs/Makefile +++ /dev/null @@ -1,48 +0,0 @@ -ifeq ($(TOPDIR),) - TOPDIR:=${CURDIR}/.. -endif -PKG_NAME=docs - -all: compile - -include $(TOPDIR)/rules.mk -include $(INCLUDE_DIR)/prereq.mk - -MAIN = openwrt.tex -DEPS = $(MAIN) Makefile config.tex network.tex network-scripts.tex network-scripts.tex wireless.tex build.tex adding.tex bugs.tex debugging.tex $(TMP_DIR)/.prereq-docs - -compile: $(TMP_DIR)/.prereq-docs - $(NO_TRACE_MAKE) cleanup - latex $(MAIN) - $(NO_TRACE_MAKE) openwrt.pdf openwrt.html - $(NO_TRACE_MAKE) cleanup - -$(TMP_DIR)/.prereq-docs: - mkdir -p $(TMP_DIR) - $(NO_TRACE_MAKE) prereq - touch $@ - -openwrt.html: $(DEPS) - htlatex $(MAIN) - -openwrt.pdf: $(DEPS) - pdflatex $(MAIN) - -clean: cleanup - rm -f openwrt.pdf openwrt.html openwrt.css - -cleanup: FORCE - rm -f *.log *.aux *.toc *.out *.lg *.dvi *.idv *.4ct *.4tc *.xref *.tmp *.dvi - -$(eval $(call RequireCommand,latex, \ - You need to install LaTeX to build the OpenWrt documentation \ -)) -$(eval $(call RequireCommand,pdflatex, \ - You need to install PDFLaTeX to build the OpenWrt documentation \ -)) -$(eval $(call RequireCommand,htlatex, \ - You need to install tex4ht to build the OpenWrt documentation \ -)) - -FORCE: -.PHONY: FORCE diff --git a/docs/adding.tex b/docs/adding.tex deleted file mode 100644 index 7b80c0d1d6..0000000000 --- a/docs/adding.tex +++ /dev/null @@ -1,590 +0,0 @@ -Linux is now one of the most widespread operating system for embedded devices due -to its openess as well as the wide variety of platforms it can run on. Many -manufacturer actually use it in firmware you can find on many devices: DVB-T -decoders, routers, print servers, DVD players ... Most of the time the stock -firmware is not really open to the consumer, even if it uses open source software. - -You might be interested in running a Linux based firmware for your router for -various reasons: extending the use of a network protocol (such as IPv6), having -new features, new piece of software inside, or for security reasons. A fully -open-source firmware is de-facto needed for such applications, since you want to -be free to use this or that version of a particular reason, be able to correct a -particular bug. Few manufacturers do ship their routers with a Sample Development Kit, -that would allow you to create your own and custom firmware and most of the time, -when they do, you will most likely not be able to complete the firmware creation process. - -This is one of the reasons why OpenWrt and other firmware exists: providing a -version independent, and tools independent firmware, that can be run on various -platforms, known to be running Linux originally. - -\subsection{Which Operating System does this device run?} - -There is a lot of methods to ensure your device is running Linux. Some of them do -need your router to be unscrewed and open, some can be done by probing the device -using its external network interfaces. - -\subsubsection{Operating System fingerprinting and port scanning} - -A large bunch of tools over the Internet exists in order to let you do OS -fingerprinting, we will show here an example using \textbf{nmap}: - -\begin{Verbatim} -nmap -P0 -O -Starting Nmap 4.20 ( http://insecure.org ) at 2007-01-08 11:05 CET -Interesting ports on 192.168.2.1: -Not shown: 1693 closed ports -PORT STATE SERVICE -22/tcp open ssh -23/tcp open telnet -53/tcp open domain -80/tcp open http -MAC Address: 00:13:xx:xx:xx:xx (Cisco-Linksys) -Device type: broadband router -Running: Linksys embedded -OS details: Linksys WRT54GS v4 running OpenWrt w/Linux kernel 2.4.30 -Network Distance: 1 hop -\end{Verbatim} - -nmap is able to report whether your device uses a Linux TCP/IP stack, and if so, -will show you which Linux kernel version is probably runs. This report is quite -reliable and it can make the distinction between BSD and Linux TCP/IP stacks and others. - -Using the same tool, you can also do port scanning and service version discovery. -For instance, the following command will report which IP-based services are running -on the device, and which version of the service is being used: - -\begin{verbatim} -nmap -P0 -sV -Starting Nmap 4.20 ( http://insecure.org ) at 2007-01-08 11:06 CET -Interesting ports on 192.168.2.1: -Not shown: 1693 closed ports -PORT STATE SERVICE VERSION -22/tcp open ssh Dropbear sshd 0.48 (protocol 2.0) -23/tcp open telnet Busybox telnetd -53/tcp open domain ISC Bind dnsmasq-2.35 -80/tcp open http OpenWrt BusyBox httpd -MAC Address: 00:13:xx:xx:xx:xx (Cisco-Linksys) -Service Info: Device: WAP -\end{verbatim} - -The web server version, if identified, can be determining in knowing the Operating -System. For instance, the \textbf{BOA} web server is typical from devices running -an open-source Unix or Unix-like. - -\subsubsection{Wireless Communications Fingerprinting} - -Although this method is not really known and widespread, using a wireless scanner -to discover which OS your router or Access Point run can be used. We do not have -a clear example of how this could be achieved, but you will have to monitor raw -802.11 frames and compare them to a very similar device running a Linux based firmware. - -\subsubsection{Web server security exploits} - -The Linksys WRT54G was originally hacked by using a "ping bug" discovered in the -web interface. This tip has not been fixed for months by Linksys, allowing people -to enable the "boot\_wait" helper process via the web interface. Many web servers -used in firmwares are open source web server, thus allowing the code to be audited -to find an exploit. Once you know the web server version that runs on your device, -by using \textbf{nmap -sV} or so, you might be interested in using exploits to reach -shell access on your device. - -\subsubsection{Native Telnet/SSH access} - -Some firmwares might have restricted or unrestricted Telnet/SSH access, if so, -try to log in with the web interface login/password and see if you can type in -some commands. This is actually the case for some Broadcom BCM963xx based firmwares -such as the one in Neuf/Cegetel ISP routers, Club-Internet ISP CI-Box and many -others. Some commands, like \textbf{cat} might be left here and be used to -determine the Linux kernel version. - -\subsubsection{Analysing a binary firmware image} - -You are very likely to find a firmware binary image on the manufacturer website, -even if your device runs a proprietary operating system. If so, you can download -it and use an hexadecimal editor to find printable words such as \textbf{vmlinux}, -\textbf{linux}, \textbf{ramdisk}, \textbf{mtd} and others. - -Some Unix tools like \textbf{hexdump} or \textbf{strings} can be used to analyse -the firmware. Below there is an example with a binary firmware found other the Internet: - -\begin{verbatim} -hexdump -C | less (more) -00000000 46 49 52 45 32 2e 35 2e 30 00 00 00 00 00 00 00 |FIRE2.5.0.......| -00000010 00 00 00 00 31 2e 30 2e 30 00 00 00 00 00 00 00 |....1.0.0.......| -00000020 00 00 00 00 00 00 00 38 00 43 36 29 00 0a e6 dc |.......8.C6)..??| -00000030 54 49 44 45 92 89 54 66 1f 8b 08 08 f8 10 68 42 |TIDE..Tf....?.hB| -00000040 02 03 72 61 6d 64 69 73 6b 00 ec 7d 09 bc d5 d3 |..ramdisk.?}.???| -00000050 da ff f3 9b f7 39 7b ef 73 f6 19 3b 53 67 ea 44 |???.?9{?s?.;Sg?D| -\end{verbatim} - -Scroll over the firmware to find printable words that can be significant. - -\subsubsection{Amount of flash memory} - -Linux can hardly fit in a 2MB flash device, once you have opened the device and -located the flash chip, try to find its characteristics on the Internet. If -your flash chip is a 2MB or less device, your device is most likely to run a -proprietary OS such as WindRiver VxWorks, or a custom manufacturer OS like Zyxel ZynOS. - -OpenWrt does not currently run on devices which have 2MB or less of flash memory. -This limitation will probably not be worked around since those devices are most -of the time micro-routers, or Wireless Access Points, which are not the main -OpenWrt target. - -\subsubsection{Pluging a serial port} - -By using a serial port and a level shifter, you may reach the console that is being shown by the device -for debugging or flashing purposes. By analysing the output of this device, you can -easily notice if the device uses a Linux kernel or something different. - -\subsection{Finding and using the manufacturer SDK} - -Once you are sure your device run a Linux based firmware, you will be able to start -hacking on it. If the manufacturer respected the GPL, it will have released a Sample -Development Kit with the device. - -\subsubsection{GPL violations} - -Some manufacturers do release a Linux based binary firmware, with no sources at all. -The first step before doing anything is to read the license coming with your device, -then write them about this lack of Open Source code. If the manufacturer answers -you they do not have to release a SDK containing Open Source software, then we -recommend you get in touch with the gpl-violations.org community. - -You will find below a sample letter that can be sent to the manufacturer: - -\begin{verse} -Miss, Mister, - -I am using a , and I cannot find neither on your website nor on the -CD-ROM the open source software used to build or modify the firmware. - -In conformance to the GPL license, you have to release the following sources: - -\begin{itemize} -\item complete toolchain that made the kernel and applications be compiled (gcc, binutils, libc) -\item tools to build a custom firmware (mksquashfs, mkcramfs ...) -\item kernel sources with patches to make it run on this specific hardware, this does not include binary drivers -\end{itemize} - -Thank you very much in advance for your answer. - -Best regards, -\end{verse} - -\subsubsection{Using the SDK} - -Once the SDK is available, you are most likely not to be able to build a complete -or functional firmware using it, but parts of it, like only the kernel, or only -the root filesystem. Most manufacturers do not really care releasing a tool that -do work every time you uncompress and use it. - -You should anyway be able to use the following components: - -\begin{itemize} -\item kernel sources with more or less functional patches for your hardware -\item binary drivers linked or to be linked with the shipped kernel version -\item packages of the toolchain used to compile the whole firmware: gcc, binutils, libc or uClibc -\item binary tools to create a valid firmware image -\end{itemize} - -Your work can be divided into the following tasks: - -\begin{itemize} -\item create a clean patch of the hardware specific part of the linux kernel -\item spot potential kernel GPL violations especially on netfilter and USB stack stuff -\item make the binary drivers work, until there are open source drivers -\item use standard a GNU toolchain to make working executables -\item understand and write open source tools to generate a valid firmware image -\end{itemize} - -\subsubsection{Creating a hardware specific kernel patch} - -Most of the time, the kernel source that comes along with the SDK is not really -clean, and is not a standard Linux version, it also has architecture specific -fixes backported from the \textbf{CVS} or the \textbf{git} repository of the -kernel development trees. Anyway, some parts can be easily isolated and used as -a good start to make a vanilla kernel work your hardware. - -Some directories are very likely to have local modifications needed to make your -hardware be recognized and used under Linux. First of all, you need to find out -the linux kernel version that is used by your hardware, this can be found by -editing the \textbf{linux/Makefile} file. - -\begin{verbatim} -head -5 linux-2.x.x/Makefile -VERSION = 2 -PATCHLEVEL = x -SUBLEVEL = y -EXTRAVERSION = z -NAME=A fancy name -\end{verbatim} - -So now, you know that you have to download a standard kernel tarball at -\textbf{kernel.org} that matches the version being used by your hardware. - -Then you can create a \textbf{diff} file between the two trees, especially for the -following directories: - -\begin{verbatim} -diff -urN linux-2.x.x/arch/ linux-2.x.x-modified/arch/ > 01-architecture.patch -diff -urN linux-2.x.x/include/ linux-2.x.x-modified/include > 02-includes.patch -diff -urN linux-2.x.x/drivers/ linux-2.x.x-modified/drivers > 03-drivers.patch -\end{verbatim} - -This will constitute a basic set of three patches that are very likely to contain -any needed modifications that has been made to the stock Linux kernel to run on -your specific device. Of course, the content produced by the \textbf{diff -urN} -may not always be relevant, so that you have to clean up those patches to only -let the "must have" code into them. - -The first patch will contain all the code that is needed by the board to be -initialized at startup, as well as processor detection and other boot time -specific fixes. - -The second patch will contain all useful definitions for that board: addresses, -kernel granularity, redefinitions, processor family and features ... - -The third patch may contain drivers for: serial console, ethernet NIC, wireless -NIC, USB NIC ... Most of the time this patch contains nothing else than "glue" -code that has been added to make the binary driver work with the Linux kernel. -This code might not be useful if you plan on writing drivers from scratch for -this hardware. - -\subsubsection{Using the device bootloader} - -The bootloader is the first program that is started right after your device has -been powered on. This program, can be more or less sophisticated, some do let you -do network booting, USB mass storage booting ... The bootloader is device and -architecture specific, some bootloaders were designed to be universal such as -RedBoot or U-Boot so that you can meet those loaders on totally different -platforms and expect them to behave the same way. - -If your device runs a proprietary operating system, you are very likely to deal -with a proprietary boot loader as well. This may not always be a limitation, -some proprietary bootloaders can even have source code available (i.e : Broadcom CFE). - -According to the bootloader features, hacking on the device will be more or less -easier. It is very probable that the bootloader, even exotic and rare, has a -documentation somewhere over the Internet. In order to know what will be possible -with your bootloader and the way you are going to hack the device, look over the -following features : - -\begin{itemize} -\item does the bootloader allow net booting via bootp/DHCP/NFS or tftp -\item does the bootloader accept loading ELF binaries ? -\item does the bootloader have a kernel/firmware size limitation ? -\item does the bootloader expect a firmware format to be loaded with ? -\item are the loaded files executed from RAM or flash ? -\end{itemize} - -Net booting is something very convenient, because you will only have to set up network -booting servers on your development station, and keep the original firmware on the device -till you are sure you can replace it. This also prevents your device from being flashed, -and potentially bricked every time you want to test a modification on the kernel/filesystem. - -If your device needs to be flashed every time you load a firmware, the bootlader might -only accept a specific firmware format to be loaded, so that you will have to -understand the firmware format as well. - -\subsubsection{Making binary drivers work} - -As we have explained before, manufacturers do release binary drivers in their GPL -tarball. When those drivers are statically linked into the kernel, they become GPL -as well, fortunately or unfortunately, most of the drivers are not statically linked. -This anyway lets you a chance to dynamically link the driver with the current kernel -version, and try to make them work together. - -This is one of the most tricky and grey part of the fully open source projects. -Some drivers require few modifications to be working with your custom kernel, -because they worked with an earlier kernel, and few modifications have been made -to the kernel in-between those versions. This is for instance the case with the -binary driver of the Broadcom BCM43xx Wireless Chipsets, where only few differences -were made to the network interface structures. - -Some general principles can be applied no matter which kernel version is used in -order to make binary drivers work with your custom kernel: - -\begin{itemize} -\item turn on kernel debugging features such as: -\begin{itemize} -\item CONFIG\_DEBUG\_KERNEL -\item CONFIG\_DETECT\_SOFTLOCKUP -\item CONFIG\_DEBUG\_KOBJECT -\item CONFIG\_KALLSYMS -\item CONFIG\_KALLSYMS\_ALL -\end{itemize} -\item link binary drivers when possible to the current kernel version -\item try to load those binary drivers -\item catch the lockups and understand them -\end{itemize} - -Most of the time, loading binary drivers will fail, and generate a kernel oops. -You can know the last symbol the binary drivers attempted to use, and see in the -kernel headers file, if you do not have to move some structures field before or -after that symbol in order to keep compatibily with both the binary driver and -the stock kernel drivers. - -\subsubsection{Understanding the firmware format} - -You might want to understand the firmware format, even if you are not yet capable -of running a custom firmware on your device, because this is sometimes a blocking -part of the flashing process. - -A firmware format is most of the time composed of the following fields: - -\begin{itemize} -\item header, containing a firmware version and additional fields: Vendor, Hardware version ... -\item CRC32 checksum on either the whole file or just part of it -\item Binary and/or compressed kernel image -\item Binary and/or compressed root filesystem image -\item potential garbage -\end{itemize} - -Once you have figured out how the firmware format is partitioned, you will have -to write your own tool that produces valid firmware binaries. One thing to be very -careful here is the endianness of either the machine that produces the binary -firmware and the device that will be flashed using this binary firmware. - -\subsubsection{Writing a flash map driver} - -The flash map driver has an important role in making your custom firmware work -because it is responsible of mapping the correct flash regions and associated -rights to specific parts of the system such as: bootloader, kernel, user filesystem. - -Writing your own flash map driver is not really a hard task once you know how your -firmware image and flash is structured. You will find below a commented example -that covers the case of the device where the bootloader can pass to the kernel its partition plan. - -First of all, you need to make your flash map driver be visible in the kernel -configuration options, this can be done by editing the file \ -\textbf{linux/drivers/mtd/maps/Kconfig}: - -\begin{verbatim} -config MTD_DEVICE_FLASH - tristate "Device Flash device" - depends on ARCHITECTURE && DEVICE - help - Flash memory access on DEVICE boards. Currently only works with - Bootloader Foo and Bootloader Bar. -\end{verbatim} - -Then add your source file to the \textbf{linux/drivers/mtd/maps/Makefile}, so -that it will be compiled along with the kernel. - -\begin{verbatim} -obj-\$(CONFIG_MTD_DEVICE_FLASH) += device-flash.o -\end{verbatim} - -You can then write the kernel driver itself, by creating a -\textbf{linux/drivers/mtd/maps/device-flash.c} C source file. - -\begin{verbatim} -// Includes that are required for the flash map driver to know of the prototypes: -#include -#include -#include -#include -#include -#include -#include - -// Put some flash map definitions here: -#define WINDOW_ADDR 0x1FC00000 /* Real address of the flash */ -#define WINDOW_SIZE 0x400000 /* Size of flash */ -#define BUSWIDTH 2 /* Buswidth */ - -static void __exit device_mtd_cleanup(void); - -static struct mtd_info *device_mtd_info; - -static struct map_info devicd_map = { - .name = "device", - .size = WINDOW_SIZE, - .bankwidth = BUSWIDTH, - .phys = WINDOW_ADDR, -}; - -static int __init device_mtd_init(void) -{ - // Display that we found a flash map device - printk("device: 0x\%08x at 0x\%08x\n", WINDOW_SIZE, WINDOW_ADDR); - // Remap the device address to a kernel address - device_map.virt = ioremap(WINDOW_ADDR, WINDOW_SIZE); - - // If impossible to remap, exit with the EIO error - if (!device_map.virt) { - printk("device: Failed to ioremap\n"); - return -EIO; - } - - // Initialize the device map - simple_map_init(&device_map); - - /* MTD informations are closely linked to the flash map device - you might also use "jedec_probe" "amd_probe" or "intel_probe" */ - device_mtd_info = do_map_probe("cfi_probe", &device_map); - - if (device_mtd_info) { - device_mtd_info->owner = THIS_MODULE; - - int parsed_nr_parts = 0; - - // We try here to use the partition schema provided by the bootloader specific code - if (parsed_nr_parts == 0) { - int ret = parse_bootloader_partitions(device_mtd_info, &parsed_parts, 0); - if (ret > 0) { - part_type = "BootLoader"; - parsed_nr_parts = ret; - } - } - - add_mtd_partitions(devicd_mtd_info, parsed_parts, parsed_nr_parts); - - return 0; - } - iounmap(device_map.virt); - - return -ENXIO; -} - -// This function will make the driver clean up the MTD device mapping -static void __exit device_mtd_cleanup(void) -{ - // If we found a MTD device before - if (device_mtd_info) { - // Delete every partitions - del_mtd_partitions(device_mtd_info); - // Delete the associated map - map_destroy(device_mtd_info); - } - - // If the virtual address is already in use - if (device_map.virt) { - // Unmap the physical address to a kernel space address - iounmap(device_map.virt); - // Reset the structure field - device_map.virt = 0; - } -} - - -// Macros that indicate which function is called on loading/unloading the module -module_init(device_mtd_init); -module_exit(device_mtd_cleanup); - - -// Macros defining license and author, parameters can be defined here too. -MODULE_LICENSE("GPL"); -MODULE_AUTHOR("Me, myself and I $(KDIR)/vmlinux.bin.gz - $(STAGING_DIR_HOST)/bin/lzma e $(KDIR)/vmlinux $(KDIR)/vmlinux.bin.l7 - dd if=$(KDIR)/vmlinux.bin.l7 of=$(BIN_DIR)/openwrt-$(BOARD)-vmlinux.lzma bs=65536 conv=sync - dd if=$(KDIR)/vmlinux.bin.gz of=$(BIN_DIR)/openwrt-$(BOARD)-vmlinux.gz bs=65536 conv=sync -endef - -define Image/Build/squashfs - $(call prepare_generic_squashfs,$(KDIR)/root.squashfs) -endef - -define Image/Build - $(call Image/Build/$(1)) - dd if=$(KDIR)/root.$(1) of=$(BIN_DIR)/openwrt-$(BOARD)-root.$(1) bs=128k conv=sync - - -$(STAGING_DIR_HOST)/bin/mkfwimage \ - -B XS2 -v XS2.ar2316.OpenWrt \ - -k $(BIN_DIR)/openwrt-$(BOARD)-vmlinux.lzma \ - -r $(BIN_DIR)/openwrt-$(BOARD)-root.$(1) \ - -o $(BIN_DIR)/openwrt-$(BOARD)-ubnt2-$(1).bin -endef - -$(eval $(call BuildImage)) - -\end{Verbatim} - -\begin{itemize} - \item \texttt{Image/BuildKernel} \\ - This template defines changes to be made to the ELF kernel file - \item \texttt{Image/Build} \\ - This template defines the final changes to apply to the rootfs and kernel, either combined or separated - firmware creation tools can be called here as well. -\end{itemize} diff --git a/docs/bugs.tex b/docs/bugs.tex deleted file mode 100644 index 9c46b5a7e6..0000000000 --- a/docs/bugs.tex +++ /dev/null @@ -1,52 +0,0 @@ -OpenWrt as an open source software opens its development to the community by -having a publicly browseable subversion repository. The Trac software which -comes along with a Subversion frontend, a Wiki and a ticket reporting system -is used as an interface between developers, users and contributors in order to -make the whole development process much easier and efficient. - -We make distinction between two kinds of people within the Trac system: - -\begin{itemize} -\item developers, able to report, close and fix tickets -\item reporters, able to add a comment, patch, or request ticket status -\end{itemize} - -\subsubsection{Opening a ticket} - -A reporter might want to open a ticket for the following reasons: - -\begin{itemize} -\item a bug affects a specific hardware and/or software and needs to be fixed -\item a specific software package would be seen as part of the official OpenWrt repository -\item a feature should be added or removed from OpenWrt -\end{itemize} - -Regarding the kind of ticket that is open, a patch is welcome in those cases: - -\begin{itemize} -\item new package to be included in OpenWrt -\item fix for a bug that works for the reporter and has no known side effect -\item new features that can be added by modifying existing OpenWrt files -\end{itemize} - -Once the ticket is open, a developer will take care of it, if so, the ticket is marked -as "accepted" with the developer name. You can add comments at any time to the ticket, -even when it is closed. - -\subsubsection{Closing a ticket} - -A ticket might be closed by a developer because: - -\begin{itemize} -\item the problem is already fixed (wontfix) -\item the problem described is not judged as valid, and comes along with an explanation why (invalid) -\item the developers know that this bug will be fixed upstream (wontfix) -\item the problem is very similar to something that has already been reported (duplicate) -\item the problem cannot be reproduced by the developers (worksforme) -\end{itemize} - -At the same time, the reporter may want to get the ticket closed since he is not -longer able to trigger the bug, or found it invalid by himself. - -When a ticket is closed by a developer and marked as "fixed", the comment contains -the subversion changeset which corrects the bug. diff --git a/docs/build.tex b/docs/build.tex deleted file mode 100644 index 6e1539acf2..0000000000 --- a/docs/build.tex +++ /dev/null @@ -1,594 +0,0 @@ -One of the biggest challenges to getting started with embedded devices is that you -cannot just install a copy of Linux and expect to be able to compile a firmware. -Even if you did remember to install a compiler and every development tool offered, -you still would not have the basic set of tools needed to produce a firmware image. -The embedded device represents an entirely new hardware platform, which is -most of the time incompatible with the hardware on your development machine, so in a process called -cross compiling you need to produce a new compiler capable of generating code for -your embedded platform, and then use it to compile a basic Linux distribution to -run on your device. - -The process of creating a cross compiler can be tricky, it is not something that is -regularly attempted and so there is a certain amount of mystery and black magic -associated with it. In many cases when you are dealing with embedded devices you will -be provided with a binary copy of a compiler and basic libraries rather than -instructions for creating your own -- it is a time saving step but at the same time -often means you will be using a rather dated set of tools. Likewise, it is also common -to be provided with a patched copy of the Linux kernel from the board or chip vendor, -but this is also dated and it can be difficult to spot exactly what has been -modified to make the kernel run on the embedded platform. - -\subsection{Building an image} - -OpenWrt takes a different approach to building a firmware; downloading, patching -and compiling everything from scratch, including the cross compiler. To put it -in simpler terms, OpenWrt does not contain any executables or even sources, it is an -automated system for downloading the sources, patching them to work with the given -platform and compiling them correctly for that platform. What this means is that -just by changing the template, you can change any step in the process. - -As an example, if a new kernel is released, a simple change to one of the Makefiles -will download the latest kernel, patch it to run on the embedded platform and produce -a new firmware image -- there is no work to be done trying to track down an unmodified -copy of the existing kernel to see what changes had been made, the patches are -already provided and the process ends up almost completely transparent. This does not -just apply to the kernel, but to anything included with OpenWrt -- It is this one -simple understated concept which is what allows OpenWrt to stay on the bleeding edge -with the latest compilers, latest kernels and latest applications. - -So let's take a look at OpenWrt and see how this all works. - - -\subsubsection{Download OpenWrt} - -OpenWrt can be downloaded via subversion using the following command: - -\begin{Verbatim} -$ svn checkout svn://svn.openwrt.org/openwrt/trunk openwrt-trunk -\end{Verbatim} - -Additionally, there is a trac interface on \href{https://dev.openwrt.org/}{https://dev.openwrt.org/} -which can be used to monitor svn commits and browse the source repository. - - -\subsubsection{The directory structure} - -There are four key directories in the base: - -\begin{itemize} - \item \texttt{tools} - \item \texttt{toolchain} - \item \texttt{package} - \item \texttt{target} -\end{itemize} - -\texttt{tools} and \texttt{toolchain} refer to common tools which will be -used to build the firmware image, the compiler, and the C library. -The result of this is three new directories, \texttt{build\_dir/host}, which is a temporary -directory for building the target independent tools, \texttt{build\_dir/toolchain-\textit{}*} -which is used for building the toolchain for a specific architecture, and -\texttt{staging\_dir/toolchain-\textit{}*} where the resulting toolchain is installed. -You will not need to do anything with the toolchain directory unless you intend to -add a new version of one of the components above. - -\begin{itemize} - \item \texttt{build\_dir/host} - \item \texttt{build\_dir/toolchain-\textit{}*} -\end{itemize} - -\texttt{package} is for exactly that -- packages. In an OpenWrt firmware, almost everything -is an \texttt{.ipk}, a software package which can be added to the firmware to provide new -features or removed to save space. Note that packages are also maintained outside of the main -trunk and can be obtained from subversion using the package feeds system: - -\begin{Verbatim} -$ ./scripts/feeds update -\end{Verbatim} - -Those packages can be used to extend the functionality of the build system and need to be -symlinked into the main trunk. Once you do that, the packages will show up in the menu for -configuration. You would do something like this: - -\begin{Verbatim} -$ ./scripts/feeds search nmap -Search results in feed 'packages': -nmap Network exploration and/or security auditing utility - -$ ./scripts/feeds install nmap -\end{Verbatim} - -To include all packages, issue the following command: - -\begin{Verbatim} -$ make package/symlinks -\end{Verbatim} - -\texttt{target} refers to the embedded platform, this contains items which are specific to -a specific embedded platform. Of particular interest here is the "\texttt{target/linux}" -directory which is broken down by platform \textit{} and contains the patches to the -kernel, profile config, for a particular platform. There's also the "\texttt{target/image}" directory -which describes how to package a firmware for a specific platform. - -Both the target and package steps will use the directory "\texttt{build\_dir/\textit{}}" -as a temporary directory for compiling. Additionally, anything downloaded by the toolchain, -target or package steps will be placed in the "\texttt{dl}" directory. - -\begin{itemize} - \item \texttt{build\_dir/\textit{}} - \item \texttt{dl} -\end{itemize} - -\subsubsection{Building OpenWrt} - -While the OpenWrt build environment was intended mostly for developers, it also has to be -simple enough that an inexperienced end user can easily build his or her own customized firmware. - -Running the command "\texttt{make menuconfig}" will bring up OpenWrt's configuration menu -screen, through this menu you can select which platform you're targeting, which versions of -the toolchain you want to use to build and what packages you want to install into the -firmware image. Note that it will also check to make sure you have the basic dependencies for it -to run correctly. If that fails, you will need to install some more tools in your local environment -before you can begin. - -Similar to the linux kernel config, almost every option has three choices, -\texttt{y/m/n} which are represented as follows: - -\begin{itemize} - \item{\texttt{<*>} (pressing y)} \\ - This will be included in the firmware image - \item{\texttt{} (pressing m)} \\ - This will be compiled but not included (for later install) - \item{\texttt{< >} (pressing n)} \\ - This will not be compiled -\end{itemize} - -After you've finished with the menu configuration, exit and when prompted, save your -configuration changes. - -If you want, you can also modify the kernel config for the selected target system. -simply run "\texttt{make kernel\_menuconfig}" and the build system will unpack the kernel sources -(if necessary), run menuconfig inside of the kernel tree, and then copy the kernel config -to \texttt{target/linux/\textit{}/config} so that it is preserved over -"\texttt{make clean}" calls. - -To begin compiling the firmware, type "\texttt{make}". By default -OpenWrt will only display a high level overview of the compile process and not each individual -command. - -\subsubsection{Example:} - -\begin{Verbatim} -make[2] toolchain/install -make[3] -C toolchain install -make[2] target/compile -make[3] -C target compile -make[4] -C target/utils prepare - -[...] -\end{Verbatim} - -This makes it easier to monitor which step it's actually compiling and reduces the amount -of noise caused by the compile output. To see the full output, run the command -"\texttt{make V=99}". - -During the build process, buildroot will download all sources to the "\texttt{dl}" -directory and will start patching and compiling them in the "\texttt{build\_dir/\textit{}}" -directory. When finished, the resulting firmware will be in the "\texttt{bin}" directory -and packages will be in the "\texttt{bin/packages}" directory. - - -\subsection{Creating packages} - -One of the things that we've attempted to do with OpenWrt's template system is make it -incredibly easy to port software to OpenWrt. If you look at a typical package directory -in OpenWrt you'll find several things: - -\begin{itemize} - \item \texttt{package/\textit{}/Makefile} - \item \texttt{package/\textit{}/patches} - \item \texttt{package/\textit{}/files} -\end{itemize} - -The patches directory is optional and typically contains bug fixes or optimizations to -reduce the size of the executable. The package makefile is the important item, provides -the steps actually needed to download and compile the package. - -The files directory is also optional and typicall contains package specific startup scripts or default configuration files that can be used out of the box with OpenWrt. - -Looking at one of the package makefiles, you'd hardly recognize it as a makefile. -Through what can only be described as blatant disregard and abuse of the traditional -make format, the makefile has been transformed into an object oriented template which -simplifies the entire ordeal. - -Here for example, is \texttt{package/bridge/Makefile}: - -\begin{Verbatim}[frame=single,numbers=left] - -include $(TOPDIR)/rules.mk - -PKG_NAME:=bridge -PKG_VERSION:=1.0.6 -PKG_RELEASE:=1 - -PKG_SOURCE:=bridge-utils-$(PKG_VERSION).tar.gz -PKG_SOURCE_URL:=@SF/bridge -PKG_MD5SUM:=9b7dc52656f5cbec846a7ba3299f73bd -PKG_CAT:=zcat - -PKG_BUILD_DIR:=$(BUILD_DIR)/bridge-utils-$(PKG_VERSION) - -include $(INCLUDE_DIR)/package.mk - -define Package/bridge - SECTION:=net - CATEGORY:=Base system - TITLE:=Ethernet bridging configuration utility - URL:=http://bridge.sourceforge.net/ -endef - -define Package/bridge/description - Manage ethernet bridging: - a way to connect networks together to form a larger network. -endef - -define Build/Configure - $(call Build/Configure/Default, \ - --with-linux-headers="$(LINUX_DIR)" \ - ) -endef - -define Package/bridge/install - $(INSTALL_DIR) $(1)/usr/sbin - $(INSTALL_BIN) $(PKG_BUILD_DIR)/brctl/brctl $(1)/usr/sbin/ -endef - -$(eval $(call BuildPackage,bridge)) -\end{Verbatim} - -As you can see, there's not much work to be done; everything is hidden in other makefiles -and abstracted to the point where you only need to specify a few variables. - -\begin{itemize} - \item \texttt{PKG\_NAME} \\ - The name of the package, as seen via menuconfig and ipkg - \item \texttt{PKG\_VERSION} \\ - The upstream version number that we are downloading - \item \texttt{PKG\_RELEASE} \\ - The version of this package Makefile - \item \texttt{PKG\_SOURCE} \\ - The filename of the original sources - \item \texttt{PKG\_SOURCE\_URL} \\ - Where to download the sources from (no trailing slash), you can add multiple download sources by separating them with a \\ and a carriage return. - \item \texttt{PKG\_MD5SUM} \\ - A checksum to validate the download - \item \texttt{PKG\_CAT} \\ - How to decompress the sources (zcat, bzcat, unzip) - \item \texttt{PKG\_BUILD\_DIR} \\ - Where to compile the package -\end{itemize} - -The \texttt{PKG\_*} variables define where to download the package from; -\texttt{@SF} is a special keyword for downloading packages from sourceforge. There is also -another keyword of \texttt{@GNU} for grabbing GNU source releases. If any of the above mentionned download source fails, the OpenWrt mirrors will be used as source. - -The md5sum (if present) is used to verify the package was downloaded correctly and -\texttt{PKG\_BUILD\_DIR} defines where to find the package after the sources are -uncompressed into \texttt{\$(BUILD\_DIR)}. - -At the bottom of the file is where the real magic happens, "BuildPackage" is a macro -set up by the earlier include statements. BuildPackage only takes one argument directly -- -the name of the package to be built, in this case "\texttt{bridge}". All other information -is taken from the define blocks. This is a way of providing a level of verbosity, it's -inherently clear what the contents of the \texttt{description} template in -\texttt{Package/bridge} is, which wouldn't be the case if we passed this information -directly as the Nth argument to \texttt{BuildPackage}. - -\texttt{BuildPackage} uses the following defines: - -\textbf{\texttt{Package/\textit{}}:} \\ - \texttt{\textit{}} matches the argument passed to buildroot, this describes - the package the menuconfig and ipkg entries. Within \texttt{Package/\textit{}} - you can define the following variables: - - \begin{itemize} - \item \texttt{SECTION} \\ - The section of package (currently unused) - \item \texttt{CATEGORY} \\ - Which menu it appears in menuconfig: Network, Sound, Utilities, Multimedia ... - \item \texttt{TITLE} \\ - A short description of the package - \item \texttt{URL} \\ - Where to find the original software - \item \texttt{MAINTAINER} (optional) \\ - Who to contact concerning the package - \item \texttt{DEPENDS} (optional) \\ - Which packages must be built/installed before this package. To reference a dependency defined in the - same Makefile, use \textit{}. If defined as an external package, use - \textit{+}. For a kernel version dependency use: \textit{@LINUX\_2\_} - \item \texttt{BUILDONLY} (optional) \\ - Set this option to 1 if you do NOT want your package to appear in menuconfig. - This is useful for packages which are only used as build dependencies. - \end{itemize} - -\textbf{\texttt{Package/\textit{}/conffiles} (optional):} \\ - A list of config files installed by this package, one file per line. - -\textbf{\texttt{Build/Prepare} (optional):} \\ - A set of commands to unpack and patch the sources. You may safely leave this - undefined. - -\textbf{\texttt{Build/Configure} (optional):} \\ - You can leave this undefined if the source doesn't use configure or has a - normal config script, otherwise you can put your own commands here or use - "\texttt{\$(call Build/Configure/Default,\textit{})}" as above to - pass in additional arguments for a standard configure script. The first list of arguments will be passed - to the configure script like that: \texttt{--arg 1} \texttt{--arg 2}. The second list contains arguments that should be - defined before running the configure script such as autoconf or compiler specific variables. - - To make it easier to modify the configure command line, you can either extend or completely override the following variables: - \begin{itemize} - \item \texttt{CONFIGURE\_ARGS} \\ - Contains all command line arguments (format: \texttt{--arg 1} \texttt{--arg 2}) - \item \texttt{CONFIGURE\_VARS} \\ - Contains all environment variables that are passed to ./configure (format: \texttt{NAME="value"}) - \end{itemize} - -\textbf{\texttt{Build/Compile} (optional):} \\ - How to compile the source; in most cases you should leave this undefined. - - As with \texttt{Build/Configure} there are two variables that allow you to override - the make command line environment variables and flags: - \begin{itemize} - \item \texttt{MAKE\_FLAGS} \\ - Contains all command line arguments (typically variable overrides like \texttt{NAME="value"} - \item \texttt{MAKE\_VARS} \\ - Contains all environment variables that are passed to the make command - \end{itemize} - -\textbf{\texttt{Build/InstallDev} (optional):} \\ - If your package provides a library that needs to be made available to other packages, - you can use the \texttt{Build/InstallDev} template to copy it into the staging directory - which is used to collect all files that other packages might depend on at build time. - When it is called by the build system, two parameters are passed to it. \texttt{\$(1)} points to - the regular staging dir, typically \texttt{staging\_dir/\textit{ARCH}}, while \texttt{\$(2)} points - to \texttt{staging\_dir/host}. The host staging dir is only used for binaries, which are - to be executed or linked against on the host and its \texttt{bin/} subdirectory is included - in the \texttt{PATH} which is passed down to the build system processes. - Please use \texttt{\$(1)} and \texttt{\$(2)} here instead of the build system variables - \texttt{\$(STAGING\_DIR)} and \texttt{\$(STAGING\_DIR\_HOST)}, because the build system behavior - when staging libraries might change in the future to include automatic uninstallation. - -\textbf{\texttt{Package/\textit{}/install}:} \\ - A set of commands to copy files out of the compiled source and into the ipkg - which is represented by the \texttt{\$(1)} directory. Note that there are currently - 4 defined install macros: - \begin{itemize} - \item \texttt{INSTALL\_DIR} \\ - install -d -m0755 - \item \texttt{INSTALL\_BIN} \\ - install -m0755 - \item \texttt{INSTALL\_DATA} \\ - install -m0644 - \item \texttt{INSTALL\_CONF} \\ - install -m0600 - \end{itemize} - -The reason that some of the defines are prefixed by "\texttt{Package/\textit{}}" -and others are simply "\texttt{Build}" is because of the possibility of generating -multiple packages from a single source. OpenWrt works under the assumption of one -source per package Makefile, but you can split that source into as many packages as -desired. Since you only need to compile the sources once, there's one global set of -"\texttt{Build}" defines, but you can add as many "Package/" defines as you want -by adding extra calls to \texttt{BuildPackage} -- see the dropbear package for an example. - -After you have created your \texttt{package/\textit{}/Makefile}, the new package -will automatically show in the menu the next time you run "make menuconfig" and if selected -will be built automatically the next time "\texttt{make}" is run. - -\subsection{Creating binary packages} - -You might want to create binary packages and include them in the resulting images as packages. -To do so, you can use the following template, which basically sets to nothing the Configure and -Compile templates. - -\begin{Verbatim}[frame=single,numbers=left] - -include $(TOPDIR)/rules.mk - -PKG_NAME:=binpkg -PKG_VERSION:=1.0 -PKG_RELEASE:=1 - -PKG_SOURCE:=binpkg-$(PKG_VERSION).tar.gz -PKG_SOURCE_URL:=http://server -PKG_MD5SUM:=9b7dc52656f5cbec846a7ba3299f73bd -PKG_CAT:=zcat - -include $(INCLUDE_DIR)/package.mk - -define Package/binpkg - SECTION:=net - CATEGORY:=Network - TITLE:=Binary package -endef - -define Package/bridge/description - Binary package -endef - -define Build/Configure -endef - -define Build/Compile -endef - -define Package/bridge/install - $(INSTALL_DIR) $(1)/usr/sbin - $(INSTALL_BIN) $(PKG_BUILD_DIR)/* $(1)/usr/sbin/ -endef - -$(eval $(call BuildPackage,bridge)) -\end{Verbatim} - -Provided that the tarball which contains the binaries reflects the final -directory layout (/usr, /lib ...), it becomes very easy to get your package -look like one build from sources. - -Note that using the same technique, you can easily create binary pcakages -for your proprietary kernel modules as well. - -\subsection{Creating kernel modules packages} - -The OpenWrt distribution makes the distinction between two kind of kernel modules, those coming along with the mainline kernel, and the others available as a separate project. We will see later that a common template is used for both of them. - -For kernel modules that are part of the mainline kernel source, the makefiles are located in \textit{package/kernel/modules/*.mk} and they appear under the section "Kernel modules" - -For external kernel modules, you can add them to the build system just like if they were software packages by defining a KernelPackage section in the package makefile. - -Here for instance the Makefile for the I2C subsytem kernel modules : - -\begin{Verbatim}[frame=single,numbers=left] - -I2CMENU:=I2C Bus - -define KernelPackage/i2c-core - TITLE:=I2C support - DESCRIPTION:=Kernel modules for i2c support - SUBMENU:=$(I2CMENU) - KCONFIG:=CONFIG_I2C_CORE CONFIG_I2C_DEV - FILES:=$(MODULES_DIR)/kernel/drivers/i2c/*.$(LINUX_KMOD_SUFFIX) - AUTOLOAD:=$(call AutoLoad,50,i2c-core i2c-dev) -endef -$(eval $(call KernelPackage,i2c-core)) -\end{Verbatim} - -To group kernel modules under a common description in menuconfig, you might want to define a \textit{MENU} variable on top of the kernel modules makefile. - -\begin{itemize} - \item \texttt{TITLE} \\ - The name of the module as seen via menuconfig - \item \texttt{DESCRIPTION} \\ - The description as seen via help in menuconfig - \item \texttt{SUBMENU} \\ - The sub menu under which this package will be seen - \item \texttt{KCONFIG} \\ - Kernel configuration option dependency. For external modules, remove it. - \item \texttt{FILES} \\ - Files you want to inlude to this kernel module package, separate with spaces. - \item \texttt{AUTOLOAD} \\ - Modules that will be loaded automatically on boot, the order you write them is the order they would be loaded. -\end{itemize} - -After you have created your \texttt{package/kernel/modules/\textit{}.mk}, the new kernel modules package -will automatically show in the menu under "Kernel modules" next time you run "make menuconfig" and if selected -will be built automatically the next time "\texttt{make}" is run. - -\subsection{Conventions} - -There are a couple conventions to follow regarding packages: - -\begin{itemize} - \item \texttt{files} - \begin{enumerate} - \item configuration files follow the convention \\ - \texttt{\textit{}.conf} - \item init files follow the convention \\ - \texttt{\textit{}.init} - \end{enumerate} - \item \texttt{patches} - \begin{enumerate} - \item patches are numerically prefixed and named related to what they do - \end{enumerate} -\end{itemize} - -\subsection{Troubleshooting} - -If you find your package doesn't show up in menuconfig, try the following command to -see if you get the correct description: - -\begin{Verbatim} - TOPDIR=$PWD make -C package/ DUMP=1 V=99 -\end{Verbatim} - -If you're just having trouble getting your package to compile, there's a few -shortcuts you can take. Instead of waiting for make to get to your package, you can -run one of the following: - -\begin{itemize} - \item \texttt{make package/\textit{}/clean V=99} - \item \texttt{make package/\textit{}/install V=99} -\end{itemize} - -Another nice trick is that if the source directory under \texttt{build\_dir/\textit{}} -is newer than the package directory, it won't clobber it by unpacking the sources again. -If you were working on a patch you could simply edit the sources under the -\texttt{build\_dir/\textit{}/\textit{}} directory and run the install command above, -when satisfied, copy the patched sources elsewhere and diff them with the unpatched -sources. A warning though - if you go modify anything under \texttt{package/\textit{}} -it will remove the old sources and unpack a fresh copy. - -Other useful targets include: - -\begin{itemize} - \item \texttt{make package/\textit{}/prepare V=99} - \item \texttt{make package/\textit{}/compile V=99} - \item \texttt{make package/\textit{}/configure V=99} -\end{itemize} - - -\subsection{Using build environments} -OpenWrt provides a means of building images for multiple configurations -which can use multiple targets in one single checkout. These \emph{environments} -store a copy of the .config file generated by \texttt{make menuconfig} and the contents -of the \texttt{./files} folder. -The script \texttt{./scripts/env} is used to manage these environments, it uses -\texttt{git} (which needs to be installed on your system) as backend for version control. - -The command -\begin{Verbatim} - ./scripts/env help -\end{Verbatim} -produces a short help text with a list of commands. - -To create a new environment named \texttt{current}, run the following command -\begin{Verbatim} - ./scripts/env new current -\end{Verbatim} -This will move your \texttt{.config} file and \texttt{./files} (if it exists) to -the \texttt{env/} subdirectory and create symlinks in the base folder. - -After running make menuconfig or changing things in files/, your current state will -differ from what has been saved before. To show these changes, use: -\begin{Verbatim} - ./scripts/env diff -\end{Verbatim} - -If you want to save these changes, run: -\begin{Verbatim} - ./scripts/env save -\end{Verbatim} -If you want to revert your changes to the previously saved copy, run: -\begin{Verbatim} - ./scripts/env revert -\end{Verbatim} - -If you want, you can now create a second environment using the \texttt{new} command. -It will ask you whether you want to make it a clone of the current environment (e.g. -for minor changes) or if you want to start with a clean version (e.g. for selecting -a new target). - -To switch to a different environment (e.g. \texttt{test1}), use: -\begin{Verbatim} - ./scripts/env switch test1 -\end{Verbatim} - -To rename the current branch to a new name (e.g. \texttt{test2}), use: -\begin{Verbatim} - ./scripts/env rename test2 -\end{Verbatim} - -If you want to get rid of environment switching and keep everything in the base directory -again, use: -\begin{Verbatim} - ./scripts/env clear -\end{Verbatim} diff --git a/docs/config.tex b/docs/config.tex deleted file mode 100644 index 08318b4b6c..0000000000 --- a/docs/config.tex +++ /dev/null @@ -1,101 +0,0 @@ -\subsubsection{Structure of the configuration files} - -The config files are divided into sections and options/values. - -Every section has a type, but does not necessarily have a name. -Every option has a name and a value and is assigned to the section -it was written under. - -Syntax: - -\begin{Verbatim} -config [""] # Section - option "" # Option -\end{Verbatim} - -Every parameter needs to be a single string and is formatted exactly -like a parameter for a shell function. The same rules for Quoting and -special characters also apply, as it is parsed by the shell. - -\subsubsection{Parsing configuration files in custom scripts} - -To be able to load configuration files, you need to include the common -functions with: - -\begin{Verbatim} -. /lib/functions.sh -\end{Verbatim} - -Then you can use \texttt{config\_load \textit{}} to load config files. The function -first checks for \textit{} as absolute filename and falls back to loading -it from \texttt{/etc/config} (which is the most common way of using it). - -If you want to use special callbacks for sections and/or options, you -need to define the following shell functions before running \texttt{config\_load} -(after including \texttt{/lib/functions.sh}): - -\begin{Verbatim} -config_cb() { - local type="$1" - local name="$2" - # commands to be run for every section -} - -option_cb() { - # commands to be run for every option -} -\end{Verbatim} - -You can also alter \texttt{option\_cb} from \texttt{config\_cb} based on the section type. -This allows you to process every single config section based on its type -individually. - -\texttt{config\_cb} is run every time a new section starts (before options are being -processed). You can access the last section through the \texttt{CONFIG\_SECTION} -variable. Also an extra call to \texttt{config\_cb} (without a new section) is generated -after \texttt{config\_load} is done. -That allows you to process sections both before and after all options were -processed. - -Another way of iterating on config sections is using the \texttt{config\_foreach} command. - -Syntax: -\begin{Verbatim} -config_foreach [] [] -\end{Verbatim} - -This command will run the supplied function for every single config section in the currently -loaded config. The section name will be passed to the function as argument 1. -If the section type is added to the command line, the function will only be called for -sections of the given type. - - -You can access already processed options with the \texttt{config\_get} command -Syntax: - -\begin{Verbatim} -# print the value of the option -config_get