Sortix nightly manual
This manual documents Sortix nightly, a development build that has not been officially released. You can instead view this document in the latest official manual.
PORTING(7) | Miscellaneous Information Manual | PORTING(7) |
NAME
porting
—
guide for porting software
SYNOPSIS
/src/ports/example/example.portDESCRIPTION
This manual documents how to port software to this operating system as packaged ports in the port(5) format.Philosophy
The ports collection is maintained according to a set of principles designed to uphold quality, keep the ports maintainable, and to enable giving back improvements to the upstream projects. Ports are usually named after their primary program or library if relevant. Libraries should always be prefixed with lib. The port should be ideally split if it happens to contain both a major program and a major library. Upstream projects should consider this operating system to be yet another unknown operating system implementing the standard interfaces. Explicit support for this operating system should generally not be upstreamed and should instead be maintained in the patches. This policy is to avoid burdening upstreams with maintaining support that is subject to change. Simply registering the operating system as existing is exempted, but preferably the software should be patched to compile for any unknown operating system that has the needed interfaces. Patches should ideally be of upstreamable quality, although it may not be reasonably practical to make a general solution the upstream could accept. Non-trivial patches should be prefaced with a comment explaining the rationale for the patch. Third party software should be selected into the ports collection with care, ideally because they're widely considered high quality and stable. Low quality software with poor code quality should preferably not be ported, instead a better replacement should be standardized on. The build system is expected to follow the relevant conventions, such as the GNU coding conventions for ./configure and Makefile scripts and the de-facto conventional behavior. Deficiencies are fixed via patches instead of being worked around. Ports with their own custom build system implementation should be improved if they don't implement the interface correctly. Generated files are preferably patched instead of their input files due to the difficulty regenerating them. For instance configure should be patched directly instead of configure.ac and likewise Makefile.in instead of Makefile.am. Ports must not bundle their dependencies as it's problematic to silently have multiple stale versions of the same library. If a port bundles a dependency unless it's installed, then the dependency must be made mandatory. Ports must both compile natively and cross-compile. Ports must assume the best about the host operating system when cross-compiling and unable to test for bugs. It must be possible to reasonably boostrap the latest ports collection from the latest stable release.Metadata
A port named example can be built along with the operating system by creating the /src/ports/example/example.port file with the meta information documented in port(5). Start the port by defining how to get the latest stable version of the upstream release:NAME=example BUILD_LIBRARIES='libfoo libbar? libqux?' VERSION=1.2.3 DISTNAME=$NAME-$VERSION COMPRESSION=tar.gz ARCHIVE=$DISTNAME.$COMPRESSION SHA256SUM=b8d67e37894726413f0d180b2c5a8369b02d4a2961bb50d2a8ab6f553f430976 UPSTREAM_SITE=https://example.com/pub/example UPSTREAM_ARCHIVE=$ARCHIVE BUILD_SYSTEM=configure LICENSE=example DEVELOPMENT=true
Building
The port can be finished by repeatedly trying to build it and iteratively fixing any issues that occur per development(7):cd /src make clean-sysroot make PACKAGES='example!'
rm -f repository/*/example.tix.tar.xz # delete stale binary package rm -f repository/*/example.version # optional make clean-repository # or: remove all binary packages
Finishing
The port can be finished by testing it thoroughly and reviewing the details. Test the port works with all the optional dependencies:make clean-sysroot make PACKAGES='example!!'
du -h repository/*/example.tix.tar.xz tar -t -f repository/*/example.tix.tar.xz
Upgrading ports
Ports can be upgraded by switching the port back in development mode with a updated version number and checksum.make available-ports # Search for newly available versions of ports make upgrade-ports PACKAGES=example # Update example port make PACKAGES='example!!' # Find the new sha256sum # Verify the new sha256sum is authentic. make PACKAGES='example!!' # Any old patches may fail to apply. find ports/example -name '*.rej' -o -name '*.orig' make PACKAGES='example!!' # Regenerate patches on a successful build. sed -E '/^DEVELOPMENT=true$/d' ports/example/example.port make PACKAGES='example!!' # Final build and testing.
PACKAGES
environment variable can be set to
a subset of ports to upgrade. The build will fail and report the sha256sum
observed upstream and the SHA256SUM variable must
be manually updated once it has been verified as as authentic.
The old patch will be reapplied and the build will fail if any hunks could not
be applied. In that case, the .rej files
contains the rejected hunks and the merge conflicts need to be resolved.
Delete any .rej and
.orig files afterwards and continue the
build. The patch files with be regenerated on a successful build as usual.
Finally perform the quality assurance and testing of new ports.
EXAMPLES
This section describes common situations and portability issues and how to resolve them.config.sub
The config.sub file parses the build/host/target triplet and is duplicated into all software using autoconf. If the port ships a version older than 2015-08-20, configure will fail to parse the operating system:checking host system type... Invalid configuration `x86_64-sortix': system `sortix' not recognized
| -aos* | -aros* | -cloudabi* | -sortix* \
Configure
If the configure script fails, then autoconf configure scripts produce a config.log file which contain useful diagnostic information, such as which programs were compiled to check for a particular feature and what happened when they were compiled, linked, and executed. This information can be used to locate the relevant logic in the configure script.Non-verbose make
Some ports default to a non-verbose make mode that doesn't show the commands being run. Ports should show the commands invoked by make, which can done with an environment variable assignmentV=1
which can be made permanent using MAKE_VARS=V=1.
Patching
The build may fail or warn for various reasons which needs to be patched, or the port may need extra support for the operating system. The sources become writable once the port is in development mode and can simply be edited to fix the problem. The patch files are regenerated whenever the port builds successfully. Non-trivial patches should contain a “// PATCH:” comment explaining why the patch had to be made and make it clear whether the patch fixes a problem in the upstream release or is specific to the operating system. Patches may be read by many people, including upstream developers, and the context helps them understand if they want the patch too. Ports should ideally continue to work on good operating systems after being patched. If the port uses an obsolete function, the port should be patched to unconditionally use the modern replacement instead. If the patch is specific to the operating system, the patch should be guarded with the operating system preprocessor macro.// PATCH: Use foo instead because bar doesn't work yet. #ifdef __sortix__ foo(); #else bar(); #endif
#if __has_include(<foo.h>) #include <foo.h> #endif
Configuration files
Configuration files in /etc belongs to the system administrator and must not be installed by ports, as local changes will otherwise be undone whenever the port is upgraded. Ports shipping default configuration files should instead install them in /etc/default and search this directory as a fallback if the system administrator has not made their own file. Example configuration files can be installed in /etc/example.Portability issues
The port might not be portable and requires patching. Often the port uses a non-standard interface when a standard interface is available and should be used instead. Other times the operating system is missing functionality which should ideally be implemented before proceeding with the port, or perhaps the absence can be worked around. Common portability problems and their resolution are documented in portability(7).libtool .la files
Libraries using libtool may install /lib/*.la files which are unnecessary and contain absolute paths which cause cross-compilation to fail. Ports should never install such files and instead rely on pkg-config(1) for locating dependencies. The tix-eradicate-libtool-la(1) program can be used to remove any installed .la files in the DESTDIR as a post-install script:POST_INSTALL=tix-eradicate-libtool-la
Post-install command
Ports may install files with an unconventional directory layout or may install unnecessary large files. Ideally the makefile should be patched to install correctly in the first place, but if doing so is impractical, then a post-install command can be used to fix up the installation in the DESTDIR before the binary package is created. An executable example.post-install script can be placed next to the example.port files:POST_INSTALL=../example.post-install
#!/bin/sh set -e [ -n "$TIX_INSTALL_DIR" ] mv "$TIX_INSTALL_DIR$PREFIX/share/foo" "$TIX_INSTALL_DIR$PREFIX/share/bar"
Splitting into multiple ports
Upstream releases might have multiple parts, such as a program that also comes with a library, which should be split into multiple ports whenever reasonable. The port whose name matches the upstream release should be the source package, and the other ports can use the SOURCE_PORT variable to reuse its source code. The SUBDIR variable can be used to build inside a subdirectory, if the port has already been logically organized. CONFIGURE_ARGS variable can be used to pass options to configure to request the subset of functionality. MAKE_BUILD_TARGET and MAKE_INSTALL_TARGET can be used to make a subset of the port.Gnulib
Gnulib is a GNU portability layer that has three purposes:- Providing fallback implementations of missing standard library interfaces.
- Replacing buggy implementations of standard library interfaces.
- Sharing common utility functions between projects.
- The replacement implementations tend to be non-portable and one is supposed to modify the source code to rely on private standard library details that may be subject to change.
- Gnulib is highly forked by design and adding support for new operating systems needs to be done upstream and it may take years for the support to reach new downstream releases.
- Gnulib has assumed the worst in the past when cross-compiling, assuming unknown operating systems are buggy, injecting a non-portable replacement that doesn't compile, even when the standard library function is correct and could just have been used.
- Replacing standard library functions can hide bugs that would otherwise have found and fixed.
- Gnulib is not organized into the three categories and it's non-trivial to find out whether any interface has been replaced that shouldn't have been.
- There is no way to satisfy gnulib by correctly implementing the standard library without contributing explicit support upstream for new systems and committing to private implementation details.
Custom configure script
Configure scripts generated by autoconf are useful because they have a consistent interface. Ports with a hand-written configure script can fail if they fail to implement the autoconf configure interface correctly. The best solution is modify the configure script to implement the missing parts of the interface. Custom configure scripts sometimes fail to implement the--prefix
and
--exec-prefix
options correctly, failing to
discern between an unset option and the empty value, which matters for the
prefix where the default prefix
(/usr/local) is different from the empty
prefix (the root directory).
Custom configure scripts sometimes fail to implement cross-compilation using the
--build
,
--host
, and
--target
options to locate the compiler and
cross-compiler.
Makefile
Ports might not have a configure script and only a makefile:BUILD_SYSTEM=makefile
Running cross-compiled program
Ports may contain logic errors and attempt to execute a cross-compiled program during the build, which fails because the program can't run on the current system. If the program is supposed to check whether a feature works at runtime, then the right solution is to simply assume it's correct when cross-compiling or to have a suitable runtime fallback. If the program is part of the compilation process, then it should be compiled with the build machine's compiler instead, which can be located using theCC_FOR_BUILD
environment variable instead
of CC
before falling back to a standard
program. Likewise each toolchain variable has a counterpart for the build
machine instead of the host machine, such as
CFLAGS_FOR_BUILD
and
CPPFLAGS_FOR_BUILD
.
pkg-config
Ports are supposed to locate locate dependencies for the host machine using thePKG_CONFIG
environment variable, and if
unset, then using the --host
option to
locate a cross-pkg-config, and finally falling back on invoking
pkg-config(1)
directly . Dependencies for the build machine should likewise be located using
the PKG_CONFIG_FOR_BUILD
environment
variable.
Ports failing to use this search order may fail to cross-compile as they
accidentally use dependencies from the build machine when cross-compiling to
the host machine. The easiest fix is to use a shell parameter expansion:
${PKG_CONFIG:-pkg-config} ${PKG_CONFIG_FOR_BUILD:-${PKG_CONFIG:-pkg-config}}
foo-config instead of pkg-config
Ports are supposed to use pkg-config(1) as above for dependencies as it's a general solution with cross-compilation support, but some ports install their own foo-config program in thePATH
. These programs are inherently unable
to support cross-compilation, as they provide answers about the build machine,
and the host machine's bin directory cannot be executed on the current
machine.
Ports installing foo-config programs must be
patched to not install them.
pkg-config(1)
configuration files should be installed instead.
Ports invoking foo-config programs, even as a
fallback, must be patched to unconditionally use
pkg-config(1)
instead.
Bootstrap
Ports may require the same version of the port to be installed on the build machine when cross-compiling. Such ports can cross-compiled with a bootstrap phase:USE_BOOTSTRAP=true
PATH
while the port is cross-compiled
during the second phase.
DESTDIR
The makefile install target must support theDESTDIR
environment variable as a secondary
temporary prefix during the installation. The makefile may need to be patched
to inherit the variable from the environment and to use it during the
installation phase. The build will erroneously attempt to install onto the
root directory of the build machine if the environment variable isn't
respected.
make distclean
The upstream release is to supposed to be distclean, i.e. not contain any files that are recreated during the compilation, and the distclean makefile target is supposed to properly clean up. If it isn't, then the patch may contain spurious hunks adding, removing, or modifying generated files which may be large. The makefile distclean target should be patched to delete any temporary files produced during the build to avoid spurious files in the patch.Dynamic linking
Dynamic linking is not currently implemented in the standard library and the compiler toolchain does not support dynamic linking. Ports with libraries should check whether the toolchain supports dynamic linking and otherwise fall back on static linking. Ports may be difficult if they use shared libraries for modules. It may be possible to link the modules statically instead using a supported mechanism or with additional patching. If there is no simple solution, it may be possible to statically link the modules and implement a fake dlopen that iterates a table of entry points for each module.Other problems
Portability issues are very varied and ports often don't properly implement the conventional interface. portability(7) lists common differences in the standard library. port(5) documents the advanced features useful for certain situations. Ports may fail at runtime instead of during compilation. Resolving these issues can require troubleshooting, debugging, research, and seeking help from the operating system developers and the upstream. Missing operating system functionality may need to be implemented.SEE ALSO
port(5), development(7), portability(7)March 19, 2022 | Debian |