
Building Open Watcom
====================

This document describes how to build Open Watcom. The Quick Start is an overview and may be
sufficient information for some developers. The following sections give additional details
describing some options provided by the build system.

Quick Start
-----------

If you haven't done so already you should read the file `README.md`. That document gives an
overview of the project and a quick description of the layout of the Open Watcom source tree.

You will first need to install a few tools to build Open Watcom (without Documentation).
- build compiler (toolchain)
- DOS emulator (to run wgml utility)

If you want to build the full documentation, then other additional tools must be installed.
- Microsoft Help Compiler
- Microsoft Html Help Compiler
- Ghost script

Because the Microsoft Help Compiler is no longer available from Microsoft, a copy of its
installation package is included in the Open Watcom git repository in build/external subdirectory.

Nominally we use Open Watcom as our build compiler, but other compilers are possible and even
necessary when targeting 64-bit code (Open Watcom currently can't generate 64-bit executables).

### Supported compilers
| build OS | building OW for 16/32-bit hosts | building OW for 64-bit hosts | prerequisite software | Notes |
| --- | --- | --- | --- | --- |
| DOS | OW | - | - | - |
| OS/2 | OW | - | - | - |
| Windows 32-bit | OW | - | - | - |
| Windows 64-bit | OW | VC++, Intel | DOSBOX | Using 32-bit OW on the 64-bit OS |
| Linux x86 32-bit | OW | - | DOSBOX | - |
| Linux x64 64-bit | OW | gcc, clang | DOSBOX | Using 32-bit OW on the 64-bit OS |
| POSIX(only build) | gcc, clang | gcc, clang | DOSBOX | Experimental, not supported |

***
Note for GCC and Clang toolchains

If you build OW 64-bit executables with gcc or clang toolchain, you must install static C and
static C++ libraries.
***

Building the documentation requires additional tools some of which may not be readily available
for all platforms. See [`docs/howto.txt`](https://raw.githubusercontent.com/open-watcom/open-watcom-v2/master/docs/howto.txt)
for more information.

Start by modifying your copy of `setvars.bat` (DOS/Windows), `setvars.cmd` (OS/2), or `setvars.sh` 
(Linux) to reflect your system. Copy the appropriate `setvars` script outside OW directory tree and 
rename it to something useful (you can have multiple files with different configurations). See 
the comments in the file for additional information. Of particular interest is the `OWROOT` variable 
that specifies the location of the OW source tree, and the `OWTOOLS` variable that specifies which 
build compiler you intend to use.

In addition, you need to configure your compiler's environment. This can typically be accomplished 
by calling an appropriate batch file or shell script. For example, Open Watcom comes with a script 
`owsetenv` that specifies how the path and other environment variables should be set. 
Visual C++ comes with similar batch files. You can add a call to such a script at the beginning of
your copy of `setvars`.

*WARNING*: Some platforms (notably Windows 9x) require DOS style line endings in batch files.

The build process consists of two phases. The first phase creates a minimal set of Open Watcom
tools which are sufficient to build the entire system. The second phase builds all of Open
Watcom using the minimal set of tools built during the first phase.

The build and clean-up processes are handled by following scripts.

1. `build.bat` (DOS/Windows), `build.cmd` (OS/2), or `build.sh` (Linux). This script builds all
    the software by executing both build phases. Normally it is sufficient to run this script to
    build the system. More control can be had by using the `builder` command directly as
    described below. A word of warning: running a full build may take upwards of two hours even
    on a fairly fast machine. There is a *lot* to build!

    You may want to run `builder cprel` from inside the `bld` directory after successfully
    building everything to copy the complete system to the release tree `rel`.
    
    If you want to build and then copy binaries to the release tree `rel` in a single step,
    you can run `build rel` command instead of `build` command.

2. `clean.bat` (DOS/Windows), `clean.cmd` (OS/2), or `clean.sh` (Linux). This script erases all
    the object files, executable files, etc. created by any part of build process so you can
    build everything from scratch.

It is also possible to restore your code base to a pristine state using

    git clean -dfx

This command removes all files and directories unknown to git, including those that are
explicitly ignored. It should leave your code base in exactly the same state as it had after a
fresh clone operation.

Testing
-------

After building the system it is useful to test the result. This can be done by making a copy of
your modified `setvars` script and changing the `WATCOM` variable to point at the location of
the new system, typically `rel` in your source tree. Configure a console or shell with this new
`setvars` and then do

    builder test

in the `bld` directory.

For testing during development, it is possible to run tests without needing to do any installation.
To run a test do

    builder buildtest

in the `bld` directory or in individual project test directory (by example `bld/ctest` for C compiler).
Running the command in the `bld` directory will test the entire system.

Note that during testing you may see some error messages. That is not necessarily a problem
since some of the tests exercise the tools' ability to detect errors. Failed tests are reported
at the end of testing or if the testing process aborts prematurely.

More Details
------------

bootstrap:<br>
To bootstrap the OpenWatcom build, the `build.sh|build.bat|build.cmd` script in the top directory
starts building the `wmake` tool (with the host compiler and the host make tool), followed by
building the `builder` tool (with the host compiler and the fresh created `wmake` tool).
(A logfile for the bootstrap part is created at `<OWROOT>/<OWOBJDIR>/bootx.log`)
Then the `builder` tool is called for building all other parts of OpenWatcom.


builder:<br>
At the top level there is a tool that oversees traversing the build tree (`bld`), deciding which
projects to build for what platforms, logging the results to a file, and copying the finished
software into the release tree (`rel`), making fully automated builds a possibility. This tool
is called `builder`. See [`Builder DOC (bld/builder/builder.doc)`](https://raw.githubusercontent.com/open-watcom/open-watcom-v2/master/bld/builder/builder.doc) for detailed info on the tool and the source if the documentation doesn't satisfy you.

Each project has a `builder.ctl` builder script file. If you go to a project directory and run
`builder`, it will make only that project; if you go to `bld` and run `builder`, it will build
everything. The overall build uses `bld/builder.ctl` which includes all of the individual
project `builder.ctl` files. Note that `builder` will traverse directories upward until it finds
a `builder.ctl`.

The results of the build are logged to `build.log` in the current project directory (or `bld`),
the previous `build.log` file is copied to `build.lo1`.

Common builder commands:
```
builder boot
  - First build phase (bootstrap) only. Creates all tools necessary
    for building the entire Open Watcom system (phase two).

builder bootclean
  - Erases objects and tools, created during first build phase.

builder build
  - Second build phase. Builds the entire system.

builder cprel
  - Copies the system into the release tree (`rel`).

builder rel
  - Second build phase. Builds the entire system and copies it into
    the release tree (`rel`). This is equivalent to "builder build"
    followed by "builder cprel."

builder clean
  - Erase object files, executable files, etc. created during second
    build phase so you can do second build phase from scratch.

builder docs
  - Build all documentation.

builder docsclean
  - Erases all objects, etc. created during documentation build so you
    can start documentation build from scratch.

builder test
  - Run all automated tests on release tree (`rel`).

builder buildtest
  - Run automated tests on build tree (`bld`). This can be used for
    individual project testing during development.
```
Many of the projects use the `pmake` features of builder (see `builder.doc`) to determine what
to build. The `pmake` source is in `bld/pmake`.

Each `makefile` has a comment line at the top of the file which is read by `pmake`. Most of our
`builder.ctl` files will have a line similar to this:

    pmake -d build -h ...

this will cause `wmake` to be run in every subdirectory where the `makefile` contains `build` on
the `#pmake` line.

You can also specify more parameters to build a smaller subset of files. This is especially
useful if you do not have all required tools, headers, or libraries for all target platforms.

For example:

    builder rel os_nt

will build only the NT (32 bit Windows) version of the tools.

It is generally possible to build specific binaries or libraries by going to their directory and
running `wmake`. For instance to build the OS/2 version of `wlink` you can go to `bld/wl/os2386`
and run `wmake` there (note that the process won't successfully finish unless several required
libraries are already built). Builder is useful for making full builds while running `wmake` in
the right spot is handy during development.

Build process control
---------------------

To control OW build process by user specific requirements, set of environment variables is used.
These variables should be set only if this feature is requrired. Value should be setup by default
to 1 or to specific value (by example path etc.).

Bellow is list of these environment variables with short description.
```
OWDEBUGBUILD
  - build system compile/link for debugging, it means that compiled code is not
    optimized, but contains all debugging info for debugger. It is useful for
    particular project with command "builder build OWDEBUGBUILD=1"

OWDEVBUILD
  - build system compile/link development version for debugging, it means that
    compiled code contains sone special code for developpers and it is not optimized,
    but contains all debugging info for debugger.
    It is useful for particular project with command "builder build OWDEVBUILD=1"

OWDEFAULT_WINDOWING
  - Default windowing support is included only in C run-time libraries for 16-bit
    Windows because 16-bit Windows doesn't have character console.
    Use this variable if you need C run-time library with default windowing support
    for 32-bit OS/2 and NT. It has sense only for old application using this
    feature on 32-bit OS/2 and NT OS. This stuff is completely untested.

OWNOWGML
  - during OW build it suppress to use of wgml utility (wgml utility requires DOS emulator)
    it also suppress OW documentation build

OWDOCBUILD
  - during OW build OW documentation is created

OWDOCQUIET
  - documentation build is quiet, no details are output even if build verbose
    output is on

OWDOSBOX
  - set path to installed DosBox emulator for run wgml command, use it on OS-es
    which don't have DOS emulator

OWRELROOT
  - set path where you want to copy OW build, by default it is "rel" subdirectory
    in OW root

OWTOOLROOT
  - set path to root for 3rd party licensed development SDK etc.

OWTOOLS
  - set toolchain used for OW build ( WATCOM, GCC, CLANG, INTEL, VISUALC )

OWVERBOSE
  - build system generate detail listing of all command with full command line

OWGUINOBUILD
  - skip build of tools GUI version during regular OW build
    only character mode tools are build

OWNOBUILD
  - contains list of projects which are skipped during regular OW build
```
Bellow is list of internal environment variables used for OW build configuration by scripts.
```
OWBLDVER
  - build OW version as number

OWBLDVERSTR
  - build OW version as text

OWDLLOBJDIR
  - project subdirectory where created DLL specific objects are placed

OWOBJDIR
  - project subdirectory where objects created by bootstrap phase are placed. No Space, no extension allowed.

OWROOT
  - path to root of OW tree

OWTOOLSVER
  - Version of used toolchain compiler. With OWTOOLS variable it
    identify exact type/version of compiler used for bootstrap build.
    It is initialized by OW build environment setup scripts.
```
Even More Details
-----------------

This information and much more is in the Open Watcom Developer's Guide. To produce a PostScript
version go to `docs/ps` and run

    wmake hbook=devguide

If you have everything set up correctly, you should end up with `devguide.ps` which you can
print or view.


Troubleshooting
-----------------

Use of `dosemu` is no more supported. Only `DOSBOX` is supported by OW build system.

Bellow are some empiricaly investigated solutions for some problems by changing default values in DOSBOX configuration file `~/.dosbox/dosbox-0.74.conf`
- If DOSBOX produces an `DRC64:Unhandled memory reference` error, try to change line from
`core=auto`
to
`core=normal`
- If DOSBOX doesn't return to Linux OS, try to change line from
`nosound=false`
to
`nosound=true`
or try to change line from
`cycles=auto`
to
`cycles=90`.

Step by step setup and build instructions for Linux
-----------------

install needed packages on **Fedora** (tested with Fedora 40 - should work exactly like this on older versions)

1. download & install Fedora (https://fedoraproject.org)
2. `sudo dnf update`
3. `sudo dnf groupinstall "Development Tools"` - for gcc and typical build tools
4. `sudo dnf install clang` - for clang (if wanted to build with clang instead of gcc)
5. `sudo dnf install git`
6. `sudo dnf install dosbox`
7. `sudo dnf install glibc-static`
8. `sudo dnf install libstdc++-static`

install needed packages on **Ubuntu** (tested with Ubuntu 24.10 - should work exactly like this on older versions)

1. download & install Ubuntu (https://ubuntu.com)
2. `sudo apt update`
3. `sudo apt install build-essential` - for gcc and typical build tools
4. `sudo apt install g++`
5. `sudo apt install clang` - for clang (if wanted to build with clang instead of gcc)
6. `sudo apt install dosbox`
7. `sudo apt install git`

**building OpenWatcom v2**

1. `cd ~/`
2. `mkdir my_ow_build`
3. `cd my_ow_build`
4. `git clone https://github.com/open-watcom/open-watcom-v2.git`
5. `cp open-watcom-v2/setvars.sh my_setvars.sh`
open `my_setvars.sh` in editor and change:
* `OWROOT` to the absolute path of `my_ow_build/open-watcom-v2`
example: `export OWROOT=/home/test/my_ow_build/open-watcom-v2`
* `OWTOOLS` (**only** if you want to build with clang - else gcc is used)
`export OWTOOLS=CLANG`
* `OWDOSBOX` remove comment (only if you need to build documentation - the documentation is created using a dos executable)
6. `. ./my_setvars.sh` - the point in front is needed to keep the environment
7. `./build.sh`
8. run tests etc.

