64-bit Unix Squeak

(Try the FAQ if you are unsure how '64-bit' relates to Squeak VMs and images.)

October 2009: A 64-bit image suitable for use with current Squeak VMs is at http://squeakvm.org/squeak64/sq64-dtl.zip.

November 2010: A 64-bit image with more recent VMMaker for experimentation with large 64-bit images is at http://squeakvm.org/squeak64/sq64-20101106-dtl.zip. The image with complete platform sources and generated C for Linux CMake build is at http://squeakvm.org/squeak64/sq64-20101106-dtl-sources-and-image.zip.

0. Availability

The most recent version of this document is available at http://squeakvm.org/squeak64/. (If the link is broken, it means this document is no longer relevant.)

VM source code, 32- and 64-bit images, and pre-compiled 32- and 64-bit VMs for a variety of 32- and 64-bit platforms are available at http://squeakvm.org/squeak64/dist3. (If the link is broken, or the directory empty, it probably means 64-bit support has been pulled into the binaries that are available through the usual Squeak distribution channels.)

0.1 Current distribution

http://squeakvm.org/squeak64/dist3 contains the following files: With all of the above filed in you should have images identical to those in the archives above, capable of generating both 32- and 64-bit VM sources and of being traced from a 32-bit image to a 64-bit image that will load and run.

0.2 Quick Start

The source tarball (temporarily) contains a handly one-stop Makefile. To build a 32-bit VM, type:
make WIDTH=32
To make a 64-bit VM, type:
make WIDTH=64
To do it the traditional way, use the --with-src= option to configure, like this:
mkdir bld
cd bld
../platforms/unix/config/configure --with-src=src32     # or src64 or src or whatever
Pretty much everything else is as described in the standard documentation.

1. Introduction

This is a "port" of Squeak to 64 bits, where "porting" involves two related (but quite separable) tasks:

(In the remainder of this document, the words "porting" and "ported" are meant to include both of these areas and imply making the code compatible with both 32- and 64-bit images. Note that in some cases, image-side work in the former area implies runtime support work in the latter area; this is especially true for plugins.)

Generally speaking, Dan Ingalls worked on the former while Ian Piumarta worked on the latter.

Keeping the two areas properly separated leads to a significant benefit: the Squeak VM can now be compiled four ways, to run all four combinations of 32- or 64-bit images on 32- or 64-bit hardware. Most importantly, this allows volunteers helping with the plugin "porting" effort to test both 32- and 64-bit VMs regardless of the kinds of hardware to which they have access. (While porting BitBlt and B2D it was actually quite satisfying to see precisely the same rendering glitches on both 64- and 32-bit hardware running a 64-bit VM. :-)

1.1. Status

There is still much work (and cleaning up) to be done. Here is a brief list of what we have so far:

The above has been tested on Mac OS X (PowerPC, G4 and G5) and GNU/Linux (Itanium and Pentium).

A significant side effect of this work is that the InterpreterSimulator has been fixed and extended to simulate both 32- and 64-bit images.

1.2. Remaining work

The single most important thing is: pull the 64-bit changes into the mainstream Squeak release. Before that can happen for a "stable" release, at least the following must be addressed:

Desirable, but not essential, are:

2. 64 bits the easy way

You can download a 64-bit image and a precompiled binary for Mac OS X (PowerPC) or Linux (Itanium or Pentium) from the site mentioned in Section 0.

3. 64 bits the hard way

The "honest" way to obtain a 64-bit image and VM is to build them yourself. Again, everything you need is available from the site mentioned in Section 0.

3.1. Generating a 64-bit image

You will need a 3.7 image (the following has been tested and is known to work with the Squeak3.7-5989-full image) into which you have loaded the System-Tracing.1.cs changeset.

Evaluate the following in a workspace to clone the running image into a 64-bit equivalent:

    SystemTracer64 writeImage: 'squeak64.image'.
    Smalltalk garbageCollect.
Because of the "half-initialised" state of a cloned image, it is a good idea to start it up and save it immediately after cloning (in particular, before moving it out of the current working directory).

3.2. Building a 64-bit VM

You will need the modified Unix platform sources Squeak-3.7x-5.src.tar.gz and the All64BitChanges.1.cs change set.

The All64BitChanges change set should contain everything needed to add 64-bit compatibility to the generated sources. (Again this has been tested, and is known to work, with the 3.7-5989 full image.)

We did not modify VMMaker's user interface. However, the generated sources are currently different depending on the word size (4 or 8 bytes) desired. To select between the two possible sizes, evaluate one of the following expressions in a workspace:

    ObjectMemory initBytesPerWord: 4; initialize.
sets things up to generate a 32-bit VM, while
    ObjectMemory initBytesPerWord: 8; initialize.
sets things up to generate a 64-bit VM. (This only needs to be done once, when changing from one word size to the other.) Pressing the "clear out" and then "generate all" buttons in VMMaker will generate the sources as usual.

The VM can now be built in the traditional manner: create a build directory, descend into it, and then run configure followed by make.

3.3. Step-by-step guide

If the above is a little too terse, you can follow these steps to create your own 64-bit image and VM "from scratch":

  1. Download the Unix sources Squeak-3.7x-5.src.tar.gz from the above site.
  2. Unpack the sources and descend into the directory created.
  3. Download the two change sets (System-Tracing.1.cs and All64BitChanges.1.cs) from the above site to the current directory.
  4. Download the Squeak3.7-5989-full image and changes files from one of the usual Squeak sites into the current directory.
  5. Make a link to (or a copy of) SqueakV3.sources in the current directory.
  6. Start the image with your usual VM.
  7. File-in the two change sets. (I usually file-in System-Tracing followed by All64BitChanges, but the order should not matter.)
  8. Open VMMaker and make sure the paths are correct.
  9. Evaluate the following in a workspace:
    ObjectMemory initBytesPerWord: 8; initialize.
    (This sets everything up to build a 64-bit VM.)
  10. Press "clear out" and then "generate all" in the VMMaker window.
  11. Save the image, calling it "Squeak32" or something similar.
  12. Evaluate the following in a workspace:
    SystemTracer64 writeImage: 'Squeak64.image'.
    Smalltalk garbageCollect.
  13. Quit the image.
  14. Create a build directory and descend into it:
    $ mkdir bld
    $ cd bld
  15. Configure the build directory:
    $ ../platforms/unix/config/configure
  16. Build the VM:
    $ make
  17. Test the VM:
    $ cd ..
    ./bld/squeak Squeak64.image
  18. If it works, read the next section and volunteer to port a plugin or two!
To build a 32-bit VM from the same sources, follow a similar procedure but evaluate
ObjectMemory initBytesPerWord: 4; initialize.
before pressing "clear out" and "generate all" in VMMaker.

3.4. Caveats

Building a 64-bit VM on the Pentium confuses gcc-3. Use 2.95 instead. (This problem does not seem to affect any other architecture.)

4. Hacking the VM

The following guidelines should help keep the VM and plugin code working for both 32- and 64-bit VMs running on 32- and 64-bit hardware:

Two new types sqInt and usqInt are provided to help with this:

type   width
[unsigned] char   8 bits
[unsigned] short   16 bits
[unsigned] int   32 bits
usqLong, sqLong   64 bits
usqInt, sqInt   same as oop
sqInt and usqInt are defined according to the word size selected when the core VM sources are generated (as described above) and the widths of the available primitive types (as determined by the configure script). sqLong and usqLong are currently unused (pending the completion of 64-bit Bitmaps). Note that the CCodeGenerator now generates code in which all implicitly-typed quantities (i.e., oops) are declared as sqInt.

The traditional memory access macros have been extended, in the obvious manner with respect to the above types, as follows:

macro   use
byteAt: anOop [put: aValue]   read/write an 8-bit integer
shortAt: anOop [put: aValue]   read/write a 16-bit integer
intAt: anOop [put: aValue]   read/write a 32-bit integer
oopAt: anOop [put: aValue]   read/write an object pointer
longAt:[put:] is currently synonymous with intAt:[put:] (for backward compatibility with the vast majority of memory accesses in unconverted code) and should be eliminated during any conversion effort. (At some point in the near future, longAt: will refer to a 64-bit integer, corresponding to the sqLong type.)

The above all access memory based on an object pointer (which is an integer quantity, not a pointer). To access memory given a pointer to the value to be read/written, use one of the corresponding macros (of which only two are shown here, but they are available for all supported value types):

macro   use
byteAtPointer: aPointer [put: aValue]   read/write an 8-bit integer
oopAtPointer: aPointer [put: aValue]   read/write an object pointer
Conversion between an oop and a "real" pointer into memory must be performed using the two macros provided:
macro   use
pointerForOop: anOop   answer the address referenced by anOop
oopForPointer: somePointer   answer an oop corresponding to somePointer into memory

(The oop-based accessors are in fact written in terms of the pointer-based accessors, using the pointerForOop: "cast" macro to convert from their oop argument to a memory address.)

The traditional "cast" macro:

cCoerce: anyValue to: typeString
should be avoided unless absolutely necessary (and never used to coerce between integer and pointer values).

4.1. Cross-platform changes

A new file, "interp.h", is generated by VMMaker. This file defines the cpp macro SQ_VI_BYTES_PER_WORD as 4 or 8 according to the "bytesPerWord" setting in effect at the time the sources were generated.

A new file has been added: "platforms/Cross/vm/sqMemoryAccess.h". This file defines the types and macros described above. Note that the traditional memory access macros (byteAt:, longAt:, etc.) are no longer defined explicitly at the top of every generated source file. Any source that might need them must include sqMemoryAccess.h instead.

4.2. Who to call for help

Self-help might be available in the 64-bit FAQ.

We are happy to give guidance (and assistance in using the features described above) to anyone converting (or writing from scratch) VM or plugin code. Our e-mail addresses are: firstName at squeakland dot org