Nintendo 64 Part 5: My Own Code

Oct 25, 2020, Nintendo 64, Programming

It’s time to get my own code running.

As mentioned in part 3, I’m starting with LibUltra and making a 3D game. But I don’t want to copy a Nintendo example project into my game… at least this way, the source repository will be completely clean.

My rough plan for starting is as follows:

1. Get a simple blank screen.
2. Draw a moving primitive on the screen using the RDP.
3. Embed a texture in the program and draw it.
4. Embed a font in the program and draw text.

Blank Screen

There’s a big chunk of initialization and setup that needs to be done in order to get to a blank screen. The steps are outlined in the Nintendo 64 developer documentation, but I’m modifying the process to be a bit simpler at first—for one thing, I’ll start with everything in one C file, and I won’t be using multiple segments in the ROM image.

First, we need some stacks. On a modern operating system, the stack would be created for you, but on bare metal systems you have to do it yourself, and an easy way to do it is to just declare an array of the appropriate size. The stack is just a range of addresses in memory, and declaring a variable in C reserves memory the way we need.

The Nintendo code uses a stack of u64 elements, which forces the stack to be aligned to an 8-byte boundary, but we can do that with a an attribute, which is a GCC extension. This means we can just use u8[] for our stack, which makes the code simpler.

// Get a pointer to the start (top) of a stack.
#define STACK_START(x) ((x) + sizeof((x)))

enum {
  // Size of stacks for threads, in bytes.
  STACK_SIZE = 8 * 1024,
};

static u8 idle_thread_stack[STACK_SIZE] __attribute__((aligned(8)));
static u8 main_thread_stack[STACK_SIZE] __attribute__((aligned(8)));

The entry point itself just has to initialize the OS and then start the idle thread.

The boot() function is jumped to by the actual entry point, after setting up the stack, but the actual entry point is created by makerom / spicy so we don’t need to think about it. The call here to osStartThread does not return.

// Main N64 entry point.
void boot(void) {
  osInitialize();
  rom_handle = osCartRomInit();
  osCreateThread(&idle_thread, 1, idle, NULL,
                 STACK_START(idle_thread_stack), 10);
  osStartThread(&idle_thread);
}

The idle() function does more initialization, starts other threads, and then loops forever.

I see this note is in dram_stack.c in the onetri sample.

The "dram_stack" field of the RCP task structure is set to this address. It is placed in its own .c, and thus its own .o, since the linker aligns individual relocatables to data cache line size (16 byte) boundaries.

This avoids the problem where the dram_stack data is accidentally scribbled over during a writeback for data sharing the same line.

This may explain some of the structure of the example programs and how they are divided up. Maybe the IRIX IDO tools cannot align individual objects to 16 byte boundaries, but can line up object files? We are using the GNU Binutils linker, so we can just add alignment attributes to the individual objects.

Our game also needs a main thread to run the game. For now, it will just

Building my game, I got an error from Spicy. What? The game is so simple, how could I get a link error!

$ make
spicy -r game.n64 spec --toolchain-prefix=mips32-elf-
ERRO[0000] Error: spicy.LinkSpec: Error running 'mips32-elf-ld': exit status 1: `.text.startup' referenced in section `.text' of codesegment.o: defined in discarded section `.text.startup' of codesegment.o

It looks like the linker script that Spicy uses discards .text.startup, but this section is referenced from the main .text section. Presumably Spicy’s linker script is written with the assumption that all code is in .text, and something I’m doing is generating code or data in .text.startup. So, what’s in the .text.startup section? After consulting the objdump manual:

$ mips32-elf-objdump --syms --section .text.startup main.o

main.o:     file format elf32-bigmips

SYMBOL TABLE:
00000000 l    d  .text.startup  00000000 .text.startup
00000000 l     F .text.startup  0000020c main

My guess here is that GCC decides that any function named main() should go in a section named .text.startup. The laziest way to fix this is to rename the function to something else. However, we’re going to fix Spicy instead [Commit: Include more sections with wildcard].

It works! Our code displays a blank screen which cycles through colors.

From Spicy to Linker Scripts

Right now we’re relying on Spicy to make the ROM image, but we can actually do this ourselves with a linker script and a little bit of assembly.

Entry Point

The assembly we need is the entry point, which just needs to set up the stack and zero out the BSS section. This is fairly simple, since we can just call bzero to zero BSS.

# Entry point for the ROM image.
        .section .text.entry
        .global _start
_start:

        # Set up the stack
        la      $sp, _boot_stack

        # Zero BSS.
        la      $a0, _bss_start
        la      $a1, _bss_end
        jal     bzero
        subu    $a1, $a1, $a0

        # Jump to C entry point.
        j       boot
        nop

Linker Script

The first part of the linker script describes the memory regions. There are two regions we care about: ROM, which is the cartridge data, and RAM, which is where it will get loaded. Data in our program may have a location in ROM, a location in RAM, or locations in both ROM and RAM.

MEMORY {
    rom (R) : ORIGIN = 0, LENGTH = 64M
    ram (RWX) : ORIGIN = 0x80000400, LENGTH = 4M - 0x400
}

To add data to our program, we add a SECTIONS command. We’ll put individual sections inside.

SECTIONS {
    /* All our data goes here. */
}

The first 4 KiB of a ROM is the header and boot code. The header is short, so we can just put it in the linker script. The header doesn’t get loaded into RAM, so we mark this section as belonging to the ROM region. This is partially taken from /usr/lib/PR/romheader in the SDK.

.header : {
    LONG(0x80371240)
    LONG(0x0000000f)
    LONG(0x80000400)
    LONG(0x0000144c)
    . = 0x1000;
} >rom

The next part is the code and data for our program. This includes the .text, .rodata, and .data sections. To be clear—this won’t necessarily be all the data in our game, just the data in our compiled code like global variables and constants. Of special note is the .text.entry section, which is where we’ll put our entry point. This is how we make sure that the entry point is at ROM address 0x1000, where the IPL3 boot code loads data from, and at RAM address 0x80000400, which is where the IPL3 boot code jumps to when it is done.

The >ram AT>rom attributes tell the linker that the section will be loaded into RAM from ROM.

.text : {
    _text_start = .;
    *(.text.entry)
    *(.text .text.*)
    *(.rodata .rodata.*)
    *(.data .data.*)
    _text_end = .;
} >ram AT>rom

Next is the BSS section. This section contains variables which are initialized to zero. Since they’re initialized to zero, they don’t need to be stored in the ROM image at all, so they are only given a location in RAM.

If we were only writing our own code, we could just include the .bss section. However, LibUltra includes “common” variables, which are variables that may be defined in multiple files (a quirk of old C code—this happens when you define a variable in a header file without declaring it extern).

.bss (NOLOAD) : ALIGN(16) {
    _bss_start = .;
    *(.bss .bss.*)
    *(COMMON)
    *(.scommon .scommon.*)
    _bss_end = .;
} >ram

Next, we have the linker script set aside some regions of memory for the stacks. Note that the symbols we export point to the end of each stack, not the beginning, because the stack grows from higher addresses towards lower addresses, and so the highest address is where it starts.

Also note that we define _boot_stack and _main_stack to be the same. They won’t be used at the same time, so this saves a little memory.

STACK_SIZE = 8K;

.stack (NOLOAD) : ALIGN(16) {
    . += STACK_SIZE;
    . = ALIGN(8);
    _boot_stack = .;
    _main_thread_stack = .;

    . += STACK_SIZE;
    . = ALIGN(8);
    _idle_thread_stack = .;
} >ram

Our object files will contain many other sections, containing things like debugging information, the version of the compiler we used, and information for exception handling. We don’t need these sections. We discard them with a special /DISCARD section.

/DISCARD/ : {
    *(*)
}

Finally, we are going to generate an ELF file during the build process, so we’ll specify the start address at the top of our linker script.

ENTRY(_start)

Code Changes

In our main C file, we change how the stacks are defined. We declare them as extern and use the symbols we defined in the linker script. Since the symbols point to where the stack starts (the high address), we just pass them directly to osCreateThread.

extern u8 _main_thread_stack[];
extern u8 _idle_thread_stack[];

osCreateThread(&idle_thread, 1, idle, NULL, _idle_thread_stack, 10);
osCreateThread(&main_thread, 3, main, NULL, _main_thread_stack, 10);

Makefile

We replace the call to spicy with two steps: first we create an ELF file, and then we copy the data in the ELF file to a ROM image and install the boot code with makemask as before.

objs := main.o start.o
$(name).elf: link.lds $(objs)
    $(CC) -nostdlib -o $@ -T link.lds $(objs) \
        ../sdk/ultra/lib/libgultra_rom.a -lgcc

$(name).n64: $(name).elf
    mips32-elf-objcopy -O binary $< $@
    makemask $@

Objcopy works here by generating a memory dump, and it starts at the address of the lowest loaded section. Note that this means that you actually have to have data in the .header section. The following version of the linker script will not work, because the file created by objcopy will not include the 4 KiB ROM header, not even a blank header:

/* This will not work! */
.header : {
    . = 0x1000;
} >rom

• Part 4
• Nintendo 64 Homebrew
• Part 6