non-paged supervisor code calling paged application code

[sdo303]

Firstly, a proper thankyou to Boriel for the compiler.   And secondly, also a proper thankyou to Duefectu for the beginner's guide book (I bought a hard copy), which I have read cover to cover (but I did skip the sound/audio sections ).  And I bought a hard copy of Simon G's NextTech book too, again I have read it cover to cover (but skipped the sound/audio parts).

Using Boriel ZXBasic and as little Z80 machine code as I can get away with (until I need to performance tune)... my project is that I am trying to write a quasi "character cell" based application, like being on an old VT100, whereby the application uses all 64KB of the Z80 for itself, and thus no ZX Spectrum ROM, and which internally leverages a poor man's virtual memory framework, whereby I have a supervisor layer of non-paged code running in MMU0/1, which boots from $0000 and jumps over IM1 at $0038 to begin my main code properly at say $0050 where the application begins properly... and so I have my core non-paged system variables mapped in MMU2 and then I have my paged code to run from MMU3 - and then various different aspects of paged application data paging in and out MMU4/5/6, with finally MMU7 again being static non-paged to host the 5KB of screen tile map for an 80x32 screen, with the final 3KB of MMU7 being available for the stack pointer from $FFFE downwards.
phew...
I know my code will never work first time, I get that... but I feel fairly sure that I have the basic (excuse the pun) initial structure of what I want to achieve, with all of my system variables (so far identified) defined at static addresses in MMU2, and my main supervisor MMU paging and also my screen updating routines and keyboard reading routines written (to be permanently resident in MMU0/1) and the first of my paged blocks of code to run from MMU3, and I have gotten the source code clean enough pass through zxbc without the compiler spitting any hard errors... (just lots and lots of warnings)... but I've now hit a roadblock...

I get how I can have many separate pages of 8KB deposited across the ZX Next vast 200+ pages of 8KB across 1.75 MB of RAM, and I get how I can have my code paged in at MMU3 for my application to execute from $6000, and I can get my non-paged supervisor layer code to make calls such as       uw_dummy = USR $6000     to call my paged code in MMU3, and thus wait for a return... but what I don't understand is how I can have many different paged code blocks where each is aware of *ALL* of the available core supervisor routines and their parameter structures and thus able to makes calls back in to the supervisor to run any of the many core system routines...

i.e. how can I make zxbc firstly compile the application core without dropping routines that zxbc thinks are unused, and yet also somehow get the compilations of many different "paged" application code to be aware of the core supervisor routines and their parameters and any return value data types (of functions).

e.g. I get this from my compilation of my "core.bas":
core.bas:115: warning: [W170] Function 's_crash' is never called and has been ignored
core.bas:154: warning: [W170] Function 's_fc_ram_copy' is never called and has been ignored
core.bas:165: warning: [W170] Function 's_clear_screen' is never called and has been ignored
core.bas:126: warning: [W170] Function 's_print_char_at' is never called and has been ignored
core.bas:143: warning: [W170] Function 's_print_string_at' is never called and has been ignored
core.bas:217: warning: [W170] Function 's_display_monitor' is never called and has been ignored
core.bas:283: warning: [W170] Function 'fub_key_get_minutes' is never called and has been ignored

...and I get this for my first stage of my application:
stage-include.bas:37: warning: [W170] Function 's_next_reg' is never called and has been ignored
stage-include.bas:38: warning: [W170] Function 's_return_to_zxos' is never called and has been ignored
stage-include.bas:39: warning: [W170] Function 's_crash' is never called and has been ignored
stage-include.bas:40: warning: [W170] Function 's_fc_ram_copy' is never called and has been ignored
stage-include.bas:42: warning: [W170] Function 's_clear_screen' is never called and has been ignored
stage-include.bas:43: warning: [W170] Function 's_print_char_at' is never called and has been ignored
stage-include.bas:44: warning: [W170] Function 's_print_string_at' is never called and has been ignored
stage-include.bas:45: warning: [W170] Function 's_display_monitor' is never called and has been ignored
stage-include.bas:47: warning: [W170] Function 'fub_key_get_minutes' is never called and has been ignored
stage-include.bas:48: warning: [W170] Function 's_key_wait_for_vblank' is never called and has been ignored
stage-include.bas:49: warning: [W170] Function 's_get_key' is never called and has been ignored
...
stage-init.bas:8: warning: [W170] Function 's_stage_init' is never called and has been ignored
stage-init.bas:20: warning: [W170] Function 's_init_keyboard_table_ZX_keys' is never called and has been ignored
stage-init.bas:21: warning: [W170] Function 's_init_keyboard_table_ZX_symkeys' is never called and has been ignored
stage-init.bas:18: warning: [W170] Function 's_init_load_palette' is never called and has been ignored
stage-init.bas:186: warning: [W170] Function 's_init_load_palette_entry' is never called and has been ignored
stage-init.bas:223: warning: [W170] Function 's_init_clear_screen' is never called and has been ignored
stage-init.bas:26: warning: [W170] Function 's_init_title_screen' is never called and has been ignored

...where stage-include.bas was my attempt to make each "stage" aware of a jump-table of locations of core routines:
Asm
    ; fixed jump table addresses for the linker
    _s_main                          equ $0050
    _s_next_reg                      equ $0053
    _s_return_to_zxos                equ $0056
    _s_crash                         equ $0059
    _s_fc_ram_copy                   equ $005C
    _s_clear_screen                  equ $005F
    _s_print_char_at                 equ $0062
    _s_print_string_at               equ $0065
    _s_display_monitor               equ $0068
    _fub_key_get_minutes             equ $006B
    _s_key_wait_for_vblank           equ $006E
    _s_get_key                       equ $0071
End Asm

' implementation blocks to stop "not implemented" errors
' these contain no code; the equate above handles the address
Sub s_main(): End Sub
Sub s_next_reg(reg As UByte, value As UByte): End Sub
Sub s_return_to_zxos(): End Sub
Sub s_crash(error_code As UByte, error_text As String): End Sub
Sub Fastcall s_fc_ram_copy(src_address As UInteger, dest_address As UInteger, byte_count As UInteger): End Sub
Sub s_clear_screen(): End Sub
Sub s_print_char_at(x As UByte, y As UByte, char As UByte, colour As UByte): End Sub
Sub s_print_string_at(x As UByte, y As UByte, text As String, colour As UByte): End Sub
Sub s_display_monitor(): End Sub
Function fub_key_get_minutes() As UByte: return 0: End Function
Sub s_key_wait_for_vblank(): End Sub
Sub s_get_key(): End Sub


But clearly there is something I'm not getting, and I've reached the limits of capability of AI helper tools.

I had rationalised to myself that my build script should only call to zxbc to produce "-f asm" ASM files... as I expect to have to then somehow "link" the static core ASM and the many paged stage ASM together at some point... and I also seem to have rationalised that I may perhaps need to use     sjasmplus     to assemble my ".nex" package.

Does the above sound achievable?  Or am I asking too much of the "Next" as a platform?  i.e. is the "Next" only any good for paging graphics data, and no good for paging application code?

[sdo303]

ok - so I have managed to get the compilation (to ASM) of my "core.bas" supervisor to no longer drop the routines (as uncalled) by getting my s_main() routine to call this dummy "do nothing" routine:

Sub s_super_keeper()
    Dim p, q, swap As UInteger
    p = @s_next_reg
    p = @s_return_to_zxos
    p = @s_crash
    p = @s_fc_ram_copy
    p = @s_clear_screen
    p = @s_print_char_at
    p = @s_print_string_at
    p = @s_display_monitor
    p = @fub_sys_get_minutes
    p = @s_wait_for_vblank
    p = @s_get_key
    swap = p
    p = q
    q = swap
End Sub

...with a final variable swap to stop the compiler from noticing that 'p' does nothing genuinely useful...

...now back to trying to determine how to make calls back to routines in the supervisor.... hmmm.

[boriel]

A trick (for the moment) to mark a function / sub as "used", is to do something like:

Code:

DIM dummy_var As UInteger: REM will be dropped by the optimizer LET dummy_var = @my_sub: REM this marks my_sub as "used"

[sdo303]

That worked nicely - as a method for tricking the compiler optimization to not discard what appears to be (as yet) unused routines in the core supervisor. Thank you for that.

And I've been dreaming/thinking of a way of using shims for the "core" subs and functions, in the paged code setions, and I'll use the same trick to keep all the shims present in the paged code sections, and to quieten down the compiler optimisation discard warnings.

When I've got a decent demo working of core performing multiple (at least two) MMU page swaps for paged code sections, and the core calling those paged sections, and then having those paged code sections make calls back to general purpose routines in the core... when I have this working then I will share all the code and build scripts and config in case anyone else ever wants to try anything similar.

[boriel]

This is something also being worked on... for the moment you can call "submodules" compiled in paged memory. @Duefectu has documented how to do it.

[sdo303]

This is something also being worked on... for the moment you can call "submodules" compiled in paged memory. @Duefectu has documented how to do it.

Thank you.

I've gotten my code to compile to asm...

Code:

PS D:\zxnext\projects\demo> .\build.ps1 zxbc.py 1.18.5 ...compiling:  core core.bas:70: warning: [W150] Parameter 'x' is never used core.bas:70: warning: [W150] Parameter 'y' is never used core.bas:70: warning: [W150] Parameter 'char' is never used core.bas:70: warning: [W150] Parameter 'colour' is never used ...compiling:  stage-init stage-include.bas:10: warning: [W170] Function 's_ram_copy' is never called and has been ignored stage-include.bas:24: warning: [W170] Function 's_print_char_at' is never called and has been ignored ...compiling:  stage-left stage-include.bas:3: warning: [W170] Function 's_next_reg' is never called and has been ignored stage-include.bas:10: warning: [W170] Function 's_ram_copy' is never called and has been ignored stage-include.bas:17: warning: [W170] Function 's_clear_screen' is never called and has been ignored stage-include.bas:31: warning: [W170] Function 's_print_string_at' is never called and has been ignored ...compiling:  stage-right stage-include.bas:3: warning: [W170] Function 's_next_reg' is never called and has been ignored stage-include.bas:10: warning: [W170] Function 's_ram_copy' is never called and has been ignored stage-include.bas:17: warning: [W170] Function 's_clear_screen' is never called and has been ignored stage-include.bas:31: warning: [W170] Function 's_print_string_at' is never called and has been ignored PS D:\zxnext\projects\demo>

...using thisPowerShell:

Code:

$ErrorActionPreference = 'Stop' Function Abort {     Param( [string]$Text )     Write-Error $Text     Break Script } $Compiler = 'D:\zxnext\boriel\zxbc.py' python $Compiler --version $Assembler = 'unknown' # compile the core engine, this sits at address 0 and contains the IM1 interrupt vector... Write-Host '...compiling:  core' # ...help says this:  headerless mode: omit ASM prologue and epilogue # ...I have no idea as yet whether I need either or neither or both... python $Compiler core.bas -f asm -o core.asm --org 0 -W140 -W160 --headerless --explicit --strict --zxnext -i --arch zxnext --opt-strategy size --heap-address 16000 --heap-size 383 $Status = $LastExitCode If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } # compile the paged stage modules, these all target the MMU3 slot at $6000 (24576)... $Stages = @( 'init', 'left', 'right' ) ForEach ( $Stage In $Stages ) {     $Name = "stage-$($Stage)"     Write-Host "...compiling:  $Name"     python $Compiler "$($Name).bas" -f asm -o "$($Name).asm" --org 24576 -W150 --headerless --explicit --strict --zxnext -i --arch zxnext --opt-strategy size --heap-address 32000 --heap-size 767     $Status = $LastExitCode     If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } } Break Script

So, I have four new ".asm" files...

...does anyone have any tips on how I can assemble them join them together in to some kind of '.nex' package ?  Please      thank you!

[sdo303]

so, now my header of "core.bas" is:

Code:

#PRAGMA push(case_insensitive) #PRAGMA case_insensitive = True #define STANDALONE  ' remove ROM initialization and SysVars... Asm     org $0000     nop             ; your first nop     nop             ; your second nop     di              ; disable interrupts during setup     nop                     ; since SAVENEX CFG 7 killed the ROM, we are already in RAM     jp _jump_over   ; jump to the setup logic                         ; --- im 1 interrupt vector ---     org $0038       ; hardware interrupt location     ei              ; re-enable interrupts     reti            ; return from interrupt     _jump_over:     nop     im 1            ; set interrupt mode 1     nop     ld sp, $fffe    ; set stack to top of bank 57     nop End Asm s_main() #include "constants.bas" #include "globals.bas"

...and my sjasmplus linker is :

Code:

    DEVICE ZXSPECTRUMNEXT ; entry point at $0000 and stack at $FFFE     SAVENEX OPEN "project.nex", $0000, $FFFE ; enable 2mb mode     SAVENEX CFG 0, 0, 0, 1 ; *** the hardware lock *** ; this bit tells the Next to disable the ROM and allow page 50 to map to slot 0     SAVENEX CFG 7, 0, 0, 1 ; load the core logic into page 50/51     MMU 0, 50     MMU 1, 51     ORG $0000     INCBIN "core.bin" ; load font into bank 56     MMU 6, 56     ORG $C000     INCBIN "fon-056.fon" ; load stages     MMU 3, 60 : ORG $6000 : INCBIN "stage-init.bin"     MMU 3, 68 : ORG $6000 : INCBIN "stage-left.bin"     MMU 3, 69 : ORG $6000 : INCBIN "stage-right.bin" ; final hardware state setup     MMU 0, 50    ; core code     MMU 1, 51    ; core code     MMU 2, 52    ; core data     MMU 7, 57    ; screen and stack     SAVENEX AUTO     SAVENEX CLOSE

but for some reason I still see the normal Spectrum ROM in   zesarux :

Code:

¤ Debug CPU (step)   mptr:      0001H [X]  flwPC 1-7:V1 +0001  IM 1                PC 0001   0003  JP 0080            SP FFFF   0006  RST 38              AF FFFF'FFFF   0007  RST 38              SZ5H3PNC   0008  RETI                HL FFFF'FFFF   000A  RST 38              DE FFFF'FFFF   000B  RST 38              BC FFFF'FFFF   000C  RST 38              IX FFFF   000D  RST 38              IY FFFF   000E  RST 38              IR 0001   000F  RST 38            [IM0] IFF--   0010  RETI              (HL) 00 F3   0012  RST 38            (DE) 00 F3                           (BC) 00 F3 (SP)  F300 56ED 80C3 FF00 EDFF F stM dasm en:Stp Stovr cntSt Hx Chr brk wtch ToglL Run Runt Ret ClrTstpart Write vScr Memzn -1 Pc=ptr nxtpcBr stk cpuHst f1:he

I've tried for literally hours in a loop with Gemini to unmap the Spectrum ROM and get my code at page 50 to map to MMU0.

Could anyone please help me?    Thank you!

[sdo303]

well I changed the sjasmplus linker to attempt to run a short piece of ASM from $8000 and I set the SP to $BEEF and I can see in zesarux that the SP in indeed $BEEF    so.... we know that my project.nex is making it to be started but for some reason zesarux is stubbornly doing two things...  1) holding on to speccy ROM    2) but more tellingly zesarux is not executing my code at $8000 whose job is this :

named: load.bas

Code:

Asm     org $8000     di     ld a, 2            ; red color     out ($fe), a        ; set border .loop:     jr .loop            ; stay here forever     ld bc, $243b        ; select the rom lock register (nextreg $8f)     ld a, $8f          ; rom lock register     out (c), a     inc b     ld a, 0            ; value 0: unlock and map ram     out (c), a     ld bc, $243b        ; select the write enable register (nextreg $8e)     ld a, $8e          ; write control register     out (c), a     inc b     ld a, 3            ; write 3 to allow writing to the new ram in slots 0 and 1     out (c), a     ld bc, $243b        ; force page 50 into slot 0 to be absolutely certain     ld a, $50          ; mmu slot 0 register     out (c), a     inc b     ld a, 50            ; page 50     out (c), a     jp $0000            ; jump to your actual core logic End Asm

here's my build script named "build.ps1" :

Code:

$ErrorActionPreference = 'Stop' Function Abort {     Param( [string]$Text )     Write-Error $Text     Break Script } $Compiler  = 'D:\zxnext\boriel\zxbc.py' $Assembler = 'D:\zxnext\sjasmplus\sjasmplus.exe' $Emulator  = 'D:\zxnext\zesarux\zesarux.exe' $SDCard    = 'D:\zxnext\projects\demo\sdcard.mmc' python $Compiler --version Write-Host '...compiling to ASM:  load' -ForegroundColor DarkYellow python $Compiler load.bas -f asm -o load.asm --org 32768 --headerless --explicit --strict --zxnext -i --arch zxnext --opt-strategy size --heap-address 40000 --heap-size 400 $Status = $LastExitCode If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } Write-Host '...compiling to bin:  load' -ForegroundColor DarkYellow python $Compiler load.bas -f bin -o load.bin --org 32768 --headerless --explicit --strict --zxnext -i --arch zxnext --opt-strategy size --heap-address 40000 --heap-size 400 -M load.map $Status = $LastExitCode If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } # compile the core engine, this sits at address 0 and contains the IM1 interrupt vector... # ...help says this:  headerless mode: omit ASM prologue and epilogue # ...I have no idea as yet whether I need either or neither or both... Write-Host '...compiling to ASM:  core' -ForegroundColor DarkYellow python $Compiler core.bas -f asm -o core.asm --org 0 -W140 -W160 --headerless --explicit --strict --zxnext -i --arch zxnext --opt-strategy size --heap-address 16000 --heap-size 383 $Status = $LastExitCode If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } Write-Host '...compiling to bin:  core' -ForegroundColor DarkYellow python $Compiler core.bas -f bin -o core.bin --org 0 -W140 -W160 --headerless --explicit --strict --zxnext -i --arch zxnext --opt-strategy size --heap-address 16000 --heap-size 383 -M core.map $Status = $LastExitCode If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } # compile the paged stage modules, these all target the MMU3 slot at $6000 (24576)... $Stages = @( 'init', 'left', 'right' ) ForEach ( $Stage In $Stages ) {     $Name = "stage-$($Stage)"     Write-Host "...compiling to ASM:  $Name" -ForegroundColor DarkYellow     python $Compiler "$($Name).bas" -f asm -o "$($Name).asm" --org 24576 -W150 --headerless --explicit --strict --zxnext -i --arch zxnext --opt-strategy size --heap-address 32000 --heap-size 767     $Status = $LastExitCode     If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" }     Write-Host "...compiling to ASM:  $Name" -ForegroundColor DarkYellow     python $Compiler "$($Name).bas" -f bin -o "$($Name).bin" --org 24576 -W150 --headerless --explicit --strict --zxnext -i --arch zxnext --opt-strategy size --heap-address 32000 --heap-size 767 -M "$($Name).map"     $Status = $LastExitCode     If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } } # stitch everything together with sjasmplus... Write-Host '...assembling nex file...' -ForegroundColor DarkYellow & $Assembler project.cfg --nologo --msg=all --lst=project.lst --lstlab $Status = $LastExitCode If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } $CoreFile = Get-Item "core.bin" $CoreSize = [math]::Round( ( $CoreFile.Length / 16384 ) * 100, 2 ) Write-Host "...core size:  $( $CoreFile.Length) bytes  ($CoreSize% of 16KB)" -ForegroundColor Cyan     # ...clean up intermediate bin files to keep the project tidy # ...Remove-Item core.bin, stage-init.bin, stage-left.bin, stage-right.bin $FileName = "project.nex" $FileBytes = [System.Text.Encoding]::ASCII.GetBytes($FileName) # the line length is the filename + 4 bytes (for the token, two quotes, and the CR) $LineLength = $FileBytes.Count + 4 $Header = [byte[]](     0x00, 0x0A,      # line number 10     $LineLength, 0x00, # length of the line in bytes     0x00,            # dummy byte     0xEF,            # LOAD token     0x22            # opening quote (") ) $Footer = [byte[]](     0x22,            # closing quote (")     0x0D            # carriage return ) # combine everything into one array $FinalBytes = $Header + $FileBytes + $Footer $AutoExec = Join-Path -Path $PWD -ChildPath "autoexec.bas" [System.IO.File]::WriteAllBytes( $AutoExec, $FinalBytes ) # use hdfmonkey to swap it onto the mmc .\hdfmonkey.exe rm .\tbblue.mmc autoexec.dot .\hdfmonkey.exe put .\tbblue.mmc autoexec.bas Write-Host '...hdfmonkey rm...' -ForegroundColor DarkYellow .\hdfmonkey.exe rm tbblue.mmc project.nex /project.nex $Status = $LastExitCode # If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } Write-Host '...hdfmonkey ls...' -ForegroundColor DarkYellow .\hdfmonkey.exe ls  tbblue.mmc | Select-String 'project' $Status = $LastExitCode If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } Write-Host '...hdfmonkey put...' -ForegroundColor DarkYellow .\hdfmonkey.exe put tbblue.mmc project.nex /project.nex $Status = $LastExitCode If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } Write-Host '...hdfmonkey ls...' -ForegroundColor DarkYellow .\hdfmonkey.exe ls  tbblue.mmc | Select-String 'project' $Status = $LastExitCode If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } Write-Host "...launching emulator..." -ForegroundColor DarkYellow & $Emulator --configfile zesarux.cfg $Status = $LastExitCode If ( $Status -ne 0 ) { Abort "...failed, status:  $Status" } Break Script

here's my sjasmplus linker named "project.cfg" 

Code:

    DEVICE ZXSPECTRUMNEXT     SAVENEX OPEN "project.nex", $8000, $BEEF, 54     SAVENEX CFG 0, 0, 0, 1     ORG $0000 : MMU 0, 50, $0000     ORG $2000 : MMU 1, 51, $2000     ORG $4000 : MMU 2, 52, $4000     ORG $6000 : MMU 3, 53, $6000     ORG $8000 : MMU 4, 54, $8000     ORG $A000 : MMU 5, 55, $A000     ORG $C000 : MMU 6, 56, $C000     ORG $E000 : MMU 7, 57, $E000     ORG $8000 : MMU 4, 54, $8000 : INCBIN "load.bin"         ORG $0000 : MMU 0, 50, $0000 : INCBIN "core.bin"     ORG $C000 : MMU 6, 56, $C000 : INCBIN "fon-056.fon"     ORG $6000 : MMU 3, 60, $6000 : INCBIN "stage-init.bin"     ORG $6000 : MMU 3, 68, $6000 : INCBIN "stage-left.bin"     ORG $6000 : MMU 3, 69, $6000 : INCBIN "stage-right.bin" ; move the address counter back to where the loader is     ORG $8000 : MMU 4, 54, $8000     SAVENEX BANK 50     SAVENEX BANK 51     SAVENEX BANK 52     SAVENEX BANK 53     SAVENEX BANK 54     SAVENEX BANK 55     SAVENEX BANK 56     SAVENEX BANK 57     SAVENEX BANK 60     SAVENEX BANK 68     SAVENEX BANK 69     SAVENEX CLOSE

here's my zesarux.cfg file :

Code:

--machine tbblue --set-breakpoint 1 "PC=1" --set-breakpoint 2 "PC=8000" --set-breakpoint 3 "PC=8001" --set-breakpointaction 1 "" --set-breakpointaction 2 "" --set-breakpointaction 3 "" --enable-breakpoints --nowelcomemessage --quickexit --no-saveconf-on-exit --zoomx 1 --zoomy 1 --mmc-file tbblue.mmc --enable-mmc --enable-divmmc-ports --realvideo --smartloadpath project.nex

so I still cannot fathom a way to get the speccy ROM out of the way :/

[sdo303]

well, I managed to get my bas to compile to bin, and to get sjasmplus to create a nex file, and it seems to be ok in the "nex" viewer extension in vscode... but can I get my nex file to take over the machine... no way... many hours... and I'm still stuck with zesarux refusing to let me take over MMU0..5 the best I can get is : MMU=80008001000a000b0004000500360037 in the zesarux debug.