MegaLZ depacker

[britlion]

I was using MegaLZ, and I figured it worth listing how to make it a SUB.

Trivially easy, actually - it's provided as ASM source. There's a windows binary program to compress data. Just call this like a mem copy, with source (packed data) and destination (for unpacked data).

I've used it to pack down screen data, and just unpacked it to the screen - it's slow enough to be visible, so do it with attributes set to invisible, and the user won't notice.

Code:

SUB megaLZDepack (source as uInteger, dest as uInteger) ASM LD E,(IX+6) LD D,(IX+7) ;Z80 depacker for megalz V4 packed files (C) fyrex^mhm ; DESCRIPTION: ; ; Depacker is fully relocatable, not self-modifying, ;it's length is 110 bytes starting from DEC40. ;Register usage: AF,AF',BC,DE,HL. Must be CALL'ed, return is done by RET. ;Provide extra stack location for store 2 bytes (1 word). Depacker does not ;disable or enable interrupts, as well as could be interrupted at any time ;(no f*cking wicked stack usage :). ; USAGE: ; ; - put depacker anywhere you want, ; - put starting address of packed block in HL, ; - put location where you want data to be depacked in DE, ; (much like LDIR command, but without BC) ; - make CALL to depacker (DEC40). ; - enjoy! ;) ; PRECAUTIONS: ; ; Be very careful if packed and depacked blocks coincide somewhere in memory. ;Here are some advices: ; ; 1. put packed block to the highest addresses possible. ; Best if last byte of packed block has address #FFFF. ; ; 2. Leave some gap between ends of packed and depacked block. ; For example, last byte of depacked block at #FF00, ; last byte of packed block at #FFFF. ; ; 3. Place nonpackable data to the end of block. ; ; 4. Always check whether depacking occurs OK and neither corrupts depacked data ; nor hangs computer. ; ;DEC40 LD A,80h EX AF,AF' MS: LDI M0: LD BC,2FFh M1: EX AF,AF' M1X: ADD A,A JR NZ,M2 LD A,(HL) INC HL RLA M2: RL C JR NC,M1X EX AF,AF' DJNZ X2 LD A,2 SRA C JR C,N1 INC A INC C JR Z,N2 LD BC,33Fh JR M1 X2: DJNZ X3 SRL C JR C,MS INC B JR M1 X6: ADD A,C N2: LD BC,4FFh JR M1 N1: INC C JR NZ,M4 EX AF,AF' INC B N5: RR C JP C, END_DEC40 RL B ADD A,A JR NZ,N6 LD A,(HL) INC HL RLA N6: JR NC,N5 EX AF,AF' ADD A,B LD B,6 JR M1 X3: DJNZ X4 LD A,1 JR M3 X4: DJNZ X5 INC C JR NZ,M4 LD BC,51Fh JR M1 X5: DJNZ X6 LD B,C M4: LD C,(HL) INC HL M3: DEC B PUSH HL LD L,C LD H,B ADD HL,DE LD C,A LD B,0 LDIR POP HL JR M0 END_DEC40: END ASM END SUB

[boriel]

Nice :!:
I'll convert it to fastcall and will pack it with the compiler.

Update: I see you load DE from (IX+n) but not HL? Probably it's working because the compiler uses HL to load addresses and it's already set. But this could crash under some circumnstances. There should be a LD HL, (IX+n) sequence to load HL too.

But better convert it to fascall:

Code:

SUB FASTCALL megaLZDepack (source as uInteger, dest as uInteger) ASM ; HL already loaded (source) ; dest in the stack, before the return address pop bc ; Remove return address pop de ; DE <-- dest address ; NOTE: Remove LD D, (IX+n) and LD E, (IX+n) instructions push bc ; Restores return address ; The remaing code is untouched: ;Z80 depacker for megalz V4 packed files (C) fyrex^mhm ; DESCRIPTION: ; ; Depacker is fully relocatable, not self-modifying, ;it's length is 110 bytes starting from DEC40. ...

[britlion]

I thought fastcall only worked with one parameter?

Still confused as to how those work!

Anyway, if you can make it fastcall, you can probably do ret c instead of jp c, end_of_subroutine; which makes it tighter. (it originally would have had that).

[boriel]

I thought fastcall only worked with one parameter?

Still confused as to how those work!

Anyway, if you can make it fastcall, you can probably do ret c instead of jp c, end_of_subroutine; which makes it tighter. (it originally would have had that).

FASTCALL can be use with more than one parameter, but only the 1st one will be passed in registers. Remaining ones will be passed in the stack, and the programmer MUST pop them out before returning, or your program could crash, etc.

If you look back to my code, I pop out the 2nd parameter in DE. The problem here is that the return address (used by RET) is always pushed automatically by the CPU on TOP, so you must preserve it (pop it out, pop out your parameters, and push it in again before returning). So this is why I use POP BC, POP DE, PUSH BC sequence (BC = temporary storage here).

[LCD]

THIS WILL be VERY USEFUL for me... But the problem is the gap between blocks of compressed blocks. I would prefer to have a parameter for length of packed or unpacked data.

[britlion]

I don't think I understand what you mean there LCD. It sort of is what it is really. Takes compressed data from source and puts it as expanded data at destination. There's no data gap internal to either format.

My use for it at the moment is expanding banners to the first 2k of the screen from compressed version elsewhere.

I think a common use might be setting up a buffer for data, and expanding compressed data to the buffer for use, replacing the buffer contents as new data is required - such as the player moving to a new screen.

[boriel]

britlion, does the FASTCALL and push pop sequence I put worked for you?

[LCD]

I don't think I understand what you mean there LCD. It sort of is what it is really. Takes compressed data from source and puts it as expanded data at destination. There's no data gap internal to either format.

But how does the dapacker know, how much to depack?
"3. Place nonpackable data to the end of block."
means surely a gap (filled with rubbish), or did I understand it wrong and the size is encoded by compressor in packed data?

Edit: I now understand it!This note is for compressor only if we want to pack multiple data from same file. My fault!!!

[britlion]

britlion, does the FASTCALL and push pop sequence I put worked for you?

No.
I still don't understand this, apparently.

e.g.: function mirror (dowedoit as uByte, number as uByte) as uByte

This gets dowedoit in the A register, and both dowedoit and number in the stack.

Stack seems to be:
return address
<some pair of bytes I can't work out>
first parameter pair
second parameter pair

However, if in the above case I do
pop hl
pop de
pop de
pop bc
push hl
ret

It still seems to behave very strangely.

I know that a first byte parameter is at IX+5 (and IX+4 if it's 16 bit)- so logically it's the return address at IX+0, and IX+1. What's at IX+2,IX+3?

[britlion]

I don't think I understand what you mean there LCD. It sort of is what it is really. Takes compressed data from source and puts it as expanded data at destination. There's no data gap internal to either format.

But how does the dapacker know, how much to depack?
"3. Place nonpackable data to the end of block."
means surely a gap (filled with rubbish), or did I understand it wrong and the size is encoded by compressor in packed data?

Edit: I now understand it!This note is for compressor only if we want to pack multiple data from same file. My fault!!!

Ah yes - LCD. A packed file knows its own length; it's not continuous. You can't unpack part of it without changing the decompression code, there.

[LCD]

I don't think I understand what you mean there LCD. It sort of is what it is really. Takes compressed data from source and puts it as expanded data at destination. There's no data gap internal to either format.

But how does the dapacker know, how much to depack?
"3. Place nonpackable data to the end of block."
means surely a gap (filled with rubbish), or did I understand it wrong and the size is encoded by compressor in packed data?

Edit: I now understand it!This note is for compressor only if we want to pack multiple data from same file. My fault!!!

Ah yes - LCD. A packed file knows its own length; it's not continuous. You can't unpack part of it without changing the decompression code, there.

Cool, then it will be extremly useful in my next projects. Great!

Edit: If this is so easy, maybe Exonomizer 2.01 depacker could be a good idea. Metalbrain made a Z80 depacker for it. There are also Chrust and Bitbuster Extreme. No idea which is faster or has better compression.

[LCD]

Okay, the Binary manager of BorIDE will have the ability to pack binaries (also imported from TAP) using MegaLZ. It is very effective. I tried to pack a previously packed screen, where PKLite saved 1 Kb, MegaLZ saved 2 Kb. So the screen packed from 6912 Bytes using Retro-X screen compressor (which will be used in BorIDE too) to 3500 Bytes was further packed by MegaLZ to 1500 Bytes.

Code:

JP C, END_DEC40

IMHO should be replaced with

Code:

JR C, END_DEC40

because all other Jumps in the source are relative, and this saves 1 Byte and is faster (okay, nobody will notice). For me every byte counts in the Dungeon Crawler project.

[slenkar]

What kind of information can be compressed?

Graphics, arrays, sounds..?

[LCD]

What kind of information can be compressed?

Graphics, arrays, sounds..?

Wow!
Excellent... Yes, I'm working hard to do so.

Any binary data can be compressed, but not arrays.
Natively: Graphics blocks and maps/screens (after adding map editor)

[LCD]

Poke 23388 with current Bank
And this is the version which can decompress from other banks to a buffer to main memory:

Code:

' MegaLZ Unpacker by Britlion, Banking by LCD SUB megaLZDepack(source as uInteger,dest as uInteger,bank as ubyte) ASM LD E,(IX+6) LD D,(IX+7) di ld a,(ix+9) ld bc,32765 out (c),a ;'Z80 depacker for megalz V4 packed files (C) fyrex^mhm ;' DESCRIPTION: ;' ;' Depacker is fully relocatable, not self-modifying, ;'it's length is 110 bytes starting from DEC40. ;'Register usage: AF,AF',BC,DE,HL. Must be CALL'ed, return is done by RET. ;'Provide extra stack location for store 2 bytes (1 word). Depacker does not ;'disable or enable interrupts, as well as could be interrupted at any time ;'(no f*cking wicked stack usage :). ;' USAGE: ;' ;' - put depacker anywhere you want, ;' - put starting address of packed block in HL, ;' - put location where you want data to be depacked in DE, ;' (much like LDIR command, but without BC) ;' - make CALL to depacker (DEC40). ;' - enjoy! ;) ;' PRECAUTIONS: ;' ;' Be very careful if packed and depacked blocks coincide somewhere in memory. ;'Here are some advices: ;' ;' 1. put packed block to the highest addresses possible. ;' Best if last byte of packed block has address #FFFF. ;' ;' 2. Leave some gap between ends of packed and depacked block. ;' For example, last byte of depacked block at #FF00, ;' last byte of packed block at #FFFF. ;' ;' 3. Place nonpackable data to the end of block. ;' ;' 4. Always check whether depacking occurs OK and neither corrupts depacked data ;' nor hangs computer. ;' ;'DEC40 LD A,80h EX AF,AF' MS: LDI M0: LD BC,2FFh M1: EX AF,AF' M1X: ADD A,A JR NZ,M2 LD A,(HL) INC HL RLA M2: RL C JR NC,M1X EX AF,AF' DJNZ X2 LD A,2 SRA C JR C,N1 INC A INC C JR Z,N2 LD BC,33Fh JR M1 X2: DJNZ X3 SRL C JR C,MS INC B JR M1 X6: ADD A,C N2: LD BC,4FFh JR M1 N1: INC C JR NZ,M4 EX AF,AF' INC B N5: RR C JP C, END_DEC40 RL B ADD A,A JR NZ,N6 LD A,(HL) INC HL RLA N6: JR NC,N5 EX AF,AF' ADD A,B LD B,6 JR M1 X3: DJNZ X4 LD A,1 JR M3 X4: DJNZ X5 INC C JR NZ,M4 LD BC,51Fh JR M1 X5: DJNZ X6 LD B,C M4: LD C,(HL) INC HL M3: DEC B PUSH HL LD L,C LD H,B ADD HL,DE LD C,A LD B,0 LDIR POP HL JR M0 END_DEC40: ld a,(23388) ld bc,32765 out (c),a ei END ASM END SUB megaLZDepack (32768,49152)