[porting suggestion] Quake 3 for PSP-Slim (with 64Mb RAM)?

[Be3f]

Finally, PSP-Slim (PSP-200X) is avaluable and it runs homebrew - my congratulations to Team C+D (for the Magic Battery) and to Teem m33 (for the Custom Firmware v3.60)! As you know, slim has 2X more main RAM, than "fat" (PSP-100X) - 64 Megs!

Low RAM (32Megs of system memory with only 24Megs avaluable for programms) was the bottleneck of the PSP-100X, that left no chance to port such huge games as Quake 3 to PSP, but now i can't find any reason, that makes Quake 3 port for the PSP-Slim (only) impossible!
McZonk's Quake-2 PSP port seems to run full singleplayer game on Slim too... :rolleyes:

So, does Slim has 56 free megs of the user memory (64 system - 8 Mb kernel memory)?

I've posted this once at the qj forums, just copy-paste here:

I already have a portable Quake 3 on my PocketPC (Dell Axim x51v) - Q3 was ported to WinCE in 2005 by NoctenWare (port is called Q3CE), it features both software and hw rendering (uses OpenGL-ES) - runs at ~15 average FPS on my x51v with max graphics (hardware rendering)... and it's package contains very interisting thing for us - pakconvert tool whitch reduces content in Quake3 *.PAK files: downsizes textures & gfx(2x), level & model geometry, converts sounds and videos to save RAM & increase perfomance - after running pakconvert scripts, main PAK0.PAK downsizes from 450 to 117 Megs and Q3CE runs on PPCs with only 36Mb free RAM! Also, Q3CE partitions memory automaticly, instead of original Q3 (and com_hunkMegs & com_zoneMegs commands do not work in it...).

This is how Q3CE accelerated runs on my PPC:


(Q3DM3 map, 2 bots deathmatch, max graphics (but downsized data), 19 FPS at the moment)

Unfortunetly, NoctemWare website (noctemware.com) is currently dead, but you can find some info about Q3CE on it's offical forum:

http://www.pdaground.com/forum/viewforum.php?f=14

=Update=

Yesterday I've spoken to the Q3CE developer and asked him for the source code. Here are the latest q3ce v1.1b sources:
http://www.filecrunch.com/file/~euhoj4

[Wally]

ooohhhh this could be interesting :)

Slim only homebrew :D

[Murdock]

I am no coder, but from what you wrote above, isn't it possible to downsize the game (i.e. gfx etc.) any further to make Q3A run on the old PSP, too? I mean of course, it'd look not that nice, but it "should" run.

Or is this 36 MB RAM the least?

[W00fer]

Ehm, what about the Dell Axim processor ?!

The psp still has only 333mhz not 624 mhz like the Dell !

Thats almost twice as much as the psp. It would be fun though if it could be done !

[Be3f]

I am no coder, but from what you wrote above, isn't it possible to downsize the game (i.e. gfx etc.) any further to make Q3A run on the old PSP, too? I mean of course, it'd look not that nice, but it "should" run.

Or is this 36 MB RAM the least?

I've posted this onсe on qj:

About a year ago, PeterM said:

id's way of coding is to have numerous large global variables. Last time I looked, Q3 had over 20MBs of them, which is unfortunately the total amount of free RAM available for use on the PSP...
Unfortunately, Quake 3's global variables use up the rest...

Many coders tried to port Q3 engine to the PSP-100X, but they were unlucky and gave up...

Ehm, what about the Dell Axim processor ?!

The psp still has only 333mhz not 624 mhz like the Dell !

Thats almost twice as much as the psp. It would be fun though if it could be done !

But PSP's GU is much faster, then i2700g in Dell ;)

Also, I've tested some original Quake 3 Arena maps with downsized (2x) textures on the LTE v2.2 engine (it uses PSP-GL for rendering - this is slower then PSP-GU code!): small maps run ~perfect at ~30-60 FPS (PSP was overclocked to 333/166MHz, Vsync turned on...) with mipmapping & lightmaps (but with some, not too heavy rendering artifacts).
Here's a photo of the Q3DM3 map:



...but large maps refuse to start - PSP freezes, while loading... out of RAM or VRAM? Maybe LTE loads all textures & mips into VRAM and freezes, when VRAM gets full? I don't know exactly, but I think, that framebuffer is set to 16 bit - if it's so, we have about 1.3MB of free VRAM for textures, right?

PSMonkeys Iris engine v0.15 also supports Q3BSP and it loads all original Q3A maps, but without textures. So, VRAM must be the main limit, but it is possible to make the engine put into main RAM all textures, that do not fit in VRAM.
Also, LTE precashes everithing while loading - level (and other...) data streaming may save much memory...

[Raphael]

Ehm, what about the Dell Axim processor ?!

The psp still has only 333mhz not 624 mhz like the Dell !

Thats almost twice as much as the psp. It would be fun though if it could be done !

Ugh... I don't understand why people don't get that you can't just compare clock speeds. You are comparing apples and eggs here. The Dell has a ARM cpu (XScale), while the PSP has a MIPS. Though both are RISC, they are different in design and instructions.
Apart from that, the PSP has much more additional hardware, like the GU which easily outplays the Intel 2700g, a second CPU (ME) plus a VFPU for doing all the vector and matrix maths.

...but large maps refuse to start - PSP freezes, while loading... out of RAM or VRAM? Maybe LTE loads all textures & mips into VRAM and freezes, when VRAM gets full?

I doubt the VRAM is the problem, as I'm pretty sure the LTE guys were at least clever enough to swap textures to sysram if they don't fit into VRAM anymore. The real problem is the sysram that runs out. Especially mipmaps take quite some additional space (though they improve rendering speed), plus all the data from the BSP tree and the map mesh. Those easily count up to more than you imagine.

Also, LTE precashes everithing while loading - level (and other...) data streaming may save much memory...

Except that Q3 maps weren't designed for streaming, so that would involve a lot of coding to get done.

[Be3f]

So, it's near impossible to dev Q3 port for the "fat" PSP...

WTH is going on with memory partitions in PSP-Slim RAM?

partition 2: (user)

topaddr = 0x08800000, size = 0x1800000 (24 MB), attr = 0x0F

-this is old one, same with PSP-100X, used for homebrew

and 2 new kernel partitions implemented for UMD-cache in Slim:

Partition 8: (new in the psp slim!)

topaddr = 0x8A000000, size = 0x1C00000 (28 MB), attr = 0x0C

Partition 10: (new in psp slim!)

topaddr = 0x8BC00000, size = 0x400000 (4 MB), attr = 0x0C

So it seems at first the new partitions are for kernel mode use, but i guess they can be unlocked for user mode use with the SetDdrMemoryProtection function :)

If only these 3 partitions could be merged to have 56 free megs of RAM for homebrews...

[J.F.]

I'm watching this myself... I also tried porting Q3 (ioquake3 to be exact), and while it would compile and run, it would just start initializing the subsystems, and then error out as it ran out of memory. While it's possible you could make something play Q3 levels using another heavily modified engine, you won't get Q3A itself going on the old PSP. Too little memory. The new PSP should have enough, and is plenty powerful enough for Q3. This issue of memory partitions needs to be looked into first.

[Vincent_M]

Wow, I hadn't realized that this already happened. I've been a stranger to the homebrew scene for like two weeks now. That's cool that the PSP Slim has double the system ram. At first, I thought it was only going to have more flash memory, but system ram is better by far... I've seen that their's compatibility issues with PSP homebrew running on the PSP Slim. What is that exactly? Does it have anything to do with the firmware or the hardware?

[Be3f]

it would just start initializing the subsystems, and then error out as it ran out of memory. While it's possible you could make something play Q3 levels using another heavily modified engine, you won't get Q3A itself going on the old PSP. Too little memory.

Oh, this words are the absolutely truth! There was such thread: http://forums.ps2dev.org/viewtopic.php?t=8465 :)

Do you have any plans to port Q3 to the Slim, when patritions problem is over? ;)

I've seen that their's compatibility issues with PSP homebrew running on the PSP Slim. What is that exactly? Does it have anything to do with the firmware or the hardware?

Unfortunetly, 3.60-m33 custom firmware hasn't got the 1.50 FW part (like all previous Custom FWs for PSP-100X had to run homebrews in the 1.50 kernel mode) - only 3.60 part, so most of the homebrews, compiled for 1.50 currently don't work on Slim :( Team m33 says, they tried to implement 1.50 part into 3.60 m33, but got lots of errors (no display and other...). Let us hope, that one day 1.50 part will appear in custom FWs for Slim... And also someone will find a way to use 56 Megs of free RAM for 3.XX homebrews ;)

[Be3f]

Yesterday I've spoken to the Q3CE developer and asked him for the source code. Here are the latest q3ce v1.1b sources:
http://files.filefront.com/q3ce+11b+src ... einfo.html

[Be3f]

sorry, the q3ce-src archive in the previous post is corrupt :( here is the fixed link:
http://www.filecrunch.com/file/~euhoj4

[PeterM]

Thanks - good to have this file around now that Noctemware vanished. :-)

[J.F.]

Thanks - good to have this file around now that Noctemware vanished. :-)

Actually, I already had it. You can find it on the net with about 10 to 15 minutes of googling. It wasn't "lost", just misplaced. :)

[Be3f]

Finally, custom FW 3.71 is out and RAM partitions problem is over:

- Psp Slim: umdcache was allocating memory even when homebrew was launched, wasting
memory that homebrew programs may want to use. Now umdcache module is stopped before
it can allocate any memory, only in the case homebrew is launched.

Also, memory is unprotected for user memory usage by M33 core (only when homebrew is launched).
Developer, for a sample of how to use the extra memory, see the extra ram sample of the M33 sdk.

Included is a sample that uses the 32 MB of extra ram of the psp slim.

[SamuraiX]

It is great news to have the additional 32 MBytes of ram available. But now it requires some sort of memory manager to fully utilize the extra ram. Would it be possible to integrate the additional memory when using malloc() and not require us to generate our own memory management schemes?

[Be3f]

It is great news to have the additional 32 MBytes of ram available. But now it requires some sort of memory manager to fully utilize the extra ram. Would it be possible to integrate the additional memory when using malloc() and not require us to generate our own memory management schemes?

well...

Techincal stuff, the memory protection of te extra slim memory is in these hardware registers: 0xbc000040-0xbc00007F.

Set all to 0xFF, and you have access to it in user mode (note: sceKernelSetDdrMemoryProtection wouldn't do it)., but anyways is done automatically by m33 when launching homebrew, so no need to care about that.

Visit this thread for details and view include dir in (a sample) the 3.71-m33 SDK ;)

[SamuraiX]

Using:

Code: Select all

u8 *buf = (u8 *)0x0a000000;

is not the same as doing buf = malloc((32*1024*1024) + (20*1024*1024)); and having it return a buf with a size of 52 Mbytes.



Otherwise we are going to have to do some serious memory management not that I'm against challenges. ;p

[J.F.]

Using:

Code: Select all

u8 *buf = (u8 *)0x0a000000;

is not the same as doing buf = malloc((32*1024*1024) + (20*1024*1024)); and having it return a buf with a size of 52 Mbytes.



Otherwise we are going to have to do some serious memory management not that I'm against challenges. ;p

You want an adaptation of the valloc code Raphael did.

[TyRaNiD]

The trouble is the malloc code assumes a contiguous block of memory to work on as its heap. As the PSP still keeps its stacks at around 09XXXXXX there is no contiguous run up to the new memory for malloc to work.

What would have beem nice would be if partition 2 had actually been extended in sysmem to encompass the entire memory region then the kernel would have allocated stacks at top end of that region giving you a nice 50+ meg contiguous block which you would have been able to allocate quite easily :P

Of course it still might be possible to do with a new operating mode (i.e. have a compatibility mode where things are the same as before but with the extra ram maybe, and a super mode with one large memory space). Still probably too late as now people will write using absolute pointers :)

[J.F.]

I think the best thing to do would be to write your own malloc that used regular malloc until it returned 0, then pass on to new routine based on valloc but with the new memory as the base. It wouldn't be that hard. You wouldn't get a block larger than ~32MB, but you'd get all 56MB in chunks... which is how most apps use memory anyway.

[PeterM]

Indeed, and using something like Doug Lea's malloc to handle the hard stuff would help you out considerably.

[J.F.]

Indeed, and using something like Doug Lea's malloc to handle the hard stuff would help you out considerably.

Or maybe ptmalloc. http://www.malloc.de/en/

[TyRaNiD]

Perhaps just dont use malloc in the first place ? :)

[J.F.]

Perhaps just dont use malloc in the first place ? :)

Some programs have their own memory allocation handling - you just give it a pointer to a block of X amount of memory, and it does the rest. Most of the id programs are that way. In that case, just give it the address of the 32MB block, and it'll do all the memory handling on its own. I wonder if that's enough for Q3A... I'll have to try that.

[PeterM]

Sounds doable.

[Cpasjuste]

I noticed that when i reboot psplink in vsh mode, i get this :

Code: Select all

host0:/> meminfo
Memory Partitions:
N |    BASE    |   SIZE   | TOTALFREE |  MAXFREE  | ATTR |
--|------------|----------|-----------|-----------|------|
1 | 0x88000000 |  6291456 |   2270464 |   1866496 | 000C |
2 | 0x08800000 | 50331648 |  24760576 |  23950592 | 000F |
3 | 0x88000000 |  6291456 |   2270464 |   1866496 | 000C |
4 | 0x88600000 |  2097152 |   2097152 |   2097152 | 000C |
5 | 0x0B800000 |  8388608 |   8388608 |   8388608 | 000F |

What does that mean ? Is there a way to have a continus memory parition by this way ?

[TyRaNiD]

Well as pointed out you could do something similar in normal game mode to allocate all of ram in one massive chunk of memory in partition 2 but that requires firmware support as early as possible which probably isn't going to happen any time soon.

[moonlight]

What would have beem nice would be if partition 2 had actually been extended in sysmem to encompass the entire memory region then the kernel would have allocated stacks at top end of that region giving you a nice 50+ meg contiguous block which you would have been able to allocate quite easily :P

Well, if i ever continue with this shit, I may try that.

[SamuraiX]

Hate to bring up an old topic but it seems Moonlight has incorporated the addition 32 MBytes to Malloc() when using his SDK for M33-3 (Looking at sample memory test).

This is great news, meaning if you include the SDK with your current project you will now have direct malloc access to the additional memory.

I can't wait to incorporate support for additional memory on my project (OpenBOR) now!


Thank You Moonlight!!!