genuine PSP font with intraFont & PGF binary format

[Raphael]

Allow me to be more specific. I have depth testing disabled. I assume if I leave it disabled I don't need to allocate memory for it?

Wrong. Disabling Depth Testing just disables depth reading (and the compare), not necessarily the depth writing (this is done with sceGuDepthMask). If you don't set the depthbuffer yourself, sceGu library just puts it after the drawbuffer. This would explain why your textures are messed up when you don't allocate the depth buffer.

[CpuWhiz]

Ok, so do it use sceGuDepthMask(1) and disable depth testing to properly rid myself of the depth buffer? Also, I added the depth buffer back in and I still have this issue.

[J.F.]

If you have the depth test disabled, be sure not to clear the depth buffer. You see this in the drawing loop in the sample:

Code: Select all

	sceGuClear(GU_COLOR_BUFFER_BIT|GU_DEPTH_BUFFER_BIT);

That tells it to clear the drawing buffer and the depth buffer. Don't clear the depth buffer if you have it disabled and are using the space for textures.

[CpuWhiz]

I wasn't clearing the depth buffer.

[J.F.]

Well, without any code to look at, all we can do is guess at the problem. You either need to post some code or fix it yourself.

[CpuWhiz]

Edit: As a lame workaround to my problem, I expressed my sceGuCopyImage call for the indexed texture in terms of GU_PSM_8888 so that it would add up to the proper amount of bytes. This is not ideal but seems to solve my problem.

Code: Select all

sceGuCopyImage(GU_PSM_8888, 0, 0, 64, 32, 64, texture2, 0, 0, 64, vram2);

Ok, here you go. This is not my program per-se, but I think it's showing the same problem. So if you can help me fix it with my sample I might be able to get it fixed in my app too. Now on my PSP there is crud in the top left corner of the screen that shouldn't be there, but if you swap the sceGuCopyImage calls to copy the indexed texture first, the crud goes away. Is this just me?

Code: Select all

/*
 * PSP Software Development Kit - http://www.pspdev.org
 * -----------------------------------------------------------------------
 * Licensed under the BSD license, see LICENSE in PSPSDK root for details.
 *
 * Copyright (c) 2005 Jesper Svennevid
 */

#include <pspkernel.h>
#include <pspdisplay.h>
#include <pspdebug.h>
#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <string.h>
#include <time.h>

#include <pspgu.h>

PSP_MODULE_INFO&#40;"CLUT Test", 0, 1, 1&#41;;
PSP_MAIN_THREAD_ATTR&#40;THREAD_ATTR_USER&#41;;
PSP_HEAP_SIZE_MAX&#40;&#41;;

static unsigned int __attribute__&#40;&#40;aligned&#40;16&#41;&#41;&#41; list&#91;262144&#93;;
static unsigned int __attribute__&#40;&#40;aligned&#40;16&#41;&#41;&#41; clut&#91;16&#93;;

struct Vertex
&#123;
	float u,v;
	float x,y,z;
&#125;;

#define BUF_WIDTH &#40;512&#41;
#define SCR_WIDTH &#40;480&#41;
#define SCR_HEIGHT &#40;272&#41;

static int exitRequest = 0;

int running&#40;&#41;
&#123;
	return !exitRequest;
&#125;

int exitCallback&#40;int arg1, int arg2, void *common&#41;
&#123;
	exitRequest = 1;
	return 0;
&#125;

int callbackThread&#40;SceSize args, void *argp&#41;
&#123;
	int cbid;

	cbid = sceKernelCreateCallback&#40;"Exit Callback", exitCallback, NULL&#41;;
	sceKernelRegisterExitCallback&#40;cbid&#41;;

	sceKernelSleepThreadCB&#40;&#41;;

	return 0;
&#125;

int setupCallbacks&#40;void&#41;
&#123;
	int thid = 0;

	thid = sceKernelCreateThread&#40;"update_thread", callbackThread, 0x11, 0xFA0, 0, 0&#41;;
	if&#40;thid >= 0&#41;
	&#123;
		sceKernelStartThread&#40;thid, 0, 0&#41;;
	&#125;

	return thid;
&#125;

int main&#40;int argc, char* argv&#91;&#93;&#41;
&#123;
	setupCallbacks&#40;&#41;;

	void* fbp0 = &#40;void*&#41;0x00000000;
	void* fbp1 = &#40;void*&#41;0x00088000;
	void* zbp =  &#40;void*&#41;0x00110000;

	void* vram1 = &#40;void*&#41;0x04154000;
	void* vram2 = &#40;void*&#41;0x04156000;

	int i, x, y;
	unsigned char r, g, b, a = 255;
	u8  *texture1 = malloc&#40;128 * 128 / 2&#41;;
	u32 *texture2 = malloc&#40;256 * 256 * 4&#41;;

	
	for&#40;i = 0; i < 16; i++&#41;
		clut&#91;i&#93; = 0xFF000000 | &#40;&#40;i * 17&#41; << 16&#41; | &#40;&#40;i * 17&#41; << 8&#41; | &#40;i * 17&#41;;

	for&#40;y = 0; y < 128; y ++&#41;
		for&#40;x = 0; x < 64; x++&#41;
			texture1&#91;y * 64 + x&#93; = 0; // &#40;&#40;x % 16&#41; << 4&#41; | &#40;x % 16&#41;;

	for&#40;y = 0; y < 256; y++&#41;
	&#123;
		g = 255 - y;
		for&#40;x = 0; x < 256; x++&#41;
		&#123;
			r = x;
			b = 255 - x;
			texture2&#91;y * 256 + x&#93; = &#40;a << 24&#41; | &#40;b << 16&#41; | &#40;g << 8&#41; | r;
		&#125;
	&#125;

	sceKernelDcacheWritebackAll&#40;&#41;;

	sceGuInit&#40;&#41;;
	sceGuStart&#40;GU_DIRECT,list&#41;;

	sceGuDrawBuffer&#40;GU_PSM_8888, fbp0, BUF_WIDTH&#41;;
	sceGuDispBuffer&#40;SCR_WIDTH, SCR_HEIGHT, fbp1, BUF_WIDTH&#41;;
	sceGuDepthBuffer&#40;zbp, BUF_WIDTH&#41;;
	sceGuOffset&#40;2048 - &#40;SCR_WIDTH/2&#41;, 2048 - &#40;SCR_HEIGHT/2&#41;&#41;;
	sceGuViewport&#40;2048, 2048, SCR_WIDTH, SCR_HEIGHT&#41;;
	sceGuDepthRange&#40;65535, 0&#41;;
	sceGuScissor&#40;0, 0, SCR_WIDTH, SCR_HEIGHT&#41;;
	sceGuEnable&#40;GU_SCISSOR_TEST&#41;;
	sceGuFrontFace&#40;GU_CW&#41;;
	sceGuEnable&#40;GU_TEXTURE_2D&#41;;
	sceGuEnable&#40;GU_BLEND&#41;;
	sceGuDisable&#40;GU_DEPTH_TEST&#41;;
	sceGuClutMode&#40;GU_PSM_8888, 0, 255, 0&#41;;
	sceGuClutLoad&#40;2, clut&#41;;
	sceGuClear&#40;GU_COLOR_BUFFER_BIT | GU_DEPTH_BUFFER_BIT&#41;;
	sceGuFinish&#40;&#41;;
	sceGuSync&#40;0,0&#41;;

	sceDisplayWaitVblankStart&#40;&#41;;
	sceGuDisplay&#40;GU_TRUE&#41;;

	int once = 1;

	while&#40;running&#40;&#41;&#41;
	&#123;
		sceGuStart&#40;GU_DIRECT,list&#41;;

		if&#40;once&#41;
		&#123;
			// Copy the non-indexed image
			sceGuCopyImage&#40;GU_PSM_8888, 0, 0, 256, 256, 256, texture2, 0, 0, 256, vram2&#41;;

			// Copy the indexed image
			sceGuCopyImage&#40;GU_PSM_T4, 0, 0, 128, 128, 128, texture1, 0, 0, 128, vram1&#41;;
	
			// Sync the copied textures
			sceGuTexSync&#40;&#41;;

			once = 0;
		&#125;

		sceGuColor&#40;0xFFFFFFFF&#41;;
		sceGuClearColor&#40;0xFF333333&#41;;
		sceGuClear&#40;GU_COLOR_BUFFER_BIT | GU_DEPTH_BUFFER_BIT&#41;;

		sceGuTexMode&#40;GU_PSM_8888, 0, 0, 0&#41;;
		sceGuTexImage&#40;0, 256, 256, 256, vram2&#41;;
		sceGuTexFunc&#40;GU_TFX_REPLACE, GU_TCC_RGBA&#41;;
		sceGuTexFilter&#40;GU_LINEAR, GU_LINEAR&#41;;
		sceGuTexScale&#40;1.0f, 1.0f&#41;;
		sceGuTexOffset&#40;0.0f, 0.0f&#41;;

		// Non-indexed Image
		struct Vertex* v = &#40;struct Vertex*&#41;sceGuGetMemory&#40;2 * sizeof&#40;struct Vertex&#41;&#41;;
		v&#91;0&#93;.u = 0;
		v&#91;0&#93;.v = 0;
		v&#91;0&#93;.x = 0;
		v&#91;0&#93;.y = 0;
		v&#91;0&#93;.z = 0;

		v&#91;1&#93;.u = 256;
		v&#91;1&#93;.v = 256;
		v&#91;1&#93;.x = 256;
		v&#91;1&#93;.y = 256;
		v&#91;1&#93;.z = 0;

		sceGuDrawArray&#40;GU_SPRITES, GU_TEXTURE_32BITF | GU_VERTEX_32BITF | GU_TRANSFORM_2D, 2, 0, v&#41;;

		sceGuTexMode&#40;GU_PSM_T4, 0, 0, 0&#41;;
		sceGuTexImage&#40;0, 128, 128, 128, vram1&#41;;
		sceGuTexFunc&#40;GU_TFX_REPLACE, GU_TCC_RGBA&#41;;
		sceGuTexFilter&#40;GU_LINEAR, GU_LINEAR&#41;;
		sceGuTexScale&#40;1.0f, 1.0f&#41;;
		sceGuTexOffset&#40;0.0f, 0.0f&#41;;

		// Indexed Image
		v = &#40;struct Vertex*&#41;sceGuGetMemory&#40;2 * sizeof&#40;struct Vertex&#41;&#41;;
		v&#91;0&#93;.u = 0;
		v&#91;0&#93;.v = 0;
		v&#91;0&#93;.x = 64;
		v&#91;0&#93;.y = 64;
		v&#91;0&#93;.z = 0;

		v&#91;1&#93;.u = 128;
		v&#91;1&#93;.v = 128;
		v&#91;1&#93;.x = 192;
		v&#91;1&#93;.y = 192;
		v&#91;1&#93;.z = 0;

		sceGuDrawArray&#40;GU_SPRITES, GU_TEXTURE_32BITF | GU_VERTEX_32BITF | GU_TRANSFORM_2D, 2, 0, v&#41;;

		sceGuFinish&#40;&#41;;
		sceGuSync&#40;0,0&#41;;

		sceDisplayWaitVblankStart&#40;&#41;;
		sceGuSwapBuffers&#40;&#41;;
	&#125;

	free&#40;texture1&#41;;
	free&#40;texture2&#41;;

	sceGuTerm&#40;&#41;;
	sceKernelExitGame&#40;&#41;;

	return 0;
&#125;

Code: Select all

TARGET = clut_test
OBJS = main.o

INCDIR =
CFLAGS = -G0 -Wall -O2
CXXFLAGS = $(CFLAGS) -fno-exceptions -fno-rtti
ASFLAGS = $(CFLAGS)

LIBDIR =
LDFLAGS =
LIBS= -lpspgu

BUILD_PRX = 1
EXTRA_TARGETS = EBOOT.PBP
PSP_EBOOT_TITLE = CLUT Test
PSP_FW_VERSION = 371

PSPSDK=$(shell psp-config --pspsdk-path)
include $(PSPSDK)/lib/build.mak

[StrmnNrmn]

There seems to be another issues relating to caching when drawing lots of text. I get a lot of 'noise' as Daedalus's UI is rendering lots of text. Notice the text for 'Boot' and 'Country' in the two screenshots below:




(hope imageshack hosted images are ok...)

The affected characters mostly stay the same, but the corruption changes, making the characters appear to flicker. It would actually make quite a cool 'decrypting-text-in-the-movies' effect if it was intentional :)

I'm not sure what exactly is going on, but as I mentioned initially it looks like the cache is somehow getting messed up over the course of a frame. I'm not sure if there's some kind of timing issue involved - like maybe a sync or flush is needed in the display list generation. I'll investigate a bit more tomorrow.

[BenHur]

There seems to be another issues relating to caching when drawing lots of text. I get a lot of 'noise' as Daedalus's UI is rendering lots of text.

Just for narrowing down the cause: have you tried INTRAFONT_CACHE_ALL or INTRAFONT_CACHE_LARGE when loading the fonts with intraFontLoad? The former finalizes the cache, so it's not touched anymore after the font has been loaded. The later provides a cache that's large enough so no chars will have to be replaced when using ltn*.pgf fonts.

If those flags improve rendering, there could be an issue with the cache, otherwise you probably have to look somewhere else. (You could also write the font->texture to a file and see if that shows signs of corruption.)

Looks great already!
BenHur

[Raphael]

A cachewriteback at the proper position might solve that issue. Not sure yet where that might be.

EDIT: On another note, it might be (seeing how only a few out of that lot of text drawn are messed up) that the artifacts are caused by hitting the font texture 'wrap' boundary (unless you use CACHEALL), where glyphs in the cache get overwritten before they were actually rendered by the GU.
In that case a sync to GU before printing new strings should help.

[BenHur]

Should pspFont library be low-level and only provide the ability to render the glyphs or strings into an image buffer (no dependency on sceGu) or have an inbuilt back-end for the rasterization too (like intraFont currently does)?

Just an idea if you're heading towards the first approach: freetype (at least the original from freetype.org) allows to "select a specific driver for opening", i.e. you can write your own FT_Open_Face driver and extend freetype to use pgf fonts as well. Since freetype is built quite modular, I don't think you have to dig deep. Then one could use standard pgeFont or similar for the rasterization, etc. This way every homebrew built upon freetype could easily be converted to use the internal font by just changing the font filename and maybe an additional line to initialize the pgf-module.

I don't know enough about freetype, though, to implement something like that. Just an idea...
Cheers, BenHur

PS: And Now For Something Completely Different: To those who wonder what characters are available in all these pgf files - you can now look it up on http://www.psp-programming.com/benhur

[Raphael]

Well, the reason for the first solution was to keep pspFont free of all dependencies (especially pspgu), so building it as a driver for FreeType is pretty much contradicting that idea. Anyway I wouldn't really have the head for digging into FreeType that much currently :P

I'm currently in the process of integrating the file parsing and glyph rendering code from intraFont. I fell back on schedule cause I caught a cold last week, but I'm up on it again.

Only problem I'm thinking about atm is how to properly handle glyph shadows. Either I'm providing a functionality to choose whether to grab the glyph OR the shadow image, then render them both using GU (as done in intraFont), or I write some software code that mixes them both together into one glyph image. Only thing I'm not sure with that method is how to properly handle the shadow scales (your document states 0.5 and 1.0 are typical values, but did you encounter others as well?), so I'm currently aiming for the first solution.

PS: I like your PGF documentation :) I'm asking myself how you came to the 99.34% number? :D

[BenHur]

Only problem I'm thinking about atm is how to properly handle glyph shadows. Either I'm providing a functionality to choose whether to grab the glyph OR the shadow image, then render them both using GU (as done in intraFont), or I write some software code that mixes them both together into one glyph image. Only thing I'm not sure with that method is how to properly handle the shadow scales (your document states 0.5 and 1.0 are typical values, but did you encounter others as well?), so I'm currently aiming for the first solution.

I only encountered shadowscales of 1.0 (old firmwares such as 1.5) and 0.5 (new firmwares), this was probably done to reduce filesize and the scaling artefacts can't really be seen anyway. But I don't think shadow scales are the issue: shadows usually overlap the preceeding char - that's why in intraFont i have to plot the shadows first and then the chars on top of it. If you can avoid the shadows darkening the preceeding glpyh during rendering (which sounds a lot more complicated to me), only then would i consider the second option also.

PS: I like your PGF documentation :) I'm asking myself how you came to the 99.34% number? :D

Every bit and byte that has an unknown meaning (???) added together is 0.66% of the total file size :-)

Cheers, BenHur

[Raphael]

I only encountered shadowscales of 1.0 (old firmwares such as 1.5) and 0.5 (new firmwares), this was probably done to reduce filesize and the scaling artefacts can't really be seen anyway.

In that case it would be rather easy to implement a software stretch that does the 0.5 scale. I was just scared of having to write a full bilinear filter in software - not because it would be hard to do, but it would crumble the speed.

But I don't think shadow scales are the issue: shadows usually overlap the preceeding char - that's why in intraFont i have to plot the shadows first and then the chars on top of it. If you can avoid the shadows darkening the preceeding glpyh during rendering (which sounds a lot more complicated to me), only then would i consider the second option also.

Well, I thought about that point too and it's only an issue when having to rasterize whole strings - in which case the easy solution would be rasterizing all shadows first, then the glyphs. Not really complicated :)


Here's the prototype for my current interface. I'm thinking about adding a flag parameter to the pspFontGet*Size/Image functions that decide whether to use the glyph, shadow or both (and maybe some other flags for later use).
Anyway, it's open for discussion - I'd like to get as much feedback as possible before I finished the actual implementation, to avoid having to change a lot of code later on. The font cache will be handled completely internal and cache single glyph images only (similar to how intraFont does it, just with a hased MFU ordering as well as a character code cache).

/*
* PSP Software Development Kit - http://www.pspdev.org
* -----------------------------------------------------------------------
* Licensed under the BSD license, see LICENSE in PSPSDK root for details.
*
* pspfont.h - Prototypes for pspFont library.
*
* Copyright (c) 2007 BenHur
* Copyright (c) 2007 Alexander Berl <[email protected]>
*
* $Id$
*/
#ifndef PSPFONT_H
#define PSPFONT_H

#ifdef __cplusplus
extern "C" {
#endif

/** @defgroup Font Font User Library */

/** @addtogroup Font */

/*@{*/


typedef struct
{
int pixelFormat;
int x, y;
int width;
int height;
int pitch;
char* data;
char* palette;
} pspFontImageBuffer;


typedef struct
{
int width;
int height;
} pspFontRect;


typedef struct
{
unsigned char width;
unsigned char height;
char left;
char top;
char vAdvance;
char hAdvance;
} pspFontGlyphInfo;


typedef struct
{
pspFontGlyphInfo minGlyhpInfo;
pspFontGlyphInfo maxGlyhpInfo;
int numChars;
int numShadows;

short* charMap;

char family[64];
char style[64];
} pspFontInfo;



/**
* Initialize the font library and the cache facility
*
* @returns An error code if less than 0.
*/
int pspFontInit();

/**
* Finish use of the font library. This frees all resources.
*
* @returns An error code if less than 0.
*/
int pspFontDone();


enum pspFontFlags
{
PSP_FONT_FLAG_DEFAULT = 0x0000,
PSP_FONT_FLAG_NOSHADOW = 0x0001,
PSP_FONT_FLAG_NOCACHE = 0x0002,
PSP_FONT_FLAG_PRECACHE = 0x8000
};

/**
* Load a .PGF font from a file.
*
* @param filename - Filename of the file to load the font from.
* @param flags - A combination of ::pspFontFlags
*
* @returns A font ID on success, an error code if less than 0.
*/
int pspFontLoad( const char* filename, unsigned int flags );

/**
* Load a .PGF font from memory.
*
* @param buffer - Memory location to load the font from.
* @param size - Size of the memory buffer holding the font.
* @param flags - A combination of ::pspFontFlags
*
* @returns A font ID on success, an error code if less than 0.
*/
int pspFontLoadMem( const void* buffer, unsigned int size, unsigned int flags );

/**
* Unload a previously loaded font and free it's resources.
*
* @param fontId - The ID of the font to unload.
*
* @returns An error code if less than 0.
*/
int pspFontUnload( int fontId );


int pspFontSetColor( int fontId, unsigned int color );
int pspFontSetShadowColor( int fontId, unsigned int color );

int pspFontSetSpacing( int fontId, int spacing );



#define PSP_FONT_MODE_ADVANCE_MASK 0x000F
#define PSP_FONT_MODE_ALIGN_MASK 0x00F0
#define PSP_FONT_MODE_WIDTH_MASK 0x0F00
#define PSP_FONT_MODE_WRAP_MASK 0xF000

#define PSP_FONT_MODE_ADVANCE_H 0x0000
#define PSP_FONT_MODE_ADVANCE_V 0x0001
#define PSP_FONT_MODE_ALIGN_LEFT 0x0000
#define PSP_FONT_MODE_ALIGN_CENTER 0x0010
#define PSP_FONT_MODE_ALIGN_RIGHT 0x0020
#define PSP_FONT_MODE_WIDTH_VAR 0x0000
#define PSP_FONT_MODE_WIDTH_FIX 0x0100
#define PSP_FONT_MODE_WRAP_DEFAULT 0x0000
#define PSP_FONT_MODE_WRAP_LINE 0x1000
#define PSP_FONT_MODE_WRAP_NONE 0x2000

int pspFontSetMode( int fontId, int mode );


int pspFontGetFontInfo( int fontId, pspFontInfo* info );

int pspFontGetGlyphInfo( int fontId, short code, pspFontGlyphInfo* info );


/**
* Calculate the required size of a single char glyph image.
*
* @param fontId - Font ID specifying which font to use.
* @param code - The character code (UCS-2).
* @param rect - A pointer to a pspFontRect structure to hold the size information.
*
* @returns An error code if less than 0.
*/
int pspFontGetCharSize( int fontId, short code, pspFontRect* rect );

int pspFontGetCharImage( int fontId, short code, pspFontImageBuffer* buffer );



/**
* Calculate the required size of a string's glyphs image.
*
* @param fontId - Font ID specifying which font to use.
* @param string - The string (UCS-2).
* @param rect - A pointer to a pspFontRect structure to hold the size information.
*
* @returns An error code if less than 0.
*/
int pspFontGetStringSize( int fontId, short* string, pspFontRect* rect );

int pspFontGetStringImage( int fontId, short* string, pspFontImageBuffer* buffer );



/*@}*/

#ifdef __cplusplus
}
#endif

#endif /* PSPFONT_H */

One function I'm thinking about adding too, is a CreateFontTexture() function that generates a font texture of a set of characters (given as parameter), similar to the way pgeFont creates it's texture. That way, an overlying wrapper to a simple render function can be done in only a few lines of code (at the cost having only a fixed number of glyphs in the texture).

[BenHur]

Well, I thought about that point too and it's only an issue when having to rasterize whole strings - in which case the easy solution would be rasterizing all shadows first, then the glyphs. Not really complicated :)

Sorry, I don't get it - maybe it's because of the word "rasterizing". At what level are you talking of rasterizing? Putting the glyphs into the cache? Using the cache to write onto an image/texture/screen for the user? Don't you usually rasterize whole strings? What do you keep in cache: char and shadow seperately or char&shadow combined?
If you plan to combine char&shadow: Do you use 8bpp so you can still draw char and shadow at whatever color you like? Another note on this topic: Keep in mind that there can be more than 10'000 chars but never more than 512 shadows - and those are usually even downscaled. A char&shadow combined uses up to 4 times as much space in the cache than a char and its shadow seperately (the shadow uses no additional space once it has been used before and it can be left downscaled). ltn0.pgf is a bad example since every char has its own shadow ("only" factor of 2 more mem needed for combining char&shadow), but jpn0.pgf is the other extreme (up to factor of 4).

Here's the prototype for my current interface. I'm thinking about adding a flag parameter to the pspFontGet*Size/Image functions that decide whether to use the glyph, shadow or both (and maybe some other flags for later use).
Anyway, it's open for discussion - I'd like to get as much feedback as possible before I finished the actual implementation, to avoid having to change a lot of code later on. The font cache will be handled completely internal and cache single glyph images only (similar to how intraFont does it, just with a hased MFU ordering as well as a character code cache).

sounds exciting! Does that mean you have to allocate the max. width and height for each glyph? Also: charmaps can be quite large (up to 128 kB) and quite empty. Plans for (future) charmap compression?
(Maybe it's too early for mem-optimizations and I don't have a clue on proper software development.)

One function I'm thinking about adding too, is a CreateFontTexture() function that generates a font texture of a set of characters (given as parameter), similar to the way pgeFont creates it's texture. That way, an overlying wrapper to a simple render function can be done in only a few lines of code (at the cost having only a fixed number of glyphs in the texture).

I think this is a usefull function. But I don't know about the character set parameter: since ltn*.pgf only has 324 glyphs for intraFont I thought "what the heck, let's just pre-cache everything" (INTRAFONT_CACHE_ALL) and got rid of that character set parameter. And for jpn0.pgf the requested character set is usually too large to fit into a single texture anyway. (In intraFont one could cache a subset by simply printing that character set with (shadow)color=0)

Sorry, if this post became something of a rant - it wasn't supposed to be. I like the prototype (just have a couple of concerns, but might be off/wrong). I hope you get a lot of input from other users... hint, hint ;-)

Cheers, BenHur

Edit: another note: for proper font rendering sub-pixel resolution is usually required. pgf-files provide 1/64, intraFont currently uses 1/4 pixel resolution. In your implementation I see char hAdvance and int *GetSize. Any plans for sub-pixel resolution?

[Raphael]

Sorry, I don't get it - maybe it's because of the word "rasterizing". At what level are you talking of rasterizing? Putting the glyphs into the cache? Using the cache to write onto an image/texture/screen for the user? Don't you usually rasterize whole strings? What do you keep in cache: char and shadow seperately or char&shadow combined?

Well, I keep using the word wherever I find it "appropriate" in its basic meaning. So in this case I'm talking about the rasterization of chars and strings equally (since the algorithm will be the same - the string rasterizer will basically just call the glyph rasterizer multiple times and do the correct adjusts). What I'm planning on doing is rasterizing the glyphs from the font file into a cache image and from there directly into the user image (basically a memcpy with stride) - or if the user want's to bypass the cache (due to having his own caching facility or whatever reason) just rasterize directly into the user image.
Currently I'm thinking about keeping chars and shadows seperate in cache, for the simple reason that I also aim at the possibility to choose whether to only render the glyphs or shadows (or both together). So keeping char+shadow in cache would be a bad idea in case the user swaps between the modes. It will cause a little cache and render overhead in the case that the user only uses char+shadow combinations, but that shouldn't be that noticeable.

If you plan to combine char&shadow: Do you use 8bpp so you can still draw char and shadow at whatever color you like?

Well, my first idea was a method similar to what I did in my font engine I wrote for PMP Mod - I used 4bit textures and used the first 8 indices for the border (here shadow) color and the following 8 for the actual font color, smoothly blending between. The results were very acceptable, but extending to 8bpp is an option indeed.

Another note on this topic: Keep in mind that there can be more than 10'000 chars but never more than 512 shadows - and those are usually even downscaled. A char&shadow combined uses up to 4 times as much space in the cache than a char and its shadow seperately (the shadow uses no additional space once it has been used before and it can be left downscaled). ltn0.pgf is a bad example since every char has its own shadow ("only" factor of 2 more mem needed for combining char&shadow), but jpn0.pgf is the other extreme (up to factor of 4).

See above :)

sounds exciting! Does that mean you have to allocate the max. width and height for each glyph? Also: charmaps can be quite large (up to 128 kB) and quite empty. Plans for (future) charmap compression?
(Maybe it's too early for mem-optimizations and I don't have a clue on proper software development.)

No, basically the functionality should be like this:
pspFontGlyphGetSize( fontID, code, &sizeRect );
// allocate min sizeRect.width*sizeRect.height/2 bytes
// create pspFontImageBuffer structure with data pointing to that memory
// and x, y being the position in that buffer to start drawing at (topleft)
pspFontGlyphGetImage( fontID, code, &pspFontImageBuffer );

So you can have glyphs rendered into arbitrary sized image buffers at any position. With this, you could easily create a simple font texture yourself.

Regarding charmap compression I'm not sure yet - I'll think about that when it becomes an issue ;)

I think this is a usefull function. But I don't know about the character set parameter: since ltn*.pgf only has 324 glyphs for intraFont I thought "what the heck, let's just pre-cache everything" (INTRAFONT_CACHE_ALL) and got rid of that character set parameter. And for jpn0.pgf the requested character set is usually too large to fit into a single texture anyway.

Well, I managed that problem in my PMP mod font engine by creating multiple texture pages. Each page held up to 256 different char glyphs (I was using 256x256 textures) and the choice on which page to use was decided by a hash function on the charcode. That way I held all character glyphs in cache. The end-user could just create his own charmap that maps a charcode to a page.
The main thought use for that function though was simple systems for latin fonts, so one could easily create a texture for a reduced charset like [0-9A-Z_,.!"], as is mainly used in games.
For japanese and korean for example I'd always advise to use the "normal" way of creating your strings directly through pspFont.

Sorry, if this post became something of a rant - it wasn't supposed to be. I like the prototype (just have a couple of concerns, but might be off/wrong). I hope you get a lot of input from other users... hint, hint ;-)

Cheers, BenHur

Don't worry, I like all kinds of feedback and your's really didn't sound like a rant :)

Edit: another note: for proper font rendering sub-pixel resolution is usually required. pgf-files provide 1/64, intraFont currently uses 1/4 pixel resolution. In your implementation I see char hAdvance and int *GetSize. Any plans for sub-pixel resolution?

Not sure yet, but seeing how this is a bitmap font and the only parts where subpixel resolution would matter in any way are the pspFontImageBuffer's x/y coordinates and the glyphInfo advance values I'll probably just stick with giving those values in subpixel accuracy.

[StrmnNrmn]

There seems to be another issues relating to caching when drawing lots of text. I get a lot of 'noise' as Daedalus's UI is rendering lots of text.

Just for narrowing down the cause: have you tried INTRAFONT_CACHE_ALL or INTRAFONT_CACHE_LARGE when loading the fonts with intraFontLoad?

...

If those flags improve rendering, there could be an issue with the cache, otherwise you probably have to look somewhere else.

Loading the font with INTRAFONT_CACHE_ALL did fix the rendering glitch.

A cachewriteback at the proper position might solve that issue. Not sure yet where that might be.

EDIT: On another note, it might be (seeing how only a few out of that lot of text drawn are messed up) that the artifacts are caused by hitting the font texture 'wrap' boundary (unless you use CACHEALL), where glyphs in the cache get overwritten before they were actually rendered by the GU.
In that case a sync to GU before printing new strings should help.

After a little playing around, I think your second suggestion is correct, i.e. the glyphs in the cache are overwritten before they're rendered by the GU.

I was worried that different strings were being rendered each frame as I scrolled through the list, so I turned off all my clipping, rendering all the text every frame. With that, I noticed that the glitches only seem to occur when a preview picture is being rendered (i.e. the Super Mario 64 preview in my screenshots above), and disappear when no preview is being displayed.

To be honest I would have thought rendering the preview would have helped reduce the glitching, as I would expect it to flush the intra font texture from the GU texture cache. I guess these kinds of caching issues are never all that intuitive though :)

INTRAFONT_CACHE_ALL is a good enough solution for me - I can live with the 128KB it requires for the texture. As Raphael suggests, it looks like there might be some kind of explicit GU sync needed. I suggest this is better implemented at the intraFont level rather than the client code, as it's not necessary for every string that's rendered, just when intraFont decides to update its texture cache.

Here's the prototype for my current interface.
...

Looks good to me at first glance. Being able to specify the target surface from the client code is great - for the front end of Daedalus I have a decent sized chunk of vram free and it would be good to use this for the font texture. To be honest I'm most interested in the interface to whatever layer sits above this (i.e. some simple wrapper to generate the texture and actually render to the screen) as I think this is what most people are going to end up using.

Well, my first idea was a method similar to what I did in my font engine I wrote for PMP Mod - I used 4bit textures and used the first 8 indices for the border (here shadow) color and the following 8 for the actual font color, smoothly blending between. The results were very acceptable, but extending to 8bpp is an option indeed.

Edit: another note: for proper font rendering sub-pixel resolution is usually required. pgf-files provide 1/64, intraFont currently uses 1/4 pixel resolution. In your implementation I see char hAdvance and int *GetSize. Any plans for sub-pixel resolution?

Not sure yet, but seeing how this is a bitmap font and the only parts where subpixel resolution would matter in any way are the pspFontImageBuffer's x/y coordinates and the glyphInfo advance values I'll probably just stick with giving those values in subpixel accuracy.

Did you see this paper from Valve at this year's siggraph? http://www.valvesoftware.com/publicatio ... cation.pdf

I'd be really interested in seeing your library generate a signed distance function version of the font data, as it would allow the font to be resized dynamically with very good results. Would be great for scaling text in the ui, but also for rendering 3d text at various levels of magnification.

(edit: repeating myself)

[BenHur]

A cachewriteback at the proper position might solve that issue. Not sure yet where that might be.

EDIT: On another note, it might be (seeing how only a few out of that lot of text drawn are messed up) that the artifacts are caused by hitting the font texture 'wrap' boundary (unless you use CACHEALL), where glyphs in the cache get overwritten before they were actually rendered by the GU.
In that case a sync to GU before printing new strings should help.

After a little playing around, I think your second suggestion is correct, i.e. the glyphs in the cache are overwritten before they're rendered by the GU.

intraFont detects and fixes overwritten glyph cache within a single string (aborts if the cache cannot hold the whole string) and activates the texture with sceGUTexImage. If the cache changes in a following intraFontPrint command, it is activated again with sceGuTexImage. But I'm missing some important info here: does sceGuTexImage actually copy the texture in a "save" place (GU texture cache) or just merely remember the adress of the texture? (in which case my caching-scheme is obviously flawed)

I was worried that different strings were being rendered each frame as I scrolled through the list, so I turned off all my clipping, rendering all the text every frame. With that, I noticed that the glitches only seem to occur when a preview picture is being rendered (i.e. the Super Mario 64 preview in my screenshots above), and disappear when no preview is being displayed.

To be honest I would have thought rendering the preview would have helped reduce the glitching, as I would expect it to flush the intra font texture from the GU texture cache. I guess these kinds of caching issues are never all that intuitive though :)

I'm confused as well, since the possible flaw described above should be independent of any other texture changes - but only depend on the total number of different characters you use in your application. (Which BTW confuses me as well, since the standard intraFont cache size should be able to handle about 100 different chars. But I dont' see that many different chars in your Daedalus GUI...)

INTRAFONT_CACHE_ALL is a good enough solution for me - I can live with the 128KB it requires for the texture.

The texture size increased, but the overall mem consumption is similar (see performance section in intraFont readme).

As Raphael suggests, it looks like there might be some kind of explicit GU sync needed. I suggest this is better implemented at the intraFont level rather than the client code, as it's not necessary for every string that's rendered, just when intraFont decides to update its texture cache.

intraFont glyph cache is crap(py) as soon as it has to start overwritting old glyphs. But since a better solution appears not that far away (Raphaels pspFont), I'd rather stay away from putting a lot of thought, time and work into intraFont glyph caching. Of course if there's an easy fix, I'll do it - but I'd suggest to life with INTRAFONT_CACHE_ALL or INTRAFONT_CACHE_LARGE if the default setting causes problems.

Cheers, BenHur

[chp]

But I'm missing some important info here: does sceGuTexImage actually copy the texture in a "save" place (GU texture cache) or just merely remember the adress of the texture? (in which case my caching-scheme is obviously flawed)

sceGuTexImage() just takes the pointer you give it and uses it to render. There's no "safe" place, you have 2MB to work with and you have to use it wisely, or you could render directly from main RAM and avoid the cache if you run out of space.

[Raphael]

After a little playing around, I think your second suggestion is correct, i.e. the glyphs in the cache are overwritten before they're rendered by the GU.

I was worried that different strings were being rendered each frame as I scrolled through the list, so I turned off all my clipping, rendering all the text every frame. With that, I noticed that the glitches only seem to occur when a preview picture is being rendered (i.e. the Super Mario 64 preview in my screenshots above), and disappear when no preview is being displayed.

To be honest I would have thought rendering the preview would have helped reduce the glitching, as I would expect it to flush the intra font texture from the GU texture cache. I guess these kinds of caching issues are never all that intuitive though :)

Well, that only depends on at what place/time the preview image rendering actually happens.

I suggest this is better implemented at the intraFont level rather than the client code, as it's not necessary for every string that's rendered, just when intraFont decides to update its texture cache.

intraFont glyph cache is crap(py) as soon as it has to start overwritting old glyphs. But since a better solution appears not that far away (Raphaels pspFont), I'd rather stay away from putting a lot of thought, time and work into intraFont glyph caching. Of course if there's an easy fix, I'll do it - but I'd suggest to life with INTRAFONT_CACHE_ALL or INTRAFONT_CACHE_LARGE if the default setting causes problems.

Well, the big problem is that intraFont cannot detect when a glyph that is about to be overwritten has already been rendered by the GU, so you'd have to insert a preventive GU sync at that place, even though it might not be needed most of the time but still dragging execution speed of the application.
I'm too currently breaking my head over a fast and resilient caching scheme. The biggest problem again is the case where cached glyphs have to be overwritten - though it will not affect the rendering as in intraFont, because the glyphs aren't rendered directly from cache. However, it will cost quite some performance to properly handle this case without wasting lots of cache space. I'm up to a solution though.

Looks good to me at first glance. Being able to specify the target surface from the client code is great - for the front end of Daedalus I have a decent sized chunk of vram free and it would be good to use this for the font texture. To be honest I'm most interested in the interface to whatever layer sits above this (i.e. some simple wrapper to generate the texture and actually render to the screen) as I think this is what most people are going to end up using.

I hope I'll also be able to provide a simple layer that does that job too, with support for font textures in VRAM (through the use of libpspvram :P)

Did you see this paper from Valve at this year's siggraph? http://www.valvesoftware.com/publicatio ... cation.pdf

I'd be really interested in seeing your library generate a signed distance function version of the font data, as it would allow the font to be resized dynamically with very good results. Would be great for scaling text in the ui, but also for rendering 3d text at various levels of magnification.

Very interesting paper there :) I'll have a read on it and see what I can learn from it. Thanks

EDIT: Very interesting technique discussed in that paper indeed. Unfortunately, it cannot really be applied to our cause here, because the glyph images from the PGF file are already coverage samples and low resolution, so creating a distance field from that will create very bad results. It could however easily be applied onto any TTF font loading engine that creates a font texture from the TTF like pgeFont. Would be interesting how low-resolution you can get the distance field before quality degrades significantly (maybe a 8x8 field for each glyph would already suffice?).

[StrmnNrmn]

I'm confused as well, since the possible flaw described above should be independent of any other texture changes - but only depend on the total number of different characters you use in your application. (Which BTW confuses me as well, since the standard intraFont cache size should be able to handle about 100 different chars. But I dont' see that many different chars in your Daedalus GUI...)

That's true - it probably renders at most ~70 chars (a-zA-Z0-9()-:), and usually a lot less. I'll add a quick test to count how many characters are being rendered each frame. Maybe the caching scheme has a bug and certain characters are being evicted unnecessarily?

EDIT: Very interesting technique discussed in that paper indeed. Unfortunately, it cannot really be applied to our cause here, because the glyph images from the PGF file are already coverage samples and low resolution, so creating a distance field from that will create very bad results. It could however easily be applied onto any TTF font loading engine that creates a font texture from the TTF like pgeFont. Would be interesting how low-resolution you can get the distance field before quality degrades significantly (maybe a 8x8 field for each glyph would already suffice?)

That's a shame - I didn't know how the PGF glyph images were being generated. I'm really interested in seeing it in action on the PSP. If I get some time at the weekend I might have a go at hacking pgeFont.

It is a very nice technique - a colleague has been using it for several years now (well before the Valve paper) and it gives very good results. Typically we use a source image with 2x the horiz/vert resolution - I think it would work fine with an 8x8 glyph, assuming you were scaling up to around 16x16 rendered. If you have shaders available that opens up a whole range of new possibilities (like borders, outlines and drop shadows as described in the Valve paper.) I guess we'll have to wait for the PSP2 for that :)

[chp]

Well, the big problem is that intraFont cannot detect when a glyph that is about to be overwritten has already been rendered by the GU, so you'd have to insert a preventive GU sync at that place, even though it might not be needed most of the time but still dragging execution speed of the application.
I'm too currently breaking my head over a fast and resilient caching scheme. The biggest problem again is the case where cached glyphs have to be overwritten - though it will not affect the rendering as in intraFont, because the glyphs aren't rendered directly from cache. However, it will cost quite some performance to properly handle this case without wasting lots of cache space. I'm up to a solution though.

It's actually not too complex how to detect this, as long as you refill the cache using the GE (copying glyphs RAM -> VRAM using sceGuCopyImage()). If you do this you can very easily maintain a cache state that stays in sync with the GE, because the GE will process your displaylist in order, which means that as long as a string has been rendered you no longer need to keep the source data in VRAM and that part of the cache should be considered as "free".

If you fill the cache using the CPU then you have some work cut out for you, then you'd have to insert GE breaks whenever you overflow the cache to tell the CPU to refill using a snapshot from what a full cache should look like (whenever you overflow the cache you store the previous complete state of the cache, and use that to generate the font cache bitmap for the previous render pass). You would not be able to use GU_DIRECT rendering either since you could then possibly execute the break before you know what the complete cache state should look like.

There are of course other approaches to this, but these seems like the most straightforward and easy to implement, and I'd recommend going all GPU since it requires the least amount of interruptions.

[BenHur]

intraFont 0.22 is out, it has a couple of fixes and optimizations and two new features:

1. S-JIS string parsing for developers using Japanese
2. ASCII characters caching (for those who need only the standard character set, but want to save on memory and loadtime)

More details in the readme.

Cheers, BenHur