[SOLVED] Memory leak in libTremor?

Discuss the development of new homebrew software, tools and libraries.

Moderators: cheriff, TyRaNiD

Post Reply
sakya
Posts: 190
Joined: Fri Apr 28, 2006 5:48 pm
Contact:

[SOLVED] Memory leak in libTremor?

Post by sakya »

Hi! :)

I'm experiencing some strange memory leak with libTremor.
This code just opens and closes an OGG Vorbis file and it leaks at least 4kb of memory.
I think I'm clearing all the memory I use: clear the OggVorbis_File variable and close the file (this should be closed by libTremor, but I tried also to close it in my code).
Can someone help me find this leak, or test this code to be sure it's not related to my dev env?
Many thanks. :)

Code: Select all

/////////////////////////////////////////////////////////////////////////////////////////
//Callback for vorbis
/////////////////////////////////////////////////////////////////////////////////////////
size_t ogg_callback_read(void *ptr, size_t size, size_t nmemb, void *datasource){
    return sceIoRead(*(int *) datasource, ptr, size * nmemb);
}
int ogg_callback_seek(void *datasource, ogg_int64_t offset, int whence){
    return sceIoLseek32(*(int *) datasource, (unsigned int) offset, whence);
}
long ogg_callback_tell(void *datasource){
    return sceIoLseek32(*(int *) datasource, 0, SEEK_CUR);
}
int ogg_callback_close(void *datasource){
    return sceIoClose(*(int *) datasource);
}

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Open/close an OGG file
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
int openCloseOGG(char *filename){
    int tempFile = 0;
    OggVorbis_File vf;

	//Apro il file OGG:
	tempFile = sceIoOpen(filename, PSP_O_RDONLY, 0777);
	if (tempFile >= 0) {
        ov_callbacks ogg_callbacks;

        ogg_callbacks.read_func = ogg_callback_read;
        ogg_callbacks.seek_func = ogg_callback_seek;
        ogg_callbacks.close_func = ogg_callback_close;
        ogg_callbacks.tell_func = ogg_callback_tell;

		  if &#40;ov_open_callbacks&#40;&tempFile, &vf, NULL, 0, ogg_callbacks&#41; < 0&#41;&#123;
            sceIoClose&#40;tempFile&#41;;
            return -1;
        &#125;
        ov_clear&#40;&vf&#41;;
        if &#40;tempFile >= 0&#41;
            sceIoClose&#40;tempFile&#41;;
	&#125;
	return 0;
&#125;
I can post the full code if needed (tha main just executes openCloseOGG when pressing X and check freemem). ;)

Ciaooo
Sakya
Last edited by sakya on Fri Jan 18, 2008 3:44 am, edited 1 time in total.
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

Does it leak every time you call it, or just the FIRST time? Every time would be a leak. Only the first time is just the system needed to use some resources that don't get flushed until you exit.
sakya
Posts: 190
Joined: Fri Apr 28, 2006 5:48 pm
Contact:

Post by sakya »

Hi! :)
J.F. wrote:Does it leak every time you call it, or just the FIRST time? Every time would be a leak. Only the first time is just the system needed to use some resources that don't get flushed until you exit.
Many thanks for your reply. ;)
It leaks 4kb (sometimes 8) every time I call the function.
I tried also with ov_open instead of ov_open_callbacks but the result is the same.

Ciaooo
Sakya
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

Well, the libTremor docs say this:
Once the file is no longer needed, ov_clear() is used to close the file and deallocate decoding resources. Do not call fclose() on the file; libvorbisidec does this in the ov_clear() call.
And in vorbisfile.c, you find this at the end of the ov_clear function:

Code: Select all

    if&#40;vf->datasource&#41;&#40;vf->callbacks.close_func&#41;&#40;vf->datasource&#41;;
    memset&#40;vf,0,sizeof&#40;*vf&#41;&#41;;
What happens if you don't do the second sceIoClose()? Perhaps the leak is due to trying to close the file twice.
sakya
Posts: 190
Joined: Fri Apr 28, 2006 5:48 pm
Contact:

Post by sakya »

Hi! :)
J.F. wrote:What happens if you don't do the second sceIoClose()? Perhaps the leak is due to trying to close the file twice.
The same thing. I added the additional close to try to solve the leak...

Here's the full source I'm using:

Code: Select all

#include <pspkernel.h>
#include <pspctrl.h>
#include <pspdebug.h>
#include <pspsdk.h>
#include <stdlib.h>
#include "tremor/ivorbiscodec.h"
#include "tremor/ivorbisfile.h"

PSP_MODULE_INFO&#40;"eboot template", 0, 1, 0&#41;;
PSP_MAIN_THREAD_ATTR&#40;THREAD_ATTR_USER&#41;;
PSP_HEAP_SIZE_KB&#40;2048&#41;;

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Globals&#58;
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
int runningFlag = 1;

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Callbacks&#58;
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/* Exit callback */
int exit_callback&#40;int arg1, int arg2, void *common&#41; &#123;
    runningFlag = 0;
    return 0;
&#125;

/* Callback thread */
int CallbackThread&#40;SceSize args, void *argp&#41; &#123;
    int cbid;
    cbid = sceKernelCreateCallback&#40;"Exit Callback", exit_callback, NULL&#41;;
    sceKernelRegisterExitCallback&#40;cbid&#41;;
    sceKernelSleepThreadCB&#40;&#41;;
    return 0;
&#125;

/* Sets up the callback thread and returns its thread id */
int SetupCallbacks&#40;void&#41; &#123;
    int thid = 0;
    thid = sceKernelCreateThread&#40;"update_thread", CallbackThread, 0x11, 0xFA0, PSP_THREAD_ATTR_USER, 0&#41;;
    if&#40;thid >= 0&#41;
        sceKernelStartThread&#40;thid, 0, 0&#41;;
    return thid;
&#125;

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Total free memory&#58;
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
typedef struct
&#123;
      void *buffer;
      void *next;
&#125; _LINK;

int freemem&#40;&#41;&#123;
   int size = 4096, total = 0;
    _LINK *first = NULL, *current = NULL, *lnk;

    while&#40;runningFlag&#41;&#123;
       lnk = &#40;_LINK*&#41;malloc&#40;sizeof&#40;_LINK&#41;&#41;;
       if &#40;!lnk&#41;
          break;

       total += sizeof&#40;_LINK&#41;;

       lnk->buffer = malloc&#40;size&#41;;
       if &#40;!lnk->buffer&#41;&#123;
          free&#40;lnk&#41;;
          break;
       &#125;

       total += size;
       lnk->next = NULL;

       if &#40;current&#41;&#123;
          current->next = &#40;void*&#41;lnk;
          current = lnk;
       &#125; else &#123;
          current = first = lnk;
       &#125;
    &#125;

    lnk = first;
    while &#40;lnk&#41;&#123;
       free&#40;lnk->buffer&#41;;
       current = lnk->next;
       free&#40;lnk&#41;;
       lnk = current;
    &#125;
    return total;
&#125;

/////////////////////////////////////////////////////////////////////////////////////////
//Callback for vorbis
/////////////////////////////////////////////////////////////////////////////////////////
size_t ogg_callback_read&#40;void *ptr, size_t size, size_t nmemb, void *datasource&#41;&#123;
    return sceIoRead&#40;*&#40;int *&#41; datasource, ptr, size * nmemb&#41;;
&#125;
int ogg_callback_seek&#40;void *datasource, ogg_int64_t offset, int whence&#41;&#123;
    return sceIoLseek32&#40;*&#40;int *&#41; datasource, &#40;unsigned int&#41; offset, whence&#41;;
&#125;
long ogg_callback_tell&#40;void *datasource&#41;&#123;
    return sceIoLseek32&#40;*&#40;int *&#41; datasource, 0, SEEK_CUR&#41;;
&#125;
int ogg_callback_close&#40;void *datasource&#41;&#123;
    return sceIoClose&#40;*&#40;int *&#41; datasource&#41;;
&#125;

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Open/close an OGG file
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
int openCloseOGG&#40;char *filename&#41;&#123;
    int tempFile = 0;
    OggVorbis_File vf;

	//Apro il file OGG&#58;
	tempFile = sceIoOpen&#40;filename, PSP_O_RDONLY, 0777&#41;;
	if &#40;tempFile >= 0&#41; &#123;
        ov_callbacks ogg_callbacks;

        ogg_callbacks.read_func = ogg_callback_read;
        ogg_callbacks.seek_func = ogg_callback_seek;
        ogg_callbacks.close_func = ogg_callback_close;
        ogg_callbacks.tell_func = ogg_callback_tell;

		if &#40;ov_open_callbacks&#40;&tempFile, &vf, NULL, 0, ogg_callbacks&#41; < 0&#41;&#123;
            sceIoClose&#40;tempFile&#41;;
            return -1;
        &#125;
        ov_clear&#40;&vf&#41;;
	&#125;
	return 0;
&#125;

int openCloseOGGNoCallbacks&#40;char *filename&#41;&#123;
    FILE *tempFile;
    OggVorbis_File vf;

	//Apro il file OGG&#58;
	tempFile = fopen&#40;filename, "r"&#41;;
	if &#40;tempFile != NULL&#41; &#123;
		if &#40;ov_open&#40;tempFile, &vf, NULL, 0&#41; < 0&#41;&#123;
            fclose&#40;tempFile&#41;;
            return -1;
        &#125;
        ov_clear&#40;&vf&#41;;
	&#125;
	return 0;
&#125;

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Main&#58;
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
int main&#40;&#41;&#123;
	pspDebugScreenInit&#40;&#41;;
	SetupCallbacks&#40;&#41;;

    pspDebugScreenPrintf&#40;"Test OGG Vorbis mem. leak\n\n"&#41;;
    pspDebugScreenPrintf&#40;"Free memory&#58; %i\n", freemem&#40;&#41; / 1024&#41;;
    pspDebugScreenPrintf&#40;"Press X to load and unload with callbacks ms0&#58;/test.ogg\n"&#41;;
    pspDebugScreenPrintf&#40;"Press SQUARE to load and unload without callbacks ms0&#58;/test.ogg\n"&#41;;

    SceCtrlData pad;
    while&#40;runningFlag&#41;&#123;
        sceCtrlReadBufferPositive&#40;&pad, 1&#41;;
        if &#40;pad.Buttons & PSP_CTRL_CROSS&#41;&#123;
            pspDebugScreenPrintf&#40;"Open result&#58; %i\n", openCloseOGG&#40;"ms0&#58;/test.ogg"&#41;&#41;;
            pspDebugScreenPrintf&#40;"Free memory&#58; %i\n", freemem&#40;&#41; / 1024&#41;;
            sceKernelDelayThread&#40;200000&#41;;
        &#125;else if &#40;pad.Buttons & PSP_CTRL_SQUARE&#41;&#123;
            pspDebugScreenPrintf&#40;"Open result&#58; %i\n", openCloseOGGNoCallbacks&#40;"ms0&#58;/test.ogg"&#41;&#41;;
            pspDebugScreenPrintf&#40;"Free memory&#58; %i\n", freemem&#40;&#41; / 1024&#41;;
            sceKernelDelayThread&#40;200000&#41;;
        &#125;

    &#125;
	sceKernelExitGame&#40;&#41;;
    return 0;
&#125;
Ciaooo
Sakya
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

Okay, maybe this is the problem... look at the one line in ov_clear again:

Code: Select all

    if&#40;vf->datasource&#41;&#40;vf->callbacks.close_func&#41;&#40;vf->datasource&#41;;
It closes the file if the file is there. However, in your second close function, you check if the file handle is >= 0, If it can be 0, the ov_clear line won't be called as vf->datasource is 0. It assumes file handles cannot be 0.

Print out the handle to see what it is. If it really is 0, you'll need to make some kind of indirection on it and the callbacks to get it to work right.
sakya
Posts: 190
Joined: Fri Apr 28, 2006 5:48 pm
Contact:

Post by sakya »

Hi! :)
J.F. wrote:It closes the file if the file is there. However, in your second close function, you check if the file handle is >= 0, If it can be 0, the ov_clear line won't be called as vf->datasource is 0.
Many thanks for helping. :)
No luck, I displayed the value but it's 3 with fopen and a bigger number with sceIoOpen but never zero.

Ciaooo
Sakya
jimparis
Posts: 1145
Joined: Fri Jun 10, 2005 4:21 am
Location: Boston

Post by jimparis »

I updated libTremor to the latest xiph SVN. In my quick tests on the PC, the old version had a memory leak and the new version does not. Give it a try.
Here was the leak:

Code: Select all

==12544== 208 bytes in 2 blocks are definitely lost in loss record 2 of 5
==12544==    at 0x4C20F3F&#58; calloc &#40;vg_replace_malloc.c&#58;279&#41;
==12544==    by 0x4E34511&#58; ogg_stream_create &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x4E2FA1F&#58; _ov_open1 &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x4E2FB33&#58; ov_open_callbacks &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x4E2FB9B&#58; ov_open &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x400680&#58; main &#40;in /tmp/examples/ivorbisfile_example&#41;
==12544== 
==12544== 
==12544== 5,536 &#40;32 direct, 5,504 indirect&#41; bytes in 1 blocks are definitely lost in loss record 5 of 5
==12544==    at 0x4C21C16&#58; malloc &#40;vg_replace_malloc.c&#58;149&#41;
==12544==    by 0x4E33E52&#58; _fetch_ref &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x4E33F1D&#58; ogg_buffer_split &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x4E347F8&#58; ogg_sync_pageseek &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x4E2D854&#58; _get_next_page &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x4E2DAEE&#58; _fetch_headers &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x4E2FA3D&#58; _ov_open1 &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x4E2FB33&#58; ov_open_callbacks &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x4E2FB9B&#58; ov_open &#40;in /usr/local/lib/libvorbisidec.so.1.0.2&#41;
==12544==    by 0x400680&#58; main &#40;in /tmp/examples/ivorbisfile_example&#41;
sakya
Posts: 190
Joined: Fri Apr 28, 2006 5:48 pm
Contact:

Post by sakya »

Hi! :)
jimparis wrote:I updated libTremor to the latest xiph SVN. In my quick tests on the PC, the old version had a memory leak and the new version does not. Give it a try.
So it's true! Many thanks! :)

Can you help me please to compile the new version for psp?
I tried downloading the new source code from xiph svn and then launched:

Code: Select all

$ LDFLAGS="-L$&#40;psp-config --pspsdk-path&#41;/lib -lc -lpspuser" ./autogen.sh \
     --host psp --prefix=$&#40;psp-config --psp-prefix&#41;
$ make
But I get this error during compilation:

Code: Select all

make  all-am
make&#91;1&#93;&#58; Entering directory `/home/user/libTremor_new'
if /bin/sh ./libtool --tag=CC --mode=compile psp-gcc -DHAVE_CONFIG_H -I. -I. -I.
 -I./    -O2 -Wall -fsigned-char  -D_REENTRANT -MT mdct.lo -MD -MP -MF ".deps/md
ct.Tpo" -c -o mdct.lo mdct.c; \
        then mv -f ".deps/mdct.Tpo" ".deps/mdct.Plo"; else rm -f ".deps/mdct.Tpo
"; exit 1; fi
 psp-gcc -DHAVE_CONFIG_H -I. -I. -I. -I./ -O2 -Wall -fsigned-char -D_REENTRANT -
MT mdct.lo -MD -MP -MF .deps/mdct.Tpo -c mdct.c -o mdct.o
In file included from mdct.c&#58;37&#58;
misc.h&#58;47&#58; error&#58; redefinition of 'union magic'
make&#91;1&#93;&#58; *** &#91;mdct.lo&#93; Error 1
make&#91;1&#93;&#58; Leaving directory `/home/user/libTremor_new'
make&#58; *** &#91;all&#93; Error 2
Many thanks again. ;)
Ciaooo
Sakya
jimparis
Posts: 1145
Joined: Fri Jun 10, 2005 4:21 am
Location: Boston

Post by jimparis »

Don't download it from xiph svn, download it from pspdev svn. Or just rerun the psplibraries script.
When I say I updated it, I mean I updated the pspdev version.
sakya
Posts: 190
Joined: Fri Apr 28, 2006 5:48 pm
Contact:

Post by sakya »

Hi! :)
jimparis wrote:Don't download it from xiph svn, download it from pspdev svn.
When I say I updated it, I mean I updated the pspdev version.
Oh, sorry. O:)
Many thanks, really. :)
I'll try this evening (this damned proxy doesen't allow me to connect to svn).

Ciaooo
Sakya
jimparis
Posts: 1145
Joined: Fri Jun 10, 2005 4:21 am
Location: Boston

Post by jimparis »

I also happen to run a HTTP-accessible mirror of the SVN if that helps:
svn co http://psp.jim.sh/svn/psp/trunk/libTremor
sakya
Posts: 190
Joined: Fri Apr 28, 2006 5:48 pm
Contact:

Post by sakya »

Hi! :)
jimparis wrote:I also happen to run a HTTP-accessible mirror of the SVN if that helps:
svn co http://psp.jim.sh/svn/psp/trunk/libTremor
The damned proxy blocked also this connection (cannot connect to server) but I downloaded the new version now and the memory leak is gone! :)

Many thanks again.
Ciaooo
Sakya
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

That's great news to anyone using libTremor on the PSP. Just updating the libraries and recompiling fixes a potential problem. :)
Post Reply