libc opendir freezes PSP

Discuss the development of new homebrew software, tools and libraries.

Moderators: cheriff, TyRaNiD

Post Reply
robif
Posts: 6
Joined: Sat Oct 15, 2005 11:02 pm
Location: Maribor

libc opendir freezes PSP

Post by robif »

It seems that memory that is allocated for DIR structure in opendir() should be
initialized to zero, as current implementation causes PSP to freeze.

Changing

Code: Select all

      dirp = (DIR *)malloc(sizeof(DIR));
with

Code: Select all

    dirp = (DIR *)calloc(sizeof(DIR), sizeof(char));
in opendir() prevented freezing up, so i suggest this change to libcglue.c in newlib.

Also readdir() is potential for memory leak, because it allocates new dirent structure for each
entry. Caller must release it explicitly and that is not what programs that use this function
do (traditionally this function returns pointer to static structure).

BR,
Robert
jimparis
Posts: 1145
Joined: Fri Jun 10, 2005 4:21 am
Location: Boston

Re: libc opendir freezes PSP

Post by jimparis »

robif wrote:It seems that memory that is allocated for DIR structure in opendir() should be
initialized to zero, as current implementation causes PSP to freeze.
Do you have an example? It's working fine here. The DIR structure only contains a single element anyway, which clearly gets set in the very next line, so I don't see what initializing it to zero could possibly do.
robif wrote: Also readdir() is potential for memory leak, because it allocates new dirent structure for each
entry. Caller must release it explicitly and that is not what programs that use this function
do (traditionally this function returns pointer to static structure).
readdir()'s buffer may be overwritten only for the same directory stream, so a single static structure is no good. We probably need to allocate a buffer in struct DIR and use that.
robif
Posts: 6
Joined: Sat Oct 15, 2005 11:02 pm
Location: Maribor

Re: opendir example

Post by robif »

Example where call to opendir froze was this function:

Code: Select all

static PyObject* PyPSP_listdir(PyObject *self,
                               PyObject *args)
{
    char *path;
    DIR* dirp;
    struct dirent* ent;
    PyObject* ret;

    if (!PyArg_ParseTuple(args, "s:listdir", &path))
       return NULL;

    dirp = opendir(path);
    if (!dirp) {
        PyErr_Format(PyExc_OSError, "Can't open directory %s",  path);
        return NULL;
    }
    ret = PyList_New(0);
    while(1) 
    {
        ent = readdir(dirp);
        if (!ent) 
        {
            break;
        }
        PyList_Append(ret, PyString_FromString(ent -> d_name));
        free(ent);
    }
    if &#40;closedir&#40;dirp&#41; < 0&#41; &#123;
        Py_DECREF&#40;ret&#41;;
        ret = NULL;
        PyErr_SetFromErrno&#40;PyExc_OSError&#41;;
        return NULL;
    &#125;
    return ret;
&#125;
when i replaced malloc with calloc within opendir and readdir, it worked. This was on EU psp.
Of course it might be some other problem as it is called from within interpreter that does
other things...

Idea of storing ptr to dirent or dirent itself in DIR looks good, much better than static
struct.
jimparis
Posts: 1145
Joined: Fri Jun 10, 2005 4:21 am
Location: Boston

Post by jimparis »

I still say that there's no possible difference between calloc() and malloc() because the only variable in the DIR structure gets overwritten immediately anyway. If you can provide a small compilable example I'll check it out, but I suspect your problem is hidden elsewhere and is just being triggered by the slightly different call stack.
robif
Posts: 6
Joined: Sat Oct 15, 2005 11:02 pm
Location: Maribor

Post by robif »

It seems, you have been right.
I have created this sample:

Code: Select all

/*
 * PSP Software Development Kit - http&#58;//www.pspdev.org
 * -----------------------------------------------------------------------
 * Licensed under the BSD license, see LICENSE in PSPSDK root for details.
 *
 * main.c - Sample to desmonstrate use of opendir/readdir
 *
 * Copyright &#40;c&#41; 2005 John Kelley <[email protected]>
 *
 * $Id&#58; main.c 1095 2005-09-27 21&#58;02&#58;16Z jim $
 */
#include <pspkernel.h>
#include <pspdebug.h>
#include <stdio.h>
#include <string.h>
#include <pspmoduleinfo.h>
#include <dirent.h>

/* Define the module info section */
PSP_MODULE_INFO&#40;"Opendir Sample", 0, 1, 0&#41;;

/* Define printf, just to make typing easier */
#define printf  pspDebugScreenPrintf

/* Exit callback */
int exit_callback&#40;int arg1, int arg2, void *common&#41;
&#123;
    sceKernelExitGame&#40;&#41;;

        return 0;
&#125;

/* Callback thread */
int CallbackThread&#40;SceSize args, void *argp&#41;
&#123;
    int cbid;
    cbid = sceKernelCreateCallback&#40;"Exit Callback", exit_callback, NULL&#41;;
    sceKernelRegisterExitCallback&#40;cbid&#41;;
    sceKernelSleepThreadCB&#40;&#41;;

        return 0;
&#125;

/* Sets up the callback thread and returns its thread id */
int SetupCallbacks&#40;void&#41;
&#123;
    int thid = 0;
    thid = sceKernelCreateThread&#40;"update_thread", CallbackThread, 0x11, 0xFA0, T
HREAD_ATTR_USER, 0&#41;;
    if &#40;thid >= 0&#41;
        sceKernelStartThread&#40;thid, 0, 0&#41;;
    return thid;
&#125;

void listdir&#40;const char* path&#41;
&#123;
    DIR* dirp;
    struct dirent* en;
    dirp = opendir&#40;path&#41;;
    if &#40;!dirp&#41; &#123;
        printf&#40;"Opendir returned NULL, bailing out\n"&#41;;
        return;
    &#125;
    while&#40;1&#41; &#123;
        en = readdir&#40;dirp&#41;;
        if &#40;!en&#41; &#123;
            break;
        &#125;
        printf&#40;"Entry&#58; %s\n", en -> d_name&#41;;
    &#125;
    printf&#40;"Done\n"&#41;;
&#125;

/* main routine */
int main&#40;int argc, char *argv&#91;&#93;&#41;
&#123;
    const char* path = "ms0&#58;/";
    
    //init screen and callbacks
    pspDebugScreenInit&#40;&#41;;
    pspDebugScreenClear&#40;&#41;;
    
    SetupCallbacks&#40;&#41;;

    pspDebugScreenSetXY&#40;0, 0&#41;;
    printf&#40;"Opendir Sample v1.0 \n\n"&#41;; 
    
    printf&#40;"Here we go, list of %s\n", path&#41;;
    listdir&#40;path&#41;;

    sceKernelSleepThread&#40;&#41;;
    sceKernelExitGame&#40;&#41;;

    return 0;
&#125;
and Makefile.sample:

Code: Select all

PSPSDK = $&#40;shell psp-config --pspsdk-path&#41;
PSPLIBSDIR = $&#40;PSPSDK&#41;/..
TARGET = opendsample
OBJS = main.o
LIBS = 
EXTRA_TARGETS = EBOOT.PBP
PSP_EBOOT_TITLE= Opendir Sample


CFLAGS = -O2 -G0 -Wall
CXXFLAGS = $&#40;CFLAGS&#41; -fno-exceptions -fno-rtti
ASFLAGS = $&#40;CFLAGS&#41;

include $&#40;PSPSDK&#41;/lib/build.mak
There is no problem running this with current version of opendir. There must be some other
problem within Python itself. This thread can be closed.
raf
Posts: 57
Joined: Thu Oct 13, 2005 7:38 am

Post by raf »

Jim,

I also experienced problems when accessing a directory using the native sceIoDread calls. I found the solution in this forum, topic:
http://forums.ps2dev.org/viewtopic.php?t=2623
From mrbrown:
Actually, you have to memset() the dirent before calling sceIoDread(). Then it works on stack or global.
So maybe readdir() needs to memset dirent also before calling sceIoDread?

Must be that sceIoDread() tries to free buffers defined in the dirent? I also remember someone saying that sceIoDread() allocates memory and that the user is supposed to free it.. (unusual behaviour, if true, but it may explain the crashes when dirent is not initialized, if Dread tries to free...).

Raf.
jimparis
Posts: 1145
Joined: Fri Jun 10, 2005 4:21 am
Location: Boston

Post by jimparis »

OK, rev 1172 has these changes (move dirent inside DIR, zero dirent before calling sceIoDread). Don't free the dirent returned by readdir() anymore. Let me know if it gives you any trouble.
raf
Posts: 57
Joined: Thu Oct 13, 2005 7:38 am

Post by raf »

jimparis wrote:OK, rev 1172 has these changes (move dirent inside DIR, zero dirent before calling sceIoDread). Don't free the dirent returned by readdir() anymore. Let me know if it gives you any trouble.
Good deal; thanks Jim.

Raf.
Post Reply