Page 1 of 1
Running multiple processes concurrently (also threads)
Posted: Sat May 10, 2008 4:56 am
by whatisdot
Hi. Me again.
I was wondering, how do we execute multiple processes and manage them? There isn't a "fork()" and "exec()" system call for us to use. The only way I can see to spawn a new process is to use the "SifLoadElf()" function call, and from what I can tell from the source code, that ends up using RPCs to execute the command. Isn't there something a little more close to hardware that we can use (like fork)?
The reason I'm asking is because I'm trying to set up a process management tool that allows a user to "kill" a process if it hangs. Does anyone have any experience or any ideas to share?
Thanks
Posted: Mon May 12, 2008 8:41 am
by whatisdot
...nobody...?
Posted: Mon May 12, 2008 10:45 am
by J.F.
None of the Playstations until the PS3 are designed for that. The PS1/PS2/PSP all assume only one app is running, and they all use cooperative multitasking.
Posted: Mon May 12, 2008 4:01 pm
by Lukasz
Posted: Tue May 13, 2008 4:57 am
by cosmito
Also check Lukasz's Libito graphic library. Although I haven't tested it yet, it seems to have support for creating threads (it calls the thread support calls from the sdk) so even if you will not use Libito, you can learn to manage threads by looking at the Libito's source, I think. But like JF said, multitasking is not that great...
Posted: Tue May 13, 2008 5:49 am
by whatisdot
Thanks for all the replies.
I have to point out that there is a difference between a "process" and a "thread." A process is active machine code that has been loaded and is being executed, where a thread of execution is similar to a process but doesn't have as high of precedence. I can achieve what I wanted with threads, but I didn't know if there was a separate library provided for interfacing specifically with processes. Each process can have multiple threads, after all...
Thanks again.
Posted: Tue May 13, 2008 8:50 am
by J.F.
As the posts and the linked thread indicate, threads are supported with cooperative multitasking - processes are not. The most you see on the PS2 is one app launching another, like with ULE or similar apps.
Given your stated goal of having a "management tool" that kills a hung process, and given that the PS2 really only runs one thing at a time, the goal can be met already via the RESET button. :)
Posted: Tue May 13, 2008 5:06 pm
by Lukasz
ptek wrote:Also check Lukasz's Libito graphic library. Although I haven't tested it yet, it seems to have support for creating threads (it calls the thread support calls from the sdk) so even if you will not use Libito, you can learn to manage threads by looking at the Libito's source, I think. But like JF said, multitasking is not that great...
The thread support was included mostly for making it easier to create SIFRPC server threads, like on the IOP. Besides from threads which are activated by interrupts, I don't see much use for threads on the EE.
Posted: Wed May 14, 2008 2:16 am
by whatisdot
Okay, I think I understand now.
After taking a closer look at the kernel code, I see how the threads are handled by system calls in the PS2SDK. Now I understand J.F.s comment about "only one process" being able to execute, but as long as it can have multiple threads then that's fine.
I am hoping that I can get multiple threads working, so I can load multiple ELFs. For example, being able to load an ELF like PS2FTP, while still being able to browse using uLaunchELF. My reasoning is that if it is possible to launch an ELF from another ELF, then it should also be possible to catch a return from it. How else would they be able to get PS2Linux working?
How does this hypothesis sound?
In the meantime, I'll try to get threads to work...
Posted: Wed May 14, 2008 6:39 am
by J.F.
As the other thread showed, that while() should be doing iRotateThreadReadyQueue or you'll NEVER run another thread. Remember - COOPERATIVE multitasking. If one thread busy-loops, NOTHING else runs.
Posted: Sun May 25, 2008 8:19 am
by whatisdot
Okay, I got threads working now. Thanks to J.F. for pointing out the importance of iRotateThreadReadyQueue. At first I had some trouble understanding what was going on in the discussion that he pointed out, but after some experimenting I got it working. Here's a demo of what I got so far. (The references to the "thread" are system calls and can be found in kernel.h) If anyone has some tips about thread status, priorities, or keeping track of stack size, feel free to chime in. Any advise would be great. =)
Makefile
Code: Select all
EE_BIN = threads.elf
EE_OBJS = threads.o
EE_LIBS = -ldebug -lkernel -lstdc++
all: $(EE_BIN)
clean:
rm -f threads.elf *.o *.a
include $(PS2SDK)/samples/Makefile.pref
include $(PS2SDK)/samples/Makefile.eeglobal
Code: Select all
#include <tamtypes.h>
#include <fileio.h>
#include <stdlib.h>
#include <stdio.h>
#include <kernel.h>
#include <debug.h>
static int POTATO_DONE = false;
static int RADISH_DONE = false;
void countPotatoes() {
for(u32 i=0; i<100000000; i++) {
if( i%10000000 == 0 )
printf("%d potato...\n", i/10000000);
}
POTATO_DONE = true;
}
void countRadishes() {
for(u32 i=0; i<100000000; i++) {
if( i%10000000 == 0 )
printf("%d radish...\n", i/10000000);
}
RADISH_DONE = true;
}
int main( int argc, char **argv) {
struct t_ee_thread potatoThread, radishThread;
int potatoThreadId, radishThreadId;
printf("\nBeginning thread demo:\n");
potatoThread.status = 0;
potatoThread.func = (void*)countPotatoes;
potatoThread.stack = (void*)((int*)malloc(0x800) + 0x800 - 4);
potatoThread.initial_priority = 0x1e;
radishThread.status = 0;
radishThread.func = (void*)countRadishes;
radishThread.stack = (void*)((int*)malloc(0x800) + 0x800 - 4);
radishThread.initial_priority = 0x1e;
potatoThreadId = CreateThread( &potatoThread );
radishThreadId = CreateThread( &radishThread );
if( potatoThreadId <= 0 ) {
printf( "Server thread failed to start. %i\n", potatoThreadId );
return -1;
}
if( radishThreadId <= 0 ) {
printf( "Server thread failed to start. %i\n", radishThreadId );
return -1;
}
StartThread( potatoThreadId, NULL );
StartThread( radishThreadId, NULL );
while( !POTATO_DONE && !RADISH_DONE ) {
iRotateThreadReadyQueue( 0x1e );
}
TerminateThread( potatoThreadId );
printf("Terminated potato thread.\n");
TerminateThread( radishThreadId );
printf("Terminated radish thread.\n");
printf("Goodbye\n");
return 0;
}
Posted: Sun May 25, 2008 9:24 am
by cosmito
Interesting stuff. Thanks.
BTW : Is the -lstdc++ at EE_LIBS really needed? As far I remember it is related to C++.
---
Edited
Hmm I didn't notice your source filename is .cpp, since you're coding in C style.
Posted: Sun May 25, 2008 12:00 pm
by cheriff
whatisdot wrote:threads.cpp
Code: Select all
potatoThread.stack = (void*)malloc(0x800);
Should the stack not instead be:
Code: Select all
potatoThread.stack = (void*)malloc(0x800) + 0x800 - 4;
??
To account for the whole growing down thing? Once these threads start making function calls you might start seeing random stuff as it drops down and starts overwriting other stuff on the heap...
Posted: Mon May 26, 2008 2:30 am
by EEUG
...actually the sample is not really correct as 'iRotateThreadReadyQueue' is intended to be called from the interrupt handler, so it would be more correct to setup alarm handler ('SetAlarm' kernel API) (or timer interrupt handler) and call 'iRotateThreadReadyQueue' from it. It will not matter if threads are doing busy-loops or not as alarm handler will be called anyway so each thread will get its time interval for execution (=preemptive multithreading). Be careful with 'printf' (and any RPC calls (=file I/O etc.)), DMA transfers and other hardware related things as there can be be a chaos when multiple threads will try to use them simultaneously...
Posted: Mon May 26, 2008 7:26 am
by whatisdot
... 'iRotateThreadReadyQueue' is intended to be called from the interrupt handler...
Yes, I feel bad for writing code like this, but so far I have just been trying to create and execute a thread successfully. I am currently lurking through the kernel source to find more info on signals and interrupts.
...account for the whole growing down thing?...
You're right. Thanks! *fixed*
BTW : Is the -lstdc++ at EE_LIBS really needed? As far I remember it is related to C++
You are correct. I use C++ as my default programming language, but I don't take advantage of it here because it's just a demo.
Posted: Mon May 26, 2008 2:35 pm
by whatisdot
Umm...
I'm a little confused and I hope someone could help me out with regards to signal handling and interrupts. Each thread on the EE has the following attributes:
Code: Select all
typedef struct t_ee_thread {
int status;
void* func;
void* stack;
int stack_size;
void* gp_reg;
int initial_priority;
int current_priority;
u32 attr;
u32 option;
} ee_thread_t;
status - the code for the thread status
func - is a pointer to the function that gets created for that thread
stack - is the memory block that is preallocated and ready for the thread to use
initial_priority - is something we set before creating the thread
current_priority - can be changed with ChangeThreadPriority()
as for the others, I am at a loss...
gp_reg - ???
attr - ???
option - ???
But what integer values for 'status' correspond to the actual status of the thread?
0 = ready???
-1 = suspended???
Also, what is the difference between all of the thread functions with names like:
ResumeThread();
iResumeThread();
What does the "i" refer to?
(thanks again to those who are sticking with me on this) =)
Posted: Mon May 26, 2008 6:29 pm
by Lukasz
whatisdot wrote:Umm...
I'm a little confused and I hope someone could help me out with regards to signal handling and interrupts. Each thread on the EE has the following attributes:
Code: Select all
typedef struct t_ee_thread {
int status;
void* func;
extern void * _end;
void* stack;
int stack_size;
void* gp_reg;
int initial_priority;
int current_priority;
u32 attr;
u32 option;
} ee_thread_t;
status - the code for the thread status
func - is a pointer to the function that gets created for that thread
stack - is the memory block that is preallocated and ready for the thread to use
initial_priority - is something we set before creating the thread
current_priority - can be changed with ChangeThreadPriority()
as for the others, I am at a loss...
gp_reg - ???
attr - ???
option - ???
gp_reg is the MIPS global pointer register, which points to the global variables for a program, like the stack pointer (sp) points to the stack. (For more on MIPS registers, see here:
http://www.cs.umd.edu/class/sum2003/cms ... ltReg.html). Both _gp and _end are setup by crt0.s (C Runtime 0, which is located in the startup folder of the PS2SDK source), which is the "init" code for your C/C++ program.
You can get the value for _gp and _end by declaring
Code: Select all
extern void * _gp;
extern void * _end;
I believe that attr and option are optional regs, one is used by the kernel (attr I believe) and the other you can set, if you want point at some data, assign an ID or something else for the thread :-)
whatisdot wrote:
But what integer values for 'status' correspond to the actual status of the thread?
0 = ready???
-1 = suspended???
I remember playing around with this a one point, you can just put the thread into different states and what the values are with ReferThreadStatus(..). I seem to recall the values always are positive, just different for ready, running, waiting, etc.
whatisdot wrote:
Also, what is the difference between all of the thread functions with names like:
ResumeThread();
iResumeThread();
What does the "i" refer to?
(thanks again to those who are sticking with me on this) =)
The "i" functions are safe to call from within a interrupt handler. You have to remember that when you are inside an interrupt handler, interrupts are disabled, which means that the code inside the interrupt handler cannot be interrupted by another interrupt (because then all hell would break loose :). So you need variants of system functions which take this into account, as the code might be a little different for the syscall if called from within an interrupt. For instance the regular system call might disable interrupts, check interrupt state and what not, while the interrupt safe variant might not need to.
Posted: Fri May 30, 2008 3:36 am
by whatisdot
Lukasz to the rescue again. Thanks a bunch. =)
Right now I'm trying to improve the example in the way that everyone has suggested, by using callbacks for interrupts and semaphores to keep the threads from interfering with each other. I'm having some problems here, but I don't know if it is because I have a deadlock or because I don't know semaphores like I'm supposed to...
When I run this code in PS2Link I think that one thread generates an error, while the others continue to execute. How can I handle these errors? I figure there must be a system call for setting a interrupt handler that handles thread read/write errors and things of that nature, so we could kill a thread when it starts behaving badly. The function names for the system calls in "kernel.h" aren't that intuitive...any ideas...?
Code: Select all
#include <tamtypes.h>
#include <fileio.h>
#include <stdlib.h>
#include <stdio.h>
#include <kernel.h>
#include <debug.h>
//Flags
static int POTATO_DONE = false;
static int RADISH_DONE = false;
//Semaphores, Threads, and IDs to be used across functions
static struct t_ee_thread potatoThread, radishThread;
static int potatoThreadId, radishThreadId, counterSemaId;
static struct t_ee_sema counterSema;
//busywork thread function
void countPotatoes(void *args) {
for(u32 i=0; i<100000; i++) {
if( i%10000 == 0 ) {
//if the semaphore is triggered, wait until it's released
//and then trigger it (decrease value by 1)
WaitSema( counterSemaId );
scr_printf("This is a sentance that is long\n");
printf("This is a sentance that is long\n");
printf("%d potato...\n", i/10000);
scr_printf("%d potato...\n", i/10000);
//release the semaphore (increase value by 1)
SignalSema( counterSemaId );
}
}
POTATO_DONE = true;
}
//busywork thread function
void countRadishes(void *args) {
for(u32 i=0; i<100000; i++) {
if( i%10000 == 0 ) {
//if the semaphore is triggered, wait until it's released
//and then trigger it (decrease value by 1)
WaitSema( counterSemaId );
scr_printf("fast dogs make good friends\n");
printf("fast dogs make good friends\n");
printf("%d radish...\n", i/10000);
scr_printf("%d radish...\n", i/10000);
//release the semaphore (increase value by 1)
SignalSema( counterSemaId );
}
}
RADISH_DONE = true;
}
//interrupt handler to rotate the threads
void RotateThreads(s32 id, u16 time, void *arg) {
iRotateThreadReadyQueue( 0x1e );
}
int main( int argc, char **argv) {
init_scr();
printf("\nBeginning thread demo:\n");
//Set initial values for potato thread
potatoThread.status = 0;
potatoThread.func = (void*)countPotatoes;
potatoThread.stack = (void*)((int*)malloc(0x800) + 0x800 - 4);
potatoThread.initial_priority = 0x1e;
//Set initial values for radish thread
radishThread.status = 0;
radishThread.func = (void*)countRadishes;
radishThread.stack = (void*)((int*)malloc(0x800) + 0x800 - 4);
radishThread.initial_priority = 0x1e;
//set initial values for semaphore
counterSema.init_count = 1;
counterSemaId = CreateSema( &counterSema );
//create threads
potatoThreadId = CreateThread( &potatoThread );
radishThreadId = CreateThread( &radishThread );
//check to make sure the semaphore and the threads were created
if( counterSemaId <= 0 ) {
printf( "Server sema failed to start. %i\n", counterSemaId );
return -1;
}
if( potatoThreadId <= 0 ) {
printf( "Server thread failed to start. %i\n", potatoThreadId );
return -1;
}
if( radishThreadId <= 0 ) {
printf( "Server thread failed to start. %i\n", radishThreadId );
return -1;
}
//Begin thread execution
StartThread( potatoThreadId, NULL );
StartThread( radishThreadId, NULL );
//Set the time in miliseconds to call the function RotateThreads
SetAlarm(16, RotateThreads, 0);
//Don't do anything until both flags are set
while( !POTATO_DONE && !RADISH_DONE )
SleepThread();
//clean up threads
TerminateThread( potatoThreadId );
TerminateThread( radishThreadId );
//delete threads
DeleteThread( potatoThreadId );
DeleteThread( radishThreadId );
//delete the semaphore
DeleteSema( counterSemaId );
printf("Goodbye\n");
return 0;
}
Posted: Fri May 30, 2008 4:59 am
by Lukasz
Things are getting bit complicated now and not very PS2 specific. :-)
I suggest that you make/debug your multithreaded application on your PC with pthreads/semaphores or similar and then port it over.
This way you can validate the correctness of the PS2 port on PC and also compare output for PC and PS2 version.
Posted: Mon Jun 02, 2008 6:27 pm
by EEUG
...some remarks:
- handler set by 'SetAlarm' is executed only once. If you want to execute it continuously then you can use 'iSetAlarm' kernel API from your handler to schedule its execution again;
- time interval is expressed in H-Sync's, not in milliseconds;
- pay attention at the priority of your 'main' thread. Make sure that it is higher than threads you create (i.e. priority value is lower than 0x1E). I think ps2link sets it to 64, so as soon as you start 'countPotatoes' 'main' thread blocks until 'countPotatoes' will terminate. Then 'countRadishes' starts and 'main' thread 'freezes' again. Then alarm handler will be set up, but vegetables counters are already terminated at that moment. You may try to do following:
- set the priority of your main thread to '0x1D':
Code: Select all
ChangeThreadPriority ( GetThreadId (), 0x1D );
- create/start your "vegetables counters", set alarm handlers etc.;
- lower priority of your 'main' thread:
Code: Select all
ChangeThreadPriority ( GetThreadId (), 0x1F );
At this point main thread will "freeze" and "vegetables counters" will be executed;
Posted: Mon Jul 06, 2009 8:42 am
by cosmito
I've been trying the code listed here and applying EEUG suggestions to do a modified small example where an alarm timer increments a counter besides taking care of thread switching and a thread checks it's value to tell the main thread about its value. It seems there's something wrong with it, and I'm hoping you can spot it.
My goal is to use an alarm timer to periodically do a iRotateThreadReadyQueue so all threads get a time execution slot.
This is better than do a RotateThreadReadyQueue inside every loop cycle at the source.
Code: Select all
#include <tamtypes.h>
#include <stdio.h>
#include <kernel.h>
static int counter = 0;
static int continueloop;
#define THREAD_STACK_SIZE (8 * 1024)
static u8 thread_stack[THREAD_STACK_SIZE] ALIGNED(16);
static ee_thread_t thread_thread;
static int thread_threadid;
void TheThread(void *arg);
void alarmfunction(s32 id, u16 time, void *arg)
{
counter++;
iRotateThreadReadyQueue(31);
iSetAlarm ( 625, alarmfunction, NULL );
}
int main( int argc, char **argv)
{
extern void *_gp;
// EEUG : pay attention at the priority of your 'main' thread.
// Make sure that it is higher than threads you create
//(i.e. priority value is lower than 0x1E). I think ps2link sets it to 64
// - set the priority of your main thread to '0x1D':
// - create/start your "vegetables counters", set alarm handlers etc.;
// - lower priority of your 'main' thread
ChangeThreadPriority ( GetThreadId (), 29 ); // 29 is lower than 30 ;)
continueloop = 1;
SetAlarm( 625, alarmfunction, NULL);
printf("\ncreating thread\n");
thread_thread.func = TheThread;
thread_thread.stack = thread_stack;
thread_thread.stack_size = THREAD_STACK_SIZE;
thread_thread.gp_reg = _gp;
thread_thread.initial_priority = 30;
if ((thread_threadid = CreateThread(&thread_thread)) < 0)
{
printf("CREATE THREAD ERROR!!!\n");
return -1;
}
StartThread(thread_threadid, NULL);
printf("thread started\n");
ChangeThreadPriority ( GetThreadId (), 31 );
while (continueloop == 1)
{
printf("%d\n", counter);
}
printf("terminating\n");
TerminateThread(thread_threadid);
DeleteThread(thread_threadid);
printf("\ndone\n");
return 0;
}
///
void TheThread(void *arg)
{
while (1)
{
if (counter == 50)
continueloop = 0;
//RotateThreadReadyQueue(31);
}
}
I kept printf at a minimum and not in critical zones of the code since it seems to induce thread switching.
So the above example doesn't work. I get only the "creating thread" and
"thread started" message and that's it.
I can only get it to work if I I change all priorities to same value (31 for example) and do a RotateThreadReadyQueue(31) at TheThread().
It seems the iRotateThreadReadyQueue() is not doing anything at the alarm function...
Any hints?
Posted: Tue Jul 07, 2009 7:26 pm
by EEUG
...well, as soon as you call 'ChangeThreadPriority ( GetThreadId (), 31 );' your 'main' thread will never get a chance to run beyond that code as 'TheThread' has a higher priority of 30 and at the moment of 'ChangeThreadPriority' call it will take control. Would you try 'ChangeThreadPriority ( GetThreadId (),
30 );' instead and modify your alarm handler (iRotateThreadReadyQueue(
30);)?
Note that 'printf' will most probably switch the threads also as 'main' thread will go asleep for a while (while Rpc call will complete) and 'TheThread' will take control at that moment...
Edit: Actually iRotateThreadQueue does not perform context switch at all. It just rotates a queue (an internal data structure). Some extra "assistance" is necessary to get it work as desired. Like this working example:
Code: Select all
#include <tamtypes.h>
#include <stdio.h>
#include <kernel.h>
static volatile int counter = 0;
static volatile int continueloop;
#define THREAD_STACK_SIZE (8 * 1024)
static u8 thread_stack[THREAD_STACK_SIZE] ALIGNED(16);
static ee_thread_t thread_thread;
static int thread_threadid;
static u8 disp_stack[THREAD_STACK_SIZE] ALIGNED(16);
static int disp_threadid;
void TheThread(void *arg);
void dispatcher ( void* apParam ) {
while ( 1 ) {
SleepThread ();
} /* end while */
} /* end dispatcher */
void alarmfunction(s32 id, u16 time, void *arg)
{
counter++;
iWakeupThread ( disp_threadid );
iRotateThreadReadyQueue ( 30 );
iSetAlarm ( 625, alarmfunction, NULL );
}
int main( int argc, char **argv)
{
extern void *_gp;
// EEUG : pay attention at the priority of your 'main' thread.
// Make sure that it is higher than threads you create
//(i.e. priority value is lower than 0x1E). I think ps2link sets it to 64
// - set the priority of your main thread to '0x1D':
// - create/start your "vegetables counters", set alarm handlers etc.;
// - lower priority of your 'main' thread
ChangeThreadPriority ( GetThreadId (), 29 ); // 29 is lower than 30 ;)
continueloop = 1;
SetAlarm( 625, alarmfunction, NULL);
printf("\ncreating thread\n");
thread_thread.func = dispatcher;
thread_thread.stack = disp_stack;
thread_thread.stack_size = THREAD_STACK_SIZE;
thread_thread.gp_reg = &_gp;
thread_thread.initial_priority = 0;
StartThread ( disp_threadid = CreateThread ( &thread_thread ), NULL );
thread_thread.func = TheThread;
thread_thread.stack = thread_stack;
thread_thread.stack_size = THREAD_STACK_SIZE;
thread_thread.gp_reg = &_gp;
thread_thread.initial_priority = 30;
if ((thread_threadid = CreateThread(&thread_thread)) < 0)
{
printf("CREATE THREAD ERROR!!!\n");
return -1;
}
StartThread(thread_threadid, NULL);
printf("thread started\n");
ChangeThreadPriority ( GetThreadId (), 30 );
while (continueloop == 1)
{
}
printf("terminating\n");
TerminateThread(thread_threadid);
DeleteThread(thread_threadid);
printf("\ndone\n");
return 0;
}
void TheThread(void *arg)
{
while (1)
{
if (counter > 50)
continueloop = 0;
}
}
Alternatively iRotateThreadReadyQueue can be moved into the dispatcher and replaced by RotateThreadReadyQueue.
Posted: Wed Jul 08, 2009 8:03 am
by cosmito
Wow, many thanks EEUG!
Very clever the SleepThread() + highest priority trick :)