forums.ps2dev.org

Over the next few days I plan to start putting together ps2sdk – a collection of ps2dev projects merged into one central project. Before this is started, I need to figure out how the directory structure will be setup and what projects should be included. I’ve laid out the basic structure which is described below, and would like any suggestions people may have as to how it can be improved or what other projects can be added. If you have something to contribute then please do so now – I don’t want people bitching about things when its released :P Also if I have listed your code for inclusion in ps2sdk but you do not wish for this to happen, let me know.

Proposed ps2sdk layout:

/common/include – includes shared between EE and IOP code

/ee/doc – documentation for EE libraries
/ee/include – public include files for all EE libraries
/ee/lib – EE library files
/ee/sample – EE sample code
/ee/src/kernel – essentially libkernel from ps2lib.
/ee/src/rpc/cdfs – rpc client library for the cdfs: IRX.
/ee/src/rpc/cdvd – rpc client library for CDVDFSV in ROM. (libcdvd)
/ee/src/rpc/fileXio – rpc client library for fileXio (libfileXio)
/ee/src/rpc/keyboard – rpc client library for usb keyboard driver (libkbd)
/ee/src/rpc/memcard – rpc client library for MCSERV in ROM. (libmc)
/ee/src/rpc/mouse – rpc client library for usb mouse driver (libmouse)
/ee/src/rpc/pad – rpc client library for PADMAN and MTAPMAN in ROM. (libpad & libmtap)
/ee/src/rpc/tcpip – rpc client library for ps2ip (ps2ipc)
/ee/startup – crt0.s, linkerscript and Makefile template for EE projects.

/iop/bin – compiled IRX files end up here
/iop/doc - documentation for IOP modules
/iop/include – public include files for IOP libraries
/iop/sample – IOP sample code
/iop/src/dev9/atad – ATA driver (ps2atad)
/iop/src/dev9/dev9- DEV9 driver (ps2dev9)
/iop/src/dev9/extflash – extflash driver (extflash)
/iop/src/dev9/smap – SMAP driver (boman’s version, ps2smap2)
/iop/src/fs/devfs
/iop/src/fs/fakehost
/iop/src/fs/fileXio – fileXio rpc server
/iop/src/fs/iomanX
/iop/src/fs/netfs
/iop/src/hdd/apa – APA filesystem driver (ps2hdd)
/iop/src/hdd/pfs – PFS filesystem driver (ps2fs)
/iop/src/sound/freesd
/iop/src/sound/pcmstrm – sjpcm renamed
/iop/src/tcpip/lwip – ps2ip and lwip sources combined
/iop/src/tcpip/udptty – tty output over udp
/iop/src/usb/keyboard – usb keyboard driver (ps2kbd)
/iop/src/usb/klsi – netgear eht adapter driver (ps2klsi)
/iop/src/usb/mouse – usb mouse driver (ps2mouse)

/tools/bin – binaries of the tools
/tools/src/bin2s – converts a binary file into an asm array for linking with your code
/tools/src/elf2irx – converts an ELF file into an IRX
/tools/src/elfpack – elf packer (sjcrunch)

All EE libraries get copied to /ee/lib as part of the make process. All IOP IRX binaries get copied to /iop/bin as part of the make process. The tool executables get copied to /tools/bin as part of the make process. This should make it easy to put together a binary distribution of ps2sdk – just remove all the “src” directories and keep the rest.

That’s about it for now - let me know what you think.

sjeep wrote: /tools – binaries of the tools
/tools/src/bin2s – converts a binary file into an asm array for linking with your code
/tools/src/elf2irx – converts an ELF file into an IRX
/tools/src/elfpack – elf packer (sjcrunch)

Shouldn't there be a /tools/bin (like /iop/bin and /ee/lib) instead of just dumping the binaries in the root of tools?

IMO it might be nice to put bin2c in the tools dir too... But that's just me :)

Other than that, it look good to me :)

sjeep wrote:Over the next few days I plan to start putting together ps2sdk – a collection of ps2dev projects merged into one central project. Before this is started, I need to figure out how the directory structure will be setup and what projects should be included. I’ve laid out the basic structure which is described below, and would like any suggestions people may have as to how it can be improved or what other projects can be added. If you have something to contribute then please do so now – I don’t want people bitching about things when its released :P Also if I have listed your code for inclusion in ps2sdk but you do not wish for this to happen, let me know.

Proposed ps2sdk layout:
[SNIPPED]

All EE libraries get copied to /ee/lib as part of the make process. All IOP IRX binaries get copied to /iop/bin as part of the make process. The tool executables get copied to /tools as part of the make process. This should make it easy to put together a binary distribution of ps2sdk – just remove all the “src” directories and keep the rest.

That’s about it for now - let me know what you think.

I'm curious - what made you deviate from the directory structure provided here: http://lists.topica.com/lists/ps2dev/re ... start=2522

The original idea was to have a bunch of individually buildable modules that could be build anywhere and not dependant on the directory structure above them. If you refer to the original ps2sdk document, the "install" target installs a binary (this includes .h files) distribution somewhere else, so that the binary distribution doesn't depend on the source distribution.

Mrbrown, there are a few things I just didnt like about the "original idea":

1) IMHO, simply grouping current projects into one source tree rather than properly integrating them is a messy solution. Having "doc", "include", "lib", "sample" and "src" subdirectories for each part is also un-necessarily messy.
2) Some libraries/modules depend on header files from other libraries/modules, for example /iop/src/hdd/apa relies on headers from atad, dev9 and iomanX. IMHO it would be better to just specify the one include path when building (/iop/include) rather than several different include paths.
3) I dont see any great advantage from making everything modular in the way you have described so the user can choose which libraries to install and which to leave out. Its not like ps2sdk is huge, and its not like we are all still using 20MB hdd's - its not going to hurt if you have a bit of code in your source tree which is never used. IMHO its better to have everything nice and cleanly integrated, and have everything there for when you need it.

Apart from not having the "include", "lib", blah sub-directories in each part - the structure I havent given is basically the same as that specified in the URL you gave.

Drakonite, sorry that was a typo - I intended for tool binaries to end up in /tools/bin. Fixed.

1) This allows concurrent development of external libraries without the developer having to fuss over the internal details of ps2sdk. The two biggest hurdles that new developers face when submitting new projects to ps2dev are figuring out how to use CVS, and how to place files/directories within the project tree. Treating each library or module as a seperate "module" allows them to develop outside of ps2sdk, without interrupting their work flow or anyone else's, and submit when they're ready. It also keeps your global include/, lib/, etc. directories uncluttered.

2) This is why we use make, particularly GNU make. Of course you already need a text file or Makefile fragment that specifies each subdirectory that you want to build (dev9, smap, etc.). It's not so farfetched to automatically add each module's include/ directory to CFLAGS at build time. Matter of fact, it's as simple as doing $(addprefix). Why is it better to clutter include/ instead?

3) Re: "great advantage" see point #1. The modular structure is not really related to the install. Instead of how things are done now with the .a or .irx that remains in the source tree, now we have external directories to install to. This eliminates the need for $(PS2DEV) or $(PS2SDK) since will be able to install directly into the user's toolchain include and library path. This is where the directory structure finally gets "flattened" into single include/ and lib/ directories.

I'm not so sure what you mean with the space/unused code argument, since it's the same exact source in both situations, just placed in different directories.

I think some of the confusion we're having here is because with the old way of doing ps2lib and ps2drv, the end developer (for example someone working on a demo) and us, the library developers, were one and the same. End developers had to know how ps2lib and ps2drv physically worked in order to use them. With ps2sdk, they no longer need to know or care, and the details that we're discussing now only apply to you and I, the library developers. There's no need to oversimplify things for end developers who will never touch ps2sdk source.

The original idea was to start fresh with a design that is clean and has some thought put into it beforehand. What you've proposed is simply a concatenation of ps2lib and ps2drv, with a bit of ps2ip and ps2eth mixed in. You've also added an empty directory src/ that does absolutely nothing. If everything looks exactly the same as ps2lib and ps2drv with just a few source directories renamed, what exactly is the point of shuffling them around? Again, I'm thinking of the original design goals of ps2sdk here (which I didn't come up with, BTW, Oobles did, and all of us, including you, decided on them, and I formalized it into a document).

Another point we had discussed on IRC before (don't recall if you were around or not) was that there's no need for a common/ directory for includes or anything else. Yes, SIF shares some common routine names and structures, but that's not enough to warrant an entire common/ was what we decided.

Now you've really got me curious - I've never seen you say anything against what was originally proposed (almost a year ago) or Oobles' January ps2dev post - not on IRC or on the ML. Why the sudden change?

mrbrown, thanks for the refresher. I'm glad someone can remember the various reasons for the structure. Now that you remind me, the reasons behind having each part of the sdk as a module are good ones. The idea of being able to develop a module seperate from the sdk and then move it into the build structure is probalby worthwhile.

I must admit though, that if anyone is willing to put the time into bringing the various projects together into one build tree, I really don't mind how it ends up. :) I will however put my vote in for the modular approach of having each module seperate.

The modules do create some pain for include files from library modules. It could be somewhat elliviated by having a number of passes to build the sdk.

phase1: build libkernel, libc.
phase2: build dev9, io, etc
phase3: build ps2ip, libhdd, etc

That means that phase2 can use the ps2sdk/iop/include of modules built in phase1.. and phase3 can use all modules from phase2.

Of course, the other solution is to just include multiple include paths from various parts of the tree.

My 2c worth,
Oobles.

Mrbrown, the reason why you've never seen me say anything against what was originally proposed is that I was never really that involved in the planning for ps2sdk. The reason I have decided to finally do something about it, 1 year later, is that I am getting sick of having to keep track of several different modules and all the annoying dependencies. I just want to have a single source tree which everything is integrated into - and I dont really care how its done, as long as everything is togeather.

IMHO simply throwing ps2lib, ps2drv etc togeather would do.. however you have made a good case for taking the modular approach, and after re-thinking things it does sound like a better idea. Now we just need to figure out the best way to handle the includes etc - do we do as you've suggested and just specify multiple include/library paths when compiling the various parts of the sdk, or do we break building into multiple phases, copying the includes and library files to the install location at each phase, and just watch the order in which everything is built so that if a module relies on another module, then the module which it relies on is already installed and in the correct place?

Regarding the common directory - I think there is a need for one, and without a common directory things could get messy. There are several headers which are shared between EE and IOP - sif headers, primative data typedefs, fileio structures, fileio dev/ioctl macros, errno etc. Why duplicate all this stuff in both EE and IOP tree's?

I've just realised that the directory structre I've listed above doesnt really have a place for the IOP import headers (loadcore.h, cdvdman.h, sysmem.h etc) included with ps2drv. Where should these header files end up?

OK, I've set aside some time to help out on this... so I'd like to get things moving, before I get side tracked on something else. :)

On the topic of the common include directory. I'm a bit torn, but I tend to agree with mrbrown on this one. These are two different processors and environments. In the long run I think it will be easier to treat them as such. I could probably be easily swayed the other way though. :) One option I would accept is to keep a single source file for data structures used in RPC, etc.. and copy that header file into the two different output /iop/include and /ee/include target directory.

For the include directory during build. I'd suggest we start with just including all the different paths. If that gets painful then we can implement multiple phases.

For the IOP import headers that have no implementation. I'd put these in libkernel project.

My only current concern is putting *all* include files in a single output include directory. If someone wants certain parts of ps2sdk, but not others, it makes it difficult to seperate header files. eg. I want to use newlib instead of the minic header files. There are probably more examples of this.. Only solution I can think of is ${target}/ee/include/minic style for libraries we think could conflict.

Oobles.

It's started! I've just concentrated on moving source files into their relevant directories. Got a bit side tracked, so didn't get very far. However, I like what I see so far! I think it will work out nicely.

Will do more on the weekend.. feel free to continue on. :)
Oobles.

For the people on the ps2cvs mailing list, it will be no surprise that ps2sdk is finally coming together. Over the weekend I've performed with some help from Lukasz the pulling together of ps2lib, ps2drv, libhdd, ps2ip & ps2hid into one project.

The initial aim of ps2sdk was to bring all the projects together into a single build tree with a better structure. To achieve that I've just put all the files together without much thought for changing names, or fixing some architectural things like common files.

The status at the moment is that the build, clean and release targets all *work*. However there are still some things to do before it will be useable. These include:

* Create release targets for projects that only have include directories.
Includes iop/kernel, iop/usb/usbd [NOW DONE]
* Put files from ps2lib/ee/libkernel for sifcmd, sifrpc in ps2sdk/ee/kernel
and ensure they build and release. They were omitted at the start for
reasons I forget now. :) [NOW DONE]
* Move README files into doc directory and make release move these.
* Work out structure of samples directories and make release move
these to sample structure.
* Make release copy base LICENSE, README, etc files. [NOW DONE]

After that is done.. we need to work out what else is missing.. test if it actually works. :) and then decide if that is version 1.0, or if we want to do any restructuring before letting people use it.

Ideally, we should do any major renaming now, before people get used to it. Personally, I'm quite happy with current names and some of the legacy. I'm sure others may have differing opinions?

Other decisions to be made are.. What do we keep out of, or put in ps2sdk? I'd say..

* AFL compatible projects in only. Maybe have a ps2sdk/gpl directory so
that GPL components can sit in the same tree.
* Any projects releated to hardware incorporated. Todo USB, eyetoy, headset, etc.
* Any base graphics utilities. GS Registers, DMA, etc.
* Possibly a single simple graphics toolset so people can get started quickly. Keep engines seperate as they are many and varied, and will grow to be as big as ps2sdk.

Now is a good time to have thoughts and comments, before the ps2sdk README sets all this in stone.. or CVS. :)

Oobles.

Put in code for hardware access/control. This means GS/GIF registers, dma control functions. No need to get all fancy with graphics code - just provide a way to access the hardware and let the user write his own code around that. Sample code can use these functions to give the user a starting point.

Maybe a matrix math lib of some sort? Something with EE-only and VU0 macro mode functions, maybe switchable at runtime? A nice way to get some quick objects on the screen, and it would make nice sample code for VU coding as well.

Is there a standard VU harness that can be put in the tools dir? That may be handy...

Keep up the busy work. Your momentum will hopefully help ps2sdk gain enough support to keep the project rolling along. :)

ooPo wrote:Put in code for hardware access/control. This means GS/GIF registers, dma control functions. No need to get all fancy with graphics code - just provide a way to access the hardware and let the user write his own code around that. Sample code can use these functions to give the user a starting point.

This is actually a personal project of mine that I am just starting.

I am looking to port a small RTOS to EE and/or IOP. The idea is
that even from within a command line shell one can examine and
affect registers, dmacs, and memory. Or code a C/ASM program,
and load it, or any
combination. I am looking to do it in as easy and natural way as
possible, hence my project (and why I am not choosing Linux).

Gorim

ooPo wrote: Maybe a matrix math lib of some sort? Something with EE-only and VU0 macro mode functions, maybe switchable at runtime? A nice way to get some quick objects on the screen, and it would make nice sample code for VU coding as well.

Is there a standard VU harness that can be put in the tools dir? That may be handy...

PBDemoLib contains all that already.

Adresd's vu harness is the only "standard" harness we got ( and since its a clone of the harness used for the vu compos, no new one should be needed ).

Well then, it'll be easy to add them. :)

What will be the learning curve for the ps2sdk. And after a newbie has downloaded and install the sdk, where will they begin? How will they start?

What is stopping most of the people interested in developing homebrew for the ps2, in my opinion, is that they don't know how to do anything. There is only Dreamtime's tutorials for beginners, but there isn't anything beyond that.

A perfect metaphor for the situation. "Give a man a fish and he can eat for a day, teach a man to fish and you feed him for a lifetime."

ps2sdk is only the libraries. It simplifies the current setup by including all the important libraries in one place. Currently, you really need about two or three libraries (ps2lib, ps2drv, etc) to do anything.

The other thing which is currently being worked out is documentation and samples. The exact layout is still to be organised. I was planning to do some work this weekend, but instead I'm travelling interstate. So.. if anyone wants to help out on ps2sdk.. this is my current idea for creates the samples directory structure.

Currently lukasz has setup a basic structure for samples which creates a directory on release in $(PS2SDK)/ee/samples. It takes some premade makefiles and sets them up in this directory.

I think we should change this to have a $(PS2SDKSRC)/samples directory which has the example make files. This means they won't be confused with the project makefiles in the ee directory. The release target would then copy these to $(PS2SDK)/samples. It means that users can find the samples in one place.

The samples directory structure can then be $(PS2SDK)/samples/hello/hello.c etc. This assumes that the sample projects have a flat directory structure, unlike the full src, obj, etc structure the projects have.

In each projects samples directory eg $(PS2SDKSRC)/ee/libc/samples there can either be a single example or multiple directories with multiple samples. These only need a single release target on makefiles which will copy them to a selected target location. So ee/libc/samples/hello/Makefile will specifiy a target for samples as 'hello/'. which will be put in $(PS2SDK)/samples/hello.

In theory this will give a flexible directory structure for samples. It will also allow us to combine IOP and EE samples to a common directory structure.. eg samples/tcpip/ee-echo and samples/tcpip/iop-echo. From a users point of view it should be easy to navigate, and ensures the samples are kept seperate from the ee/lib ee/include etc directory structure.


That's my thoughts anyway. :) Feel free to disagree and go and implement someting completely different. I'll try and do some work on it in the next couple of weeks if no-one else does.

Oobles.

ps2sdk is now coming together very nicely.

Sjeep has added a DNS resolver and modified the http file system to use it.
The release target now works nicely.
Samples are now released to the samples directory.


Things still to do:

Work out how to handle documentation files from each sub project and where to put them on release.

Work out a good way of fixing the samples/Makefile so that all subdirectories can be recursively entered and all samples built.

Confirm that all source files are the latest from original projects. I thought I did a CVS update before copying files, however, someone suggested that I missed recent updates.

Write more project documentation, documenting how to add projects,etc.


That's all I can think of before version 1.0? Anything else people would like to see?

Oobles.

Third reply to myself. :)

I have done the tedious task of ensuring all the files in ps2sdk are the same as the original project they came from. Now that's done, I don't want to do it again. So the original projects in CVS are now read-only. These are, ps2lib, ps2drv, ps2ip, libhdd, ps2hid, lwip.

ps2sdk is in a good enough state to replace those libraries anyway, so hopefully there won't be too many complaints. :)

Oobles.

Just two minor additions:
ee/Rules.make seems to need a path to ee/libc/include
and malloc.h was missing from the same folder. I copied the one from ps2lib, seems to work :-)

Forgive a stupid noob, I know you guys are thinking way beyond this kinda thing, but setup dev. environ. today D/l'd the ps2sdk from cvs and there are a few issues with example code. I have fixed 'em coz they are obvious things left from directory structure changes, should I post em back to cvs or just leave it to the expert's (dont want to jump in the way of a juggernaut) this is just int the tcpip sample, host paths to irx's (ps2smap.irx is missing completely). Admittedly I know Fsk all and after my last beating on irc I didn't want to say too much.

Any changes are always welcome. CVS write access needs to be setup, and if you send me a message directly I can do that. The other option is to just upload a diff file which someone can add into CVS for you.

And yes.. IRC channel can be a bit of a trial by fire. However, they're all mostly harmless. :)

Oobles.

Oobles wrote: And yes.. IRC channel can be a bit of a trial by fire. However, they're all mostly harmless. :)

Oobles.

Yeah, once you learn to ignore some of the more boisterous brits,
things will be fine. I think it might be that streak of scottishness in
some of them...

>:)

GOrim

My experience when asking for urls for documentation in irc was interesting, though I have seen most of the pictures before :)

just change the SifLoadModule paths in echo.c sample code to
"../../../iop/irx/****.irx"
though ps2smap.irx is none existant in the sdk is this correct?.
(sorry cannot work out how to post binaries here(stupid noob))

"Yeah, once you learn to ignore some of the more boisterous brits,

That's very Canadian of you.