The new-style psptoolchain script.

[ooPo]

After releasing the previous toolchain update on my site and watching some of the problems people had, I've updated it:

http://dev.oopo.net/files/psptoolchain-20070517.tar.bz2

There's been no changes to the patches, they're the same as you'd get with the current psptoolchain in the subversion repository.

Now for the why.

1) The toolchain script has grown annoying to manage.
2) There's a lot of random stuff added to the script instead of doing things like properly merging patches.
3) Checking dependencies before running is very important - adding more should be easy to do.

So, here's the new-style script:

Code: Select all

#!/bin/sh
# toolchain.sh by Dan Peori ([email protected])

 ## Create the build directory.
 mkdir -p `dirname $0`/build || { echo "ERROR: Could not create the build directory."; exit 1; }

 ## Enter the build directory.
 cd `dirname $0`/build || { echo "ERROR: Could not enter the build directory."; exit 1; }

 ## Run the depend scripts.
 for SCRIPT in `ls ../depends/*.sh | sort`; do $SCRIPT || { echo "$SCRIPT: Failed."; exit 1; } done

 ## Run the build scripts.
 for SCRIPT in `ls ../scripts/*.sh | sort`; do $SCRIPT || { echo "$SCRIPT: Failed."; exit 1; } done

All it does is create a build directory, run all the scripts in the 'depends' directory, and then all the scripts in the 'scripts' directory.

Adding a new dependency check is as simple as adding a new script in the 'depends' directory. Here's what we currently have:

Code: Select all

psptoolchain/depends/check-autoconf.sh
psptoolchain/depends/check-automake.sh
psptoolchain/depends/check-bison.sh
psptoolchain/depends/check-flex.sh
psptoolchain/depends/check-gcc.sh
psptoolchain/depends/check-make.sh
psptoolchain/depends/check-ncurses.sh
psptoolchain/depends/check-patch.sh
psptoolchain/depends/check-pspdev.sh
psptoolchain/depends/check-subversion.sh
psptoolchain/depends/check-texinfo.sh
psptoolchain/depends/check-wget.sh

Here's what one of these scripts look like:

Code: Select all

#!/bin/sh
# check-make.sh by Dan Peori ([email protected])

 ## Check for make.
 make -v 1> /dev/null || { echo "ERROR: Install make before continuing."; exit 1; }

Simple, yes? That's the point.

Build scripts are similar, but are numbered so that they get executed in the correct order:

Code: Select all

psptoolchain/scripts/001-binutils-2.16.1.sh
psptoolchain/scripts/002-gcc-4.1.0-stage1.sh
psptoolchain/scripts/003-pspsdk-stage1.sh
psptoolchain/scripts/004-newlib-1.15.0.sh
psptoolchain/scripts/005-gcc-4.1.0-stage2.sh
psptoolchain/scripts/006-pspsdk-stage2.sh
psptoolchain/scripts/007-gdb-6.4.sh
psptoolchain/scripts/008-insight-6.4.sh

Here's what one looks like on the inside:

Code: Select all

#!/bin/sh
# binutils-2.16.1.sh by Dan Peori ([email protected])

 ## Download the source code.
 if test ! -f "binutils-2.16.1.tar.bz2"; then
  wget ftp://ftp.gnu.org/pub/gnu/binutils/binutils-2.16.1.tar.bz2 || { exit 1; }
 fi

 ## Unpack the source code.
 rm -Rf binutils-2.16.1 && tar xfvj binutils-2.16.1.tar.bz2 || { exit 1; }

 ## Enter the source directory and patch the source code.
 cd binutils-2.16.1 && cat ../../patches/binutils-2.16.1-PSP.patch | patch -p1 || { exit 1; }

 ## Create and enter the build directory.
 mkdir build-psp && cd build-psp || { exit 1; }

 ## Configure the build.
 ../configure --prefix="$PSPDEV" --target="psp" --enable-install-libbfd || { exit 1; }

 ## Compile and install.
 make clean && make -j 2 && make install && make clean || { exit 1; }

If nobody objects, my plan in the near future is to create a 1.0 branch in subversion for the old script, and start clean with this script.

Questions? Comments?

[hlide]

that's a good way to modulate this script. I think there would be no objection.

[hlide]

well i failed with ncurse because I havn't /usr/lib/libncurses.so which is normal because i'm under cygwin and not under linux.

I think you toolchain need to be aware whether the host is windows or linux (or MacOX ?).

[hlide]

oh well, if a novice wants to install it, he/she would run into troubles :

Error : PSPDEV not set <== why not to propose one by default when not set ? usually /usr/local/pspdev as it is with the current psptoolchain.

I don't expect him/her to do a "export PSPDEV=/usr/local/pspdev" every time before running the toolchain.

I think you must fix those things before replacing the current psptoolchain with this one.

1) be sure it works like a charm under every (cygwin, linux, MacOS X)
2) don't let your scripts to be too petty. Whenever you have a variable not set, try to propose one by default so that people wanting just a working compiler to be able to compile a source they retrieve can do it without any knowledge about scripts

[ooPo]

well i failed with ncurse because I havn't /usr/lib/libncurses.so which is normal because i'm under cygwin and not under linux.

What does cygwin use for a termcap library? You need one in order to compile gdb. Would searching for libtermcap.so work instead?

I don't expect him/her to do a "export PSPDEV=/usr/local/pspdev" every time before running the toolchain.

I expect a user to read the documentation and do what's needed to build and use a toolchain:

Code: Select all

 ==================
  How do I use it?
 ==================

 1) Set up your environment by installing the following software:

  autoconf, automake, bison, flex, gcc, make, ncurses, patch, subversion, terminfo, wget

 2) Add the following to your login script:

  export PSPDEV=/usr/local/pspdev
  export PATH=$PATH:$PSPDEV/bin

 3) Run the toolchain script:

  ./toolchain.sh

I do agree that the error messages could be more helpful, but having the script use defaults and try to assume things will have the user end up with a toolchain that still won't work properly until they do the setup they should have done in the first place.

[J.F.]

Go for it! Looks good to me.

[hlide]

well i failed with ncurse because I havn't /usr/lib/libncurses.so which is normal because i'm under cygwin and not under linux.

What does cygwin use for a termcap library? You need one in order to compile gdb. Would searching for libtermcap.so work instead?

i do have ncurse, but libncurses.so is linux-ish ! there is no .so in cygwin (but rather .dll). I suppose you may test libncurses.dll.a which is in the same directory instead.

I don't expect him/her to do a "export PSPDEV=/usr/local/pspdev" every time before running the toolchain.

I expect a user to read the documentation and do what's needed to build and use a toolchain:

Code: Select all

 ==================
  How do I use it?
 ==================

 1) Set up your environment by installing the following software:

  autoconf, automake, bison, flex, gcc, make, ncurses, patch, subversion, terminfo, wget

 2) Add the following to your login script:

  export PSPDEV=/usr/local/pspdev
  export PATH=$PATH:$PSPDEV/bin

 3) Run the toolchain script:

  ./toolchain.sh

I do agree that the error messages could be more helpful, but having the script use defaults and try to assume things will have the user end up with a toolchain that still won't work properly until they do the setup they should have done in the first place.

oh well i didn't read your readme.txt because i was expecting to use this psptoolchain the same way i did with the current one. So, yes, I should check if my login script has the right exports.

[hlide]

anyway, by just ignoring the ncurse check, i was able to run psptoolchaintool up to the end, so I suppose configure is able to retrieve ncurse even under cygwin.

[ooPo]

i do have ncurse, but libncurses.so is linux-ish ! there is no .so in cygwin (but rather .dll). I suppose you may test libncurses.dll.a which is in the same directory instead.

You're absolutely right. Duh. :) That's an easy enough change to make.

[ooPo]

http://dev.oopo.net/files/psptoolchain-20070518.tar.bz2

Code: Select all

#!/bin/sh
# check-ncurses.sh by Dan Peori ([email protected])

 ## Check for a ncurses library.
 ls /usr/lib/libncurses.* 1> /dev/null || { echo "ERROR: Install ncurses before continuing."; exit 1; }

This should work for everyone now, correct?

[hlide]

i tried psp-gdb, psp-gdbtui and psp-insight which seem to work perfectly (psp-gdbtui should use ncurse, right ?)

so i think you can go now

[futaris]

Don't suppose the script can be modified to download the .tar.gz and .tar.bz2's to a PSP_DL environment variable. Openembedded, buildroot, etc do something similar, so we can just download everything to one directory.

[weltall]

sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)

[J.F.]

sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)

Actually, the current stable gcc is now 4.2.0. :D

[weltall]

sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)

Actually, the current stable gcc is now 4.2.0. :D

i mean regarding available gcc patches for psp :P

[hlide]

sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)

Actually, the current stable gcc is now 4.2.0. :D

i mean regarding available gcc patches for psp :P

those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.

[ooPo]

sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)

You can still grab the old toolchain and patches:

svn export svn://svn.pspdev.org/psp/tags/psptoolchain-1.0.0

Are there problems with 4.1.0? When I asked nobody seemed to know of any outstanding issues.

[Luke]

Is this only an update to the toolchain-install-script or does this update the toolchain? If so, how do I do that? Can I simply run the new script?

[weltall]

Actually, the current stable gcc is now 4.2.0. :D

i mean regarding available gcc patches for psp :P

those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.

do you mean that those problems i experienced with it were made by not defining that command line argument?

[ooPo]

Is this only an update to the toolchain-install-script or does this update the toolchain? If so, how do I do that? Can I simply run the new script?

This is an update to the install script. If you already have a toolchain installed you don't need this - you're already up to date.

This new script just makes it easier to maintain updates in the future.

[Luke]

Cool, thanks! And by the way: Great work! Really. PSP-Programming rocks! Thanks for making this possible for me!

[hlide]

those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.

do you mean that those problems i experienced with it were made by not defining that command line argument?

there is a topic somewhere covering the fact you can have wrong code because of strict aliasing and can be worse with -O3. Try to find it. This topic explains why and how to avoid those wrong code.

So yes I think your problems can be from the fact that you don't use this command line.

[weltall]

those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.

do you mean that those problems i experienced with it were made by not defining that command line argument?

there is a topic somewhere covering the fact you can have wrong code because of strict aliasing and can be worse with -O3. Try to find it. This topic explains why and how to avoid those wrong code.

So yes I think your problems can be from the fact that you don't use this command line.

ok thanks when i have some time i will build the new toolchain and try again :)

[Alek]

Err... didn't say anything xD

Installing well, good job :)

[lmame]

Hi ^_^

First of all thank you for the toolchain and this new script, I'm quite new to psp dev world and for now I was just using oslib / pspsdk / cygwin under windows and when I saw your new script I decided to give it a try under Linux :)

I managed really simply to install and run things unit the last script until the last script is called: 008-insight-6.4.sh :
When the make -j 2 is run, I got this error (one line lost in an ocean of text):

Code: Select all

/bf/toolchain/binutils-2.17/tk/unix/../generic/tk.h:96:23: X11/Xlib.h: No such file or directory

So I had to install X11 and run it again ;)

Perhaps it would be call to check also if X11 exists in one of your script, I guess it would be cool for noobs like me ;)

Anyway, now it's working :)

[hlide]

ooPo :

gcc 4.2 is out, it has less regressions than gcc 4.1. It might be interesting to use it instead, provided that the current patches are still applyable.

gcc 4.2 : [ all : 252, serious : 129 ]
gcc 4.1 : [ all : 297, serious : 173 ]

EDIT : silly of me ! I wrote 2.x instead of 4.x !!!

[ufoz]

Surely you mean 4.2?

[hlide]

arfff yes it is 4.x not 2.x 8x !!!!

[hlide]

GRRRRRRRRRRRRRRRRR !!!!

the option "-mpreferred-stack-boundary=n" is not longer valid in this version !!!! still I modified some gcc files in gcc/config/mips directory so we can change the alignment requirement for stack !? does it means you use a different mips patch !?

now some of my sources are broken because of that.

[jimparis]

Isn't relying on stack alignment a bit broken anyway? I thought even "-mpreferred-stack-boundary" is just an attempt not a guarantee. You could use memalign if alignment matters.