Streaming large PCM from the HDD

Discuss the development of software, tools, libraries and anything else that helps make ps2dev happen.

Moderators: cheriff, Herben

Post Reply
PrimeTime
Posts: 25
Joined: Thu Aug 19, 2004 7:49 pm

Streaming large PCM from the HDD

Post by PrimeTime »

Hi again,
I'm trying to stream a large PCM audio file from my hard drive.
I'm using the SjPCM library to help. My problem is, the sound output
is very choppy. I am not sure if it is coming from reading the data
from the hard drive, or transferring the data to the right and left
audio channels. Here is the code of my stream algor.

Code: Select all

samples1 = malloc(800);
scr_printf("Loading..  ");

//Just some pre-loading
fileXioRead(fd, (short*)samples1, 800);
for &#40;readIn = 0; readIn < 800; readIn++&#41;
&#123;
   left&#91;readIn&#93; = samples1&#91;readIn&#93;;
   if &#40;channels == 2&#41; right&#91;readIn&#93; = samples1&#91;2*readIn+1&#93;; else right&#91;readIn&#93; = samples1&#91;readIn&#93;;
&#125;

while &#40;1&#41;
&#123;
   buffered = SjPCM_Buffered&#40;&#41;;
   if &#40;buffered<1*800&#41; 
   &#123;
      SjPCM_Enqueue&#40;left, right, 800, 0&#41;; // avoid underrun
      fileXioRead&#40;fd, &#40;short*&#41;samples1, 800&#41;;
      for &#40;i=0; i<800; i++&#41;
      &#123;
        left&#91;i&#93; = samples2&#91;i&#93;;
        if &#40;channels == 2&#41; right&#91;i&#93; = samples2&#91;2*i+1&#93;; else right&#91;i&#93; = samples2&#91;i&#93;;
      &#125;
      wait_vsync&#40;&#41;;
   &#125;
&#125;
my audio file is the correct format for SjPCM. I've tried different buffer sizes too, ranging from 800 to 768000.
The larger buffersizes pause during rebuffering.

Any ideas?
Thanks,
Ryan
Guest

Post by Guest »

One problem I have looking at your code is that you did not include
any of your variable declarations, such as for:
samples1, samples2, left[], right[], etc...

Its hard to debug other peoples code if you can't make sure their problem
isn't in their variable declarations.

Not that it necessarily is so, I heard a similar problem discussed in
#ps2dev[efnet] recently. Maybe someone else can drop by to offer
some hints, or you could try dropping into the channel and asking.

Gorim
PrimeTime
Posts: 25
Joined: Thu Aug 19, 2004 7:49 pm

Post by PrimeTime »

sorry about that, made a few typos

Code: Select all

samples1 = malloc&#40;800&#41;; 
scr_printf&#40;"Loading..  "&#41;; 

//Just some pre-loading 
fileXioRead&#40;fd, &#40;short*&#41;samples1, 800&#41;; 
for &#40;readIn = 0; readIn < 800; readIn++&#41; 
&#123; 
   left&#91;readIn&#93; = samples1&#91;readIn&#93;; 
   if &#40;channels == 2&#41; right&#91;readIn&#93; = samples1&#91;2*readIn+1&#93;; else right&#91;readIn&#93; = samples1&#91;readIn&#93;; 
&#125; 

while &#40;1&#41; 
&#123; 
   buffered = SjPCM_Buffered&#40;&#41;; 
   if &#40;buffered<1*800&#41; 
   &#123; 
      SjPCM_Enqueue&#40;left, right, 800, 0&#41;; // avoid underrun 
      fileXioRead&#40;fd, &#40;short*&#41;samples1, 800&#41;; 
      for &#40;i=0; i<800; i++&#41; 
      &#123; 
        left&#91;i&#93; = samples1&#91;i&#93;; 
        if &#40;channels == 2&#41; right&#91;i&#93; = samples1&#91;2*i+1&#93;; else right&#91;i&#93; = samples1&#91;i&#93;; 
      &#125; 
      wait_vsync&#40;&#41;; 
   &#125; 
&#125; 
samples1 is short*
left and right are static short __attribute__((aligned (64)))[1024]
User avatar
Saotome
Posts: 182
Joined: Sat Apr 03, 2004 3:45 am

Post by Saotome »

i'm not sure if thats the cause for your choppy output, but shouldn't it be:

Code: Select all

for &#40;i=0; i<800; i++&#41; 
&#123; 
	if &#40;channels == 2&#41; left&#91;i&#93; = samples1&#91;2*i&#93;; //<= !!!
	else left&#91;i&#93; = samples1&#91;i&#93;;
	if &#40;channels == 2&#41; right&#91;i&#93; = samples1&#91;2*i+1&#93;; else right&#91;i&#93; = samples1&#91;i&#93;; 
&#125; 
?
infj
PrimeTime
Posts: 25
Joined: Thu Aug 19, 2004 7:49 pm

Post by PrimeTime »

thats incorrect code for 2 channels :D
I'm trying a mono channel right now, so i can just use the left data for both, I have re-written the algorithm, it works a little better, but once the initial buffer is empty it gets choppy.

I can't find anything wrong, but its also late at night.

Code: Select all

short *samples1 = malloc&#40;192000&#41;;
fileXioRead&#40;fd, &#40;short*&#41;samples1, 192000&#41;;
readIn = 0;
i = 0;

while &#40;1&#41;
&#123;
   left&#91;iSamples&#93; = samples1&#91;readIn&#93;;
   if &#40; &#40;++iSamples == 800&#41;&#41; //800 samples are now in left&#91;&#93;
   &#123;
      wait_vsync&#40;&#41;;
      SjPCM_Enqueue&#40;left, left, 800, 0&#41;; // queue up the 800 samples
      //read 800 more samples in the buffer location i just transferred to left&#91;&#93;
      fileXioRead&#40;fd, &#40;short*&#41;&samples1&#91;i*800&#93;, 800&#41;;
      i++;  //increment the buffer block
      iSamples = 0;  //reset the queued samples
   &#125;
   if &#40;++readIn == 192000&#41;  // we are at the end of the buffer
   &#123;
      printf&#40;"------Getting new data--------\n"&#41;;
      readIn = 0;
      i = 0;  //since we reread the buffer while we were playing, we should be able to reset the variables and transfer the new buffer
   &#125;
&#125;

J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

Reading 800 samples at a time off the HD won't work. There's no cacheing in the HD code, so it needs to get the data off the drive in non-optimal sizes (not multiples of the blocksize). The HD is incapable of randomly getting 800 samples within one video frame. Some will, and others won't leading to chop. You need to buffer at least a couple seconds ahead.

Create a thread that keeps a two-second buffer full of data which is read (at least) 8K at a time. Have another thread that waits a video frame, then feeds data to the sound.
PrimeTime
Posts: 25
Joined: Thu Aug 19, 2004 7:49 pm

Post by PrimeTime »

Ah, I figured it out.
I haven't used threads for the streaming yet, but I did get
the file to stream correctly.

My problem was that each sample is 2 bytes which is
stored in one element of a short array
I was treating the short array as it it were an
array of 1 byte elements such as a char array.
So, my output was choppy since i was transferring
only the first portion of the stream and not the second.

Ryan
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

Wow! I'm surprised. That you can pull the data off the HDD one frame at a time says quite a bit for how good the drive code is. I'd have thought for certain you'd need a threaded buffer to keep it smooth. Good to hear otherwise, but I think it'd be much better with a threaded buffer.
mrbrown
Site Admin
Posts: 1537
Joined: Sat Jan 17, 2004 11:24 am

Post by mrbrown »

J.F. wrote:Wow! I'm surprised. That you can pull the data off the HDD one frame at a time says quite a bit for how good the drive code is. I'd have thought for certain you'd need a threaded buffer to keep it smooth. Good to hear otherwise, but I think it'd be much better with a threaded buffer.
Er, it's a hard drive, with <5ms seek time and 20-25MB/s peek transfer speed (at least under PS2/Linux, it's probably more natively).

Why wouldn't it be fast enough?
"He was warned..."
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

I'd read somewhere else on the forum that the current HD code was SLOW (getting less than 1M/s transfer rates). I guess what we need is a nice test that shows how fast certain types and size of transfers are, and what their latency is.
PrimeTime
Posts: 25
Joined: Thu Aug 19, 2004 7:49 pm

Post by PrimeTime »

Well, either way, I got both the threaded and non-threaded versions
to work :)

Ryan
mrbrown
Site Admin
Posts: 1537
Joined: Sat Jan 17, 2004 11:24 am

Post by mrbrown »

J.F. wrote:I'd read somewhere else on the forum that the current HD code was SLOW (getting less than 1M/s transfer rates). I guess what we need is a nice test that shows how fast certain types and size of transfers are, and what their latency is.
Well, HDD does compete with SMAP for the same interrupt and DMA channel. If we're still using the older, GPL'd SMAP driver (are we?) then we aren't contending for DMA because that driver doesn't use any. I remember reading the post from the guy that optimized SMAP and ps2ip and he had real abysmal rates transferring to the HDD over the network, IIRC.

I guess it would be worthwhile if someone sat down and benchmarked it.
"He was warned..."
Post Reply