Playstation Eye

[molster]

Any update on drivers from anyone?

[ps2devman]

I think I will buy one now that I've seen the desktop VR demonstration!

I wonder if we will be given a little bone... Official sdk owners will be given the code for free to locate player wearing IR glasses, according to the author of the demo (Sony dev).

But actually I think there should be way to identify a head location by analyzing the raw picture, even without the lens filtering out all but IR.

I think about a FPS where you can actually dodge opponent's bullets while in slow motion or aim more precisely an hiding target by moving aside.

I think a flood of this kind of games will happen soon...

[ps2devman]

Got my eye!
When I have free time and if I manage to code something usable, I will try to post a tutorial for noobs about eye usage on fedora 7...

Documentation says :
Video capture - 640x480
Video format - Uncompressed or JPEG (interesting!)
Frame rate - 640x480 at 60fps / 320x240 at 120fps

I will focus on the Desktop VR usage (camera detects your head position and game modifies 3D view of software)...

[pitrz]

ps2devman:
Camshift algorithm from opencv package can do just what you need to achieve this functionality. OpenCV is already ported to ps3 and some parts are also spe-optimized.

[ps2devman]

Thanks pitrz for the hint.

Demo from Sony at GDC show :
http://multiplayerblog.mtv.com/2008/02/ ... atch-this/

Sony is fast... The video can't be seen from uk, canada & japan, but here is what it shows :

Display is the same as the Desktop VR demo using Playstation Eye and IR glasses. It's right in middle of GDC show and you no longer need glasses this time.

But the screen sometimes shows what is tracked on your face :
your global head
your nose
your eyes

It seems the nose part is important. Probably helps to locate precisely your eyes.

Just need to redo the same for public free homebrew now...

[hxr]

Hi,

I am the admin and main programmer for macam,
<http://sourceforge.net/projects/webcam-osx/>

Thanks to your efforts here, a driver is working.
The key is to queue up 3 async read calls; just how
it is done in isoch drivers (gspca). While one completion
routine is returning, there isd another buffer waiting to
be filled.

Also, USB hard-drives seem to cause interference
(incomplete frames) even when connected to a different
controller!

Thanks, especially to jimparis!
hxr

[ps2devman]

Thx a lots!
Got a mac recently because of iphone sdk.
Will try and will see if a port can be made towards ps3 linux (no ETA).

[snowsquirrel]

Are there any updates to this?

I need a webcam, and am tempted to get the ps3 eye. I would use it with my computer (debian), and the ps3.

Should I just get a standard webcam (that works with PS3-XMB), or is there a chance that this camera will become 'supported' atleast in linux.

Thanks,
~S

[jimparis]

It'll happen eventually... my example code shows enough to display a basic video feed. But nobody's written a real driver yet for Linux.

[swordman]

Hi all!
I bought recently the eye and I'm testing for using it in a openCV project in Kubuntu.
First of all thanks to jimparis for the usefull sources.
With them I can read the image data from the eye, but the brightness isn't always the same... it seems to change rapidly. Any Ideas?

PS: The framerate isn't perfect, but that was already known ;)

[ps2devman]

Ok, I could spend some free time with my macbook and macam.
It worked perfectly. The video is insanely smooth and of good quality!
The max setting of macam is 30 fps 640x480 it seems.

My intel based macbook version is OS-X 10.5.4.
The XCode compiler is the one coming with iPhone SDK final version.

I just grabbed today cvs macam source set and compiled.
(project setting was 10.4 | Debug | All.)
I just got an error for localized strings not in UTF-8 format.
Just need to right click in text and click Get Info.
Then you switch to UTF-8 and click Convert button (not Interpret!).

You obtain an application that automatically detects Playstation Eye.
Click the top left button (Play/Pause) and real time vide capture appears!

I didn't have to install the current official 0.9.2 dmg. Source is enough.

Now I will slooowly study source and learn how to build similar code on other platforms... (no ETA, USB2 required).

Thanks a lots hxr!

[majortrips]

It'll happen eventually... my example code shows enough to display a basic video feed. But nobody's written a real driver yet for Linux.

I have video streaming working from an ov534.ko module I wrote using your code and the vivi V4L2 driver.

Though I am currently having an issue with the color. Very green. The driver sets up the pixel format as YUYV, but I am not so certain this YUYV is the same as the one V4L is expecting. Y,U,V,Y 4:2:2?

Will figure out a spot to upload the driver to.

[majortrips]

Though I am currently having an issue with the color. Very green. The driver sets up the pixel format as YUYV, but I am not so certain this YUYV is the same as the one V4L is expecting. Y,U,V,Y 4:2:2?

Found another peculiar issue, the bulk read comes up 4 bytes short every time (one pixel?). Only receiving 614396, so the framefill thread ends up appending the leading 4 bytes from the next frame onto the tail of the frame and subsequently discards the remainder.

I cut-back the receive size on the USB bulk transfer to match the total image and as such, no longer dropping frames except when userland is late on the pickup from the frame buffer.

[someone]

Thanks for the updates

just glad to hear someone's spending some time on it : )

[majortrips]

Have integrated a YUYV to RGB/BGR decoder into the driver (borrowed the YUV_TO_RGB_BY_THE_BOOK() interface from the usbvision.c/ibmcam.c drivers) , so color works now.
The driver is performing asynchronous streaming using a kernel thread to continually build a stack of frame buffers on the side and pass them off to the Linux videobuf library.

Have been attempting to use various applications against the driver to flesh out the code.

• Ekiga: appears to crop the output image of RGB4 output to a 320x480 portion from the top-left of the frame. Not certain this is an Ekiga bug or a bug in the driver's advertising of the format capabilities, this cropping does not occur when using YUYV output (but the color is off then). For the 1/4th of the image that is displayed, frame rates look good and color looks good when in RGB4 mode, in YUYV mode the output is neon green and banded with blank lines. Ekiga claims it can handle YUYV, but not real certain it's support is right. That or it is expecting a variation on the YUYV pixel format.

• XawTV: appears to open the V4L interface properly and enumerates the format capabilities perfectly, but then performs some sort of read that prevents the frame buffers from being released and thus the driver performs no output. Still need to look into XawTV to understand how they expect to grab the stream.

• Camorama: Fails to even open the device. Camorama fails to open the Vivi driver properly as well, and I heavily borrowed the core code from the Vivi driver, so this is not that surprising, but it is also one more thing that needs to be figured out.

• camE: Fails to open the device, similar issue as Camorama.

I had hoped that basing the code off the Vivi V4L2 test driver would have avoided these sort of issues, but apparently its a diverse world.

Outside of cleaning up application interaction, it would be very nice to figure out the command sequences for changing the output resolution, enabling/disabling compression, and manipulating the sensor.

[majortrips]

Talked with Jim a bit in email, and read through much of the V4L2 API documents. Based on both inputs I gutted the built in YUYV decoder.

I am up to v0.3 of the sources and have xawtv working at 640x480, full color. Simply had to start xawtv with the right -geometry for the image and all worked perfectly.

I believe Ekiga's handling of the camera is a result of something broken in Ekiga. The version of Ekiga I have installed wants to treat the YUYV progressive output as something else entirely. I also validated that Ekiga does tend to try to crop the camera when the output frame size is not settable (tested against the Vivi camera driver).

Will see about understanding why camE and Camorama still do not function as expected and then find some other applications that break.

Will hopefully have a place to upload these to soon.

[majortrips]

Neither camE nor Camorama support YUYV pixel formats. Kicking them out of my test list.

[majortrips]

I added a very simple YUYV to YUV420 trans-coder to the driver after looking through the sources of a number of projects. Yes the V4L group would like user land to handle this sort of conversion, but most of the user land code I have managed to look at throws errors at the user about the driver being deficient when the user land app doesn't support the output format of the camera. Luckily the code is very small and does a single pass on the frame to do the conversion.

I suppose I should forward port the code and see about adding support for the latest in V4L2 calls, as there isn't much else to be done. The camera pretty much functions for all the V4L2 applications I have to throw at it.

[majortrips]

I have submitted the driver for inclusion into the V4L source tree.

http://marc.info/?l=linux-video&m=121886284726264&w=2

[jimparis]

Nice work Majortrips. I'm still in the middle of moving so I haven't gotten a chance to test this yet, but it's really great to see this driver.

[majortrips]

Nice work Majortrips. I'm still in the middle of moving so I haven't gotten a chance to test this yet, but it's really great to see this driver.

There are some cleanups I found I need to do. Unplugging the cam while streaming has the potential to soft-lock the system, and I don't have support for power management yet. Working on cleaning all of that up, testing it, and submitting the new version for inclusion. Once the core infrastructure of the driver is taken care of more work can be done on enhancing basic configurability of the sensor and bridge.

There is still the minor issue about where the driver will officially live in the end. Be it stand-alone or intigrated in with gspca. To that I am fundementally impartial really, so long as the code is solid and the camera works.

[ao2]

Hi,

while waiting for an updated linux driver, I would like to collect here some other resources about Playstation Eye driver development.

Here are some Windows drivers:
http://nuigroup.com/forums/viewthread/2921/

they say that there could be two versions of the camera, one with a OV534 bridge and one with OV538.

and some photos of the camera internals:
http://nuigroup.com/forums/viewthread/2953/

And I found that there is also a driver for NetBSD:
http://www.invisible.ca/space/netbsd-pr ... tation-eye
which uses the same reg writes found by jimparis (with no credit to him).

Regards,
Antonio

[nbpengo]

Hey folks --

I'm the author of the NetBSD driver. The device driver is now merged with the rest of the system:

http://cvsweb.netbsd.org/cgi-bin/cvsweb ... sb/pseye.c

Here's some additional information that I've discovered, but not yet been able to do much useful with:

SCCB reg 0x29: appears to be the desired width / 4
SCCB reg 0x2c: appears to be the desired height / 2
SCCB reg 0x65: not entirely sure what it is, but based on other OmniVision drivers my guess is that it is to be set to 0x20 for 640x480, and 0x2f for any other mode.

Some more guesses:

YUYV mode:
SCCB reg 0x11: set to 0x01
MJPEG mode(?):
SCCB reg 0x11: set to 0x00
SCCB reg 0x1e: set to 0x04

Writing SCCB address to 0x60 three times should be unnecessary in the init sequence.

Most of this are guesses based on experimentation and the 7720Rev2.set file.

[nbpengo]

Here's a link to the .set file I was talking about:

http://www.invisible.ca/~jmcneill/outgoing/7720Rev2.set

If you look at the CameraSetting lines you get a bunch of magic registers, in the format: reg, value, mask.

So I take something like:

Code: Select all

HKR,%7720Combo640X480%, Tag, 0x10001, 0x64048024
HKR,%7720Combo640X480%,  CameraSetting, 1,\
12,80,ff,3d,03,ff,17,22,ff,18,a4,ff,19,07,ff,1a,f0,ff,32,00,ff,29,a0,ff,2c,f0,ff,2a,00,ff,\
11,03,ff,42,7f,ff,4d,09,ff,63,e0,ff,64,ff,ff,65,20,ff,66,00,ff,67,48,ff,13,f0,ff,0d,41,ff,\
0f,c5,ff,14,11,ff,22,3f,ff,23,07,ff,24,40,ff,25,30,ff,26,a1,ff,2b,00,ff,6b,aa,ff,13,ff,ff,\
90,05,ff,91,01,ff,92,03,ff,93,00,ff,94,b0,ff,95,9d,ff,96,13,ff,97,16,ff,98,7b,ff,99,91,ff,\
9a,1e,ff,9b,08,ff,9c,20,ff,9e,81,ff,a6,04,ff,7e,0c,ff,7f,16,ff,80,2a,ff,81,4e,ff,82,61,ff,\
83,6f,ff,84,7b,ff,85,86,ff,86,8e,ff,87,97,ff,88,a4,ff,89,af,ff,8a,c5,ff,8b,d7,ff,8c,e8,ff,\
8d,20,ff

Tag tells us that this mode is 640x480 24bpp. Now, lets diff it with another, lets say 320x240:

Code: Select all

HKR,%7720Combo320X240%, Tag, 0x10001, 0x32024024
HKR,%7720Combo320X240%, CameraSetting, 1,\ 
12,80,ff,3d,03,ff,17,22,ff,18,a4,ff,19,07,ff,1a,f0,ff,32,00,ff,29,50,ff,2c,78,ff,2a,00,ff,\
11,03,ff,42,7f,ff,4d,09,ff,63,e0,ff,64,ff,ff,65,2f,ff,66,00,ff,67,48,ff,13,f0,ff,0d,41,ff,\
0f,c5,ff,14,11,ff,22,3f,ff,23,07,ff,24,40,ff,25,30,ff,26,a1,ff,2b,00,ff,6b,aa,ff,13,ff,ff,\
90,05,ff,91,01,ff,92,03,ff,93,00,ff,94,b0,ff,95,9d,ff,96,13,ff,97,16,ff,98,7b,ff,99,91,ff,\
9a,1e,ff,9b,08,ff,9c,20,ff,9e,81,ff,a6,04,ff,7e,0c,ff,7f,16,ff,80,2a,ff,81,4e,ff,82,61,ff,\
83,6f,ff,84,7b,ff,85,86,ff,86,8e,ff,87,97,ff,88,a4,ff,89,af,ff,8a,c5,ff,8b,d7,ff,8c,e8,ff,\
8d,20,ff

Now for the diff:

Code: Select all

--- 640x480x24	2008-09-17 09:12:52.000000000 -0400
+++ 320x240x24	2008-09-17 09:12:32.000000000 -0400
@@ -1,7 +1,7 @@
-HKR,%7720Combo640X480%, Tag, 0x10001, 0x64048024
-HKR,%7720Combo640X480%,  CameraSetting, 1,\
-12,80,ff,3d,03,ff,17,22,ff,18,a4,ff,19,07,ff,1a,f0,ff,32,00,ff,29,a0,ff,2c,f0,ff,2a,00,ff,\
-11,03,ff,42,7f,ff,4d,09,ff,63,e0,ff,64,ff,ff,65,20,ff,66,00,ff,67,48,ff,13,f0,ff,0d,41,ff,\
+HKR,%7720Combo320X240%, Tag, 0x10001, 0x32024024
+HKR,%7720Combo320X240%, CameraSetting, 1,\ 
+12,80,ff,3d,03,ff,17,22,ff,18,a4,ff,19,07,ff,1a,f0,ff,32,00,ff,29,50,ff,2c,78,ff,2a,00,ff,\
+11,03,ff,42,7f,ff,4d,09,ff,63,e0,ff,64,ff,ff,65,2f,ff,66,00,ff,67,48,ff,13,f0,ff,0d,41,ff,\
 0f,c5,ff,14,11,ff,22,3f,ff,23,07,ff,24,40,ff,25,30,ff,26,a1,ff,2b,00,ff,6b,aa,ff,13,ff,ff,\
 90,05,ff,91,01,ff,92,03,ff,93,00,ff,94,b0,ff,95,9d,ff,96,13,ff,97,16,ff,98,7b,ff,99,91,ff,\
 9a,1e,ff,9b,08,ff,9c,20,ff,9e,81,ff,a6,04,ff,7e,0c,ff,7f,16,ff,80,2a,ff,81,4e,ff,82,61,ff,\

The interesting thing here is that for the registers that differ going from 640x480 to 320x240 on the 7720Rev2 -- the original register values match those in jimparis's USB sniff, so maybe we'll get lucky and be able to derive more about the camera this way. This is how I made the assumptions on which registers were for width, height, YUYV/MJPEG, etc.

Hope this helps, and I'll share more as I dig deeper.

[ao2]

Hi nbpengo,

thanks for this additional info, it will be useful.
And I appreciated you promptly added a note to credit the source of the info on which your driver is based, well done.

For now I'll use jimparis' test app and try replacing the result of your guesses.
If I came to anything useful we can integrate back the changes in linux and netbsd drivers.

Regards,
ao2

[nbpengo]

One thing I'd like to figure out is how to handle short bulk xfers. There appear to be 16-byte control packets in the stream when bandwidth is limited followed by a partial frame.

There appear to be two cases where short reads happen:

* 16-byte control packets: I have no idea how to decode these yet
* larger but still partial frames: these can be worked with

If you ignore control packets and assume that the partial frames are the lower part of the image, you can get away with it. At one point I had a hack in the code to reuse the top of the previous frame if I encountered a partial frame, but that had a few problems: required me to keep two buffers (which is > 1.2MB!) allocated, and if conditions were really bad you would notice the image trailing off at the top.

[nbpengo]

I believe Ekiga's handling of the camera is a result of something broken in Ekiga. The version of Ekiga I have installed wants to treat the YUYV progressive output as something else entirely. I also validated that Ekiga does tend to try to crop the camera when the output frame size is not settable (tested against the Vivi camera driver).

FYI, we have the upcoming ekiga development snapshot (3.0 will be released in a week or two) working on NetBSD with pseye. I saw the same issues as you with 2.0.x; one workaround IIRC was to tell Ekiga to open the camera at 640x480.

We don't do any format conversion in the kernel, but with the latest ekiga snapshots libv4l2 is used which handles format conversions and scaling automatically.

ucview, mplayer, xawtv, vlc, cheese (via gstreamer), tvtime are also working for us.

To improve performance with bulk transfers, instead of starting the transfer when the user requests a read I would suggest creating a kernel thread to continually perform transfers and copying the results out into a ring buffer. This has nearly resolved all short transfer issues for me, apart from under heavy load when the kthread is starved.

[ao2]

Hey folks --

Here's some additional information that I've discovered, but not yet been able to do much useful with:

SCCB reg 0x29: appears to be the desired width / 4
SCCB reg 0x2c: appears to be the desired height / 2
SCCB reg 0x65: not entirely sure what it is, but based on other OmniVision drivers my guess is that it is to be set to 0x20 for 640x480, and 0x2f for any other mode.

Confirmed, with the following changes to jimparis test app I can capture at 320x240, much faster than before. Register 0x65 can be for some kind of zoom/sampling factor, it should be 0x20 for 640x480, but can be either 0x20 or 0x2f for 320x240: with 0x2f you get the whole picture, with 0x20 you get the picture zoomed in (and capture is slower, so it must drive the sensor sampling factor).

Code: Select all

Index: eye.c
===================================================================
--- eye.c	(revisione 3255)
+++ eye.c	(copia locale)
@@ -297,9 +297,9 @@
 	ov_write(0x19,0x07);
 	ov_write(0x1a,0xf0);
 	ov_write(0x32,0x00);
-	ov_write(0x29,0xa0);
-	ov_write(0x2c,0xf0);
-	ov_write(0x65,0x20);
+	ov_write(0x29,0x50); /* REG 0x29 (WIDTH)  - VALUES 0xa0 = 640, 0x50 = 320 */
+	ov_write(0x2c,0x78); /* REG 0x2c (HEIGHT) - VALUES 0xf0 = 320, 0x78 = 240 */
+	ov_write(0x65,0x2f); /* REG 0x65 (a ZOOM factor?)  - VALUES 0x20 or 0x2f */
 	ov_write(0x11,0x01);
 	ov_write(0x42,0x7f);
 	ov_write(0x63,0xe0);
@@ -416,7 +416,7 @@
 
 void eye_dump_image(int skip_frames)
 {
-#define LEN (640 * 480 * 2)
+#define LEN (320 * 240 * 4)
 	uint8_t *buf;
 	int ret = 0;
 	int frame = 0;
@@ -434,8 +434,12 @@
 	info("\ndone\n");
 
 	/* convert yuyv -> rgb rgb */
-	printf("P3\n640\n480\n255\n");
+	printf("P3\n320\n240\n255\n");
 	for &#40;i = 0; i < LEN; i += 4&#41; &#123;
+		if &#40;i>0 && &#40;&#40;i&#41; % &#40;320 * 2&#41;&#41; == 0&#41;
+		&#123;
+			i += 320 * 2;
+		&#125;
 		int y1 = buf&#91;i + 0&#93;;
 		int u  = buf&#91;i + 1&#93;;
 		int y2 = buf&#91;i + 2&#93;;

Please note that I have to skip even rows, is this normal? Where can I find such kind of information about image formats and pixelformat conversions?

Regards,
ao2

[nbpengo]

Please note that I have to skip even rows, is this normal? Where can I find such kind of information about image formats and pixelformat conversions?

If you skip odd rows do you get an image as well? How big are your bulk xfers?

[ao2]

Please note that I have to skip even rows, is this normal? Where can I find such kind of information about image formats and pixelformat conversions?

If you skip odd rows do you get an image as well? How big are your bulk xfers?

Yes, I get an image, but then every row has an uniform color, it looks like this color is same color of previous (skipped) row's last pixes, or somethig very near to it.

I in the test app I have

Code: Select all

#define LIBUSB_BULK_SIZE 16384

BTW that 7720Rev2.set is quite useful as a reference, I found (trivial indeed), how to write the sensor model:

Code: Select all

info("Sensor Model: ov%02x%02x\n",  ov_read(0x0a), ov_read(0x0b));

Code: Select all

Sensor Model: ov7721