Go Back   Steve's Digicams Forums > Digital SLR and Interchangeable Lens Cameras > Olympus dSLR

Reply
 
Thread Tools Search this Thread
Old Dec 3, 2010, 12:43 PM   #1
Moderator
 
Join Date: Aug 2004
Posts: 8,529
Default requesting a real world test

So, the dxomark tests are out on the new E-5. One of the things the tests seem to suggest is the E-5s reported ISO values are "overstated" compared to say E-30 when shooting RAW. So here's a question for anyone upgraded from E-30 to E-5. Can you run a couple low-light high ISO tests in controlled lighting to validate? In other words at ISO 3200, set both cameras to manual exposure and set same aperture and shutter speed and take some test shots. See if ISO 3200 on E-5 really produces the same exposure as 3200 on E-30, when shooting RAW. The test shots I've seen so far from the camera at 3200 look great. But, that get's tempered if in a given lighting situation you're really say 2/3 stop below and have to use ISO 5000 instead of 3200 to get the same exposure given same aperture and shutter speed.

Some of the suggestion is it is the RAW > JPEG engine that really improved and thus how Oly handles noise. Which is great for people that will use JPEGs. For people shooting RAW they wouldn't see the same gains.

I put more stock in real people using cameras than a lot of tests. So for people who have upgraded I'll be interested to see if their test confirms or refutes what dxomark is reporting.
JohnG is offline   Reply With Quote
Sponsored Links
Old Dec 3, 2010, 12:57 PM   #2
Administrator
 
Join Date: Jun 2003
Location: Savannah, GA (USA)
Posts: 22,378
Default

I think you mean the E-3 versus E-30. The E-3 appears to be very close to what you set. However, the new E-5 and E-30 both appear to be less sensitive than set according to DxOMark's measurements (at the raw level).

http://dxomark.com/index.php/en/Camera-Sensor/Compare-sensors/%28appareil1%29/220|0/%28appareil2%29/574|0/%28appareil3%29/682|0/%28onglet%29/0/%28brand%29/Olympus/%28brand2%29/Olympus/%28brand3%29/Olympus

But, I'd take some of their tests with a "grain of salt", as there are many factors that can influence the results you can expect from a converted raw file (including things like lighting temperature, with different sensors having different response curves so that one may require increasing one channel over another for correct White Balance in the final demosaiced image).
JimC is offline   Reply With Quote
Old Dec 3, 2010, 1:20 PM   #3
Moderator
 
Join Date: Aug 2004
Posts: 8,529
Default

Jim,

Thanks - now it makes sense. Yes, I agree things are always more complicated than basic tests would indicate. So it would seem to suggest that if a user did a controlled test on the 3 cameras, the E-5 and E-30 would have about the same exposure at 3200 while the E-3 would be 1/2 stop brighter.

For interest sake I compared three of the new cameras people are discussing and looked at the ISO 3200 plot for each. K-5 measured at 2864, D7000 measured at 2627 and E-5 measured at 1904 while the E-3 measured at 3052. So, according to DXO marks simple tests the E-3 was the most "honest" in it's reporting of ISO.

I guess that's pretty similar to what companies do with lenses - sigma 120-300 2.8 is really about 280mm (heck we'll round to 300) and other lenses are really more like f3.2 (but we'll call it 2.8).
JohnG is offline   Reply With Quote
Old Dec 3, 2010, 1:28 PM   #4
Moderator
 
Join Date: Aug 2004
Posts: 8,529
Default

by the way - lest people think I'm picking on Oly, DXO Mark reports indicate my camera - Canon 1dIII is doing the same type of playing fast and loose with ISO values.
JohnG is offline   Reply With Quote
Old Dec 3, 2010, 2:47 PM   #5
Senior Member
 
dlpin's Avatar
 
Join Date: Jul 2008
Posts: 143
Default

Just one addendum if anyone is trying to do this test:

Do not use olympus' software to convert raw files, as olympus software has specific processes for each camera. That is, olympus studio and olympus master will convert the e-30 raw file the same way that the e-30 in body processing would do it, and the same for the e-5, and as such you don't get the actual raw differences using that software.

To get at how the raw actually looks before any camera specific profile is applied, you need to use some third party software like rawtherapee.
dlpin is offline   Reply With Quote
Old Dec 3, 2010, 2:52 PM   #6
Senior Member
 
HarjTT's Avatar
 
Join Date: Aug 2004
Posts: 4,534
Default

Hi John

Hey interesting request and if I had an e5 or access to one its something that I would be more than happy to do. I'm planning to pop into the city again to see if any of the stores in london have the e5 so I can test it with the 35-100 and if they do I will do a test to compare my E3 to the E5 at ISO 3200. I think its a valid request to see how well the E5 does in the real world and see how that compares to say DxO's tests. Greg's gonna be your best bet right now as he has both cam's and the SHG lenses to do the comparisons with. I'd really like to see whether the Olys claim that the E5 shines with the HG/SHG glass esp at high ISO's.

Cheers

Harj

HarjTT is offline   Reply With Quote
Old Dec 3, 2010, 5:46 PM   #7
Senior Member
 
Join Date: Feb 2007
Location: Wisconsin
Posts: 1,241
Default

Quote:
Originally Posted by JohnG View Post
So, the dxomark tests are out on the new E-5. One of the things the tests seem to suggest is the E-5s reported ISO values are "overstated" compared to say E-30 when shooting RAW. So here's a question for anyone upgraded from E-30 to E-5. Can you run a couple low-light high ISO tests in controlled lighting to validate? In other words at ISO 3200, set both cameras to manual exposure and set same aperture and shutter speed and take some test shots. See if ISO 3200 on E-5 really produces the same exposure as 3200 on E-30, when shooting RAW. The test shots I've seen so far from the camera at 3200 look great. But, that get's tempered if in a given lighting situation you're really say 2/3 stop below and have to use ISO 5000 instead of 3200 to get the same exposure given same aperture and shutter speed.

Some of the suggestion is it is the RAW > JPEG engine that really improved and thus how Oly handles noise. Which is great for people that will use JPEGs. For people shooting RAW they wouldn't see the same gains.

I put more stock in real people using cameras than a lot of tests. So for people who have upgraded I'll be interested to see if their test confirms or refutes what dxomark is reporting.
I'm not much for tests like that. Maybe I can do something tomorrow.

Now in real world terms, comparing the noise of the E-3 and the E-5 is more than just "how much" but the "quality of" the noise. The texture of the noise the E-5 produces is much finer. It does make noise at higher ISOs when viewing at 100%, and it tends to improve quickly when downsized or printed.
__________________
Greg

https://dogsportphoto.smugmug.com/
fldspringer is offline   Reply With Quote
Old Dec 4, 2010, 9:40 AM   #8
Moderator
 
Join Date: Aug 2004
Posts: 8,529
Default

Greg - it isn't the noise so much as the exposure. I.E. does E3.ISO3200 = E5.ISO3200

So if you keep shutter and aperture same on both cameras in low light (where exposure differences are easier to spot) are the two cameras producing two different exposures at 3200?
JohnG is offline   Reply With Quote
Old Dec 4, 2010, 11:26 AM   #9
Senior Member
 
Join Date: Feb 2007
Location: Wisconsin
Posts: 1,241
Default

Quote:
Originally Posted by JohnG View Post
Greg - it isn't the noise so much as the exposure. I.E. does E3.ISO3200 = E5.ISO3200

So if you keep shutter and aperture same on both cameras in low light (where exposure differences are easier to spot) are the two cameras producing two different exposures at 3200?
Well, this is crude and not done to your desires. It gets complicated in that Adobe doesn't support the E-5 files and while it can be fooled into processing them, it must be done on a command line hex editor and is a hassle I've done only once and hope I never need to again. Hence I did use JPEG for the E-5.

I was more interested in seeing what the cameras chose for exposure, so I left it in aperture mode. ISO was set to 3200 for both cameras (I think its the first time I ever set the E-3 to that high of ISO) The lens was the 50 f2 macro set at f2 and the only lights were two compact floresecents (26w screw in replacement for incandecents) at a distance of 20 foot behind, one from the left and one from the right. I hand held and this isn't scientific, but lighting was certainly stable.

The E-5 chose 1/25s, and the E-3 choose 1/30s, which would support the ISO fudging, but only by a fifth of a stop. Then the computer told a different story.

E-3 1/30s ISO3200


E-5 1/25s ISO3200


It would seem the difference is more than 1/5 stop and it favors the E-5, not the expected result for the fudged ISO results DXO shows.

Next is noise results. Is the E-5 just pressing the brightness up at the expense of noise.

The E-3 at about 100% (now I know why I don't use it at ISO3200)



And the E-5


Bottom line is it doesn't really mean anything. Its a poorly done comparison. Both are under exposed and the white balance was fooled by the compact florescents on both cameras. They aren't even the same exposure. The way I see it, it doesn't support the notion that the cheating on ISOs is simply under-exposing images.

I did try to bring up the exposure of the E-3 file to try to get the same brightless, but it utterly fell apart.

I don't know if additional playing is really worth it for me.
__________________
Greg

https://dogsportphoto.smugmug.com/

Last edited by fldspringer; Dec 4, 2010 at 11:28 AM.
fldspringer is offline   Reply With Quote
Old Dec 4, 2010, 1:59 PM   #10
Administrator
 
Join Date: Jun 2003
Location: Savannah, GA (USA)
Posts: 22,378
Default

There's no real good way to tell using a JPEG image from the E-5, since the camera is probably compensating for any RAW level underexposure during processing.

Also, most raw converters are going to apply some processing to the image, even if you set them not to.

Now, one good way to tell would be to use dcraw.c from command line. Then, you could do something like this to get a linear conversion using the camera's WB (substituting the correct filename for testfile.orf).

dcraw -w -W -T -4 testfile.orf

Imaging Resource has some downloadable raw file samples from the E-3 and E-5. For example, here's a link to a file from the E-5 at ISO 3200 with NR set to off, shot at ISO 3200, f/8, 1/400 second:

http://www.imaging-resource.com/PROD...NR_OFF.ORF.HTM

More here:

http://www.imaging-resource.com/PRODS/E5/E5A7.HTM

Here's where you can get the same shot from the E-3, in the same lighting, using the same settings (ISO 3200, f/8, 1/400 second)

http://www.imaging-resource.com/PROD...TEROFF.ORF.HTM

More here:

http://www.imaging-resource.com/PRODS/E3/E3THMB.HTM

You can get dcraw.c here (already compiled). See the download links on the right side of the page. The latest version already supports the E-5.

http://www.insflug.org/raw/Downloads/

Here's a direct link to a version that should work on Win 7 and Vista:

http://www.insflug.org/raw/file_download/8/dcrawMS.exe

Then, just do this to convert the E5 raw file using a linear conversion (substituting dcrawMS for dcraw if you're using that download)

dcraw -w -W -T -4 E5hSLI3200_NR_OFF.ORF

and this to convert the E3 raw file using a linear conversion:

dcraw -w -W -T -4 E3hSL_FILTEROFF.ORF

Or, do like I just did (using the MS Vista/Win 7 specific version) and convert both of them at the same time (you can specify more than one file on the same line), like this:

dcrawMS -w -W -T -4 E5hSLI3200_NR_OFF.ORF E3hSL_FILTEROFF.ORF

Then, just compare the output files (.tiff files). ;-)

Note that the dcraw .exe file will need to be in the same folder as the images unless you have it in a folder that's already in the OS search paths.

Also, you'll need the version specific to Vista or Win 7 if using those operating systems. Basically, Microsoft really messed things up for open source compilers starting with Vista (using artificial memory limitations on memory some compiled apps could use). So, you need a version compiled with something like MS Visual Studio to prevent out of memory errors with dcraw. Now, there are ways to tweak the registry to get around it (bypassing the artificial memory limitations Microsoft imposed on apps compiled with some open source compilers starting with Vista) so you can use a non MS compiler and have the app work. But, unless you want to make changes to the dcraw.c source code on a regular basis, it's just as easy to grab a copy already compiled somewhere (as in the links above), and the latest builds already support the E-5.

Now, I just converted and compared those samples, and to my eyes, there is a significant difference between them (in favor of the E-3, as the E-5 shot is much darker). Without measuring the patches (just "eyeballing" the output files), I'd say the E-3 shot is at least 1/3 stop to 1/2 stop brighter.

Try it yourself and see what you get. I'd prefer not to post those images here (as they're copyrighted images from imaging-resource.com).
JimC is offline   Reply With Quote
 
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off



All times are GMT -5. The time now is 1:51 AM.